Let's Encrypt Generation/Renew failure - /.freepbx-known/?

I have this FreePBX 15 sever to play around with. I have not touched it in over a month, and a visit the web interface today and notice the SSL certificate is not valid.

Checking Certificate Manager, I see the certificate expired back on January 6. I try to manually renew it, but fails.

It presents…

/.freepbx-known/3822eb3393279885ef039392cc669e39’ - 404 Not Found Not Found The requested URL /.freepbx-known/3822eb3393279885ef039392cc669e39 was not found on this server.

Looking in /var/www/html/.freepbx-known/ and sure enough not there.

I delete the expire certificated and try again, and getting similar error.

In addition, I also tried some systems updates, and reboot, in addition to fwconsole chown and still same.

It currently has certificate manager module version 15.0.37 on it.

I am just know nothing about what is going on behind the scenes of that .freepbx-known folder, and see nothing online. Delete everything in there?

Anyone know what’s going on?

Try it with the edge module and post the output:

fwconsole ma downloadinstall certman --edge

It likely won’t make a difference, but is a little more verbose.

I’d like to see the full un-obfuscated output, pm it to me if not comfortable posting to the forum.

It did suggest I enable Responsive LetsEncrypt Rules, but made no difference, and spit this out.

Self test error: Pest_NotFound - 404 Not Found Not Found The requested URL /.freepbx-known/dea1e2ca1cb6ffd7d6018eb16f45204d was not found on this server.

PM sent.

Hello,

I’m having the same issue.

1 Like

@briang70, please update and post as requested above. Also go ahead and post results as requested below.

@GeekBoy:
Please post output of below:

echo -e "----TESTFILE ONE----\n\n" > /var/www/html/.freepbx-known/testfile1
echo -e "----TESTFILE TWO----\n\n" > /var/www/html/.freepbx-known/testfile2
echo -e "----TESTFILE THREE----\n\n" > /var/www/html/.freepbx-known/testfile3
ls -Al /var/www/html
ls -Al /var/www/html/.freepbx-known
curl http://127.0.0.1/.freepbx-known/testfile1
curl http://your.fqdn.tld/.freepbx-known/testfile2
curl http://yourpublicip/.freepbx-known/testfile3
echo ""

What I’ve usually seen with 404’s is the a gateway firewall is forwarding to the wrong server, but it looks like your in a cloud VPS, so that is doubtful.

It looks like the token file is not being written, which would be very odd.

Hi, can you tell me the command to get the desire Output?

Also I have a ticket open with PBXact Sangoma Support no response yet.

Thanks!

fwconsole ma downloadinstall certman --edge
fwconsole cert --updateall --force
ls -Al /var/www/html

total 24
drwxrwxr-x. 10 asterisk asterisk 4096 Oct  8 21:33 admin
drwxrwxr-x   2 asterisk asterisk   24 Oct  9 16:17 default
drwxrwxr-x.  2 asterisk asterisk   59 Oct  8 22:06 digium_phones
drwxrwxr-x   2 asterisk asterisk 4096 Jan 28 21:35 .freepbx-known
-rw-rw-r--.  1 asterisk asterisk   16 Jan 28 05:47 .htaccess
-rw-rw-r--.  1 asterisk asterisk  453 Jan 28 05:47 index.php
drwxrwxr-x.  2 asterisk asterisk 4096 Jan 28 05:49 restapps
-rw-rw-r--.  1 asterisk asterisk  361 Jan 28 05:47 robots.txt
lrwxrwxrwx.  1 asterisk asterisk   38 Oct  8 17:36 ucp -> /var/www/html/admin/modules/ucp/htdocs
lrwxrwxrwx.  1 asterisk asterisk   47 Oct  8 17:36 wcb.php -> /var/www/html/admin/modules/webcallback/wcb.php
drwxrwxr-x   3 asterisk asterisk   28 Oct  8 22:02 .well-known

ls -Al /var/www/html/.freepbx-known

-rw-rw-r-- 1 asterisk asterisk 25 Jan 28 21:35 testfile
-rw-rw-r-- 1 asterisk asterisk 23 Jan 28 21:35 testfile1
-rw-rw-r-- 1 asterisk asterisk 23 Jan 28 21:35 testfile2

curl http://127.0.0.1/.freepbx-known/testfile1
----TESTFILE ONE----

All the others reporting 404

I assume the x.x.x.116 address in the PM is the correct public address for the box?

Looks like http requests hitting that x.x.x.116 address are going somewhere else. They aren’t being blocked because the 404 is being returned by whatever server they are hitting.

Have you done any custom config to apache? It’s possible it’s something in the apache config, but wouldn’t be anything generated by FreePBX.

@GeekBoy:

Try this. Temporarily enable http access:

iptables -I INPUT -p tcp --dport 80 -j ACCEPT

Then try to access one of the test files from another machine:

curl -vL http://your.fqdn.tld/.freepbx-known/testfile1
curl -vL http://x.x.x.116/.freepbx-known/testfile1

Worth trying curl verbose (-v) from the FreePBX box as well. The headers might provide some clue, particularly the “Server” header.

Lastly, be sure to remove the temp iptables rule:

iptables -D INPUT -p tcp --dport 80 -j ACCEPT

I have done nothing to Apache at all.

And looking at the access log, it is hitting the server

[28/Jan/2021:22:23:56 +0000] “GET /.freepbx-known/testfile2 HTTP/1.1” 404 222 “-” “curl/7.29.0”

Did that and still same result

Really odd, the file is there and with OK rights/ownership. No clue why apache would be returning 404. Not sure how that could be a certman module issue.

I doubt it will help, but I’d be curious to see /etc/httpd/conf.d/schmoozecom.conf.

Also, would be curious…

What happens with accessing testfile2, using 127.0.0.1 as the address?

Same as tesfile1

This was at the bottom of the file, along with similar others with ports 81-84 and 8080 listed

<VirtualHost *:80>
Alias /.well-known /var/www/html/.well-known
Alias /.freepbx-known /var/www/html/.freepbx-known
RewriteEngine on
RewriteRule ^/.(well-known|freepbx-known)/ - [H=text/plain,L]
RewriteRule (^.|/.) - [F]
DocumentRoot /invalid/folder/name

That’s expected template code when the LE Service is enabled. It’s missing some backslashes in the rewrite rules, but I’m assuming the forum software absorbed them as escapes. Even so, I changed my file to match and still can’t reproduce the behavior.

Seems the forum manipulated the paste. Here is again

  <VirtualHost *:80>
      Alias /.well-known /var/www/html/.well-known
      Alias /.freepbx-known /var/www/html/.freepbx-known
      RewriteEngine on
      RewriteRule ^/\.(well-known|freepbx-known)/ - [H=text/plain,L]
      RewriteRule (^\.|/\.) - [F]
      DocumentRoot /invalid/folder/name
    </VirtualHost>

I don’t know why accessing the file at 127.0.0.1 is successful, but accessing at the public IP gives a 404 error. The only thing that comes to mind is they are being processed by different virtual hosts.

Just to make sure there isn’t some stray apache config, look at the output of:

# parameter is upper case
httpd -S

There shouldn’t be any IPs in the output .

There should only be a single port 80 reference on a line like:

*:80                   your.host.name (/etc/httpd/conf.d/schmoozecom.conf:115)
1 Like

Yeah, that was it.

I guess I was working on it, thinking I was on a different server. I should not have been doing any customization on this one like this.

Thanks a lot for the help!

Trust me, I’ve been there.

Just glad to know where the problem was. Some folks would have just gone dark and left us hanging.

1 Like