Let's Encrypt certificate error Pest_NotFound - 404: Nothing to do

@jerrm Thanks for your reply.

If DNS and firewall forwarding/loopback are properly configured, it should “just work.”

www.fqdn.com is working for remote IP phones in numerous locations across the internet and with other LE certs on my network, so I don’t think DNS is the problem.

There are three rules in pfSense (for the WordPress LE cert, the pfSense, and for the FreePBX LE cert) to forward TCP port 80 for LE certs. identical except for the destination NAT IP on the LAN. All are normally disabled (i.e., port 80 is closed). When I need to update the LE cert, I open up port 80 for that specific rule. The WordPress LE cert “just works” each and every time with certbot. The FreePBX LE certs not so much.

I am surprised that this is an issue because with literally millions of FreePBX installs, I expect there are more than a few who are using ACME clients, and there also must be lots of these folks with expertise in how to make this work. Right now this is my number one complaint with FreePBX. Otherwise I’m quite happy.

I don’t know much about pfSense, but suspect that your issue relates to an interaction between your special rules and hairpinning. WordPress works fine because only LE (from outside) is accessing www.fqdn.com:80 when renewing.

However, FreePBX does a pretest to first confirm that it can access www.fqdn.com:80 and I am guessing that pfSense is not handling that correctly. You can confirm or refute this hypothesis by doing a curl or wget from a PBX shell prompt to
http://www.fqdn.com/.well-known/acme-challenge/<TOKEN>

If this is your issue and you can’t easily fix it in pfSense, try adding an entry in /etc/hosts to map www.fqdn.com to 127.0.0.1

It’s a 100% certainty the self test failure is a DNS issue.

The DNS problems can be overcome with firewall configuration, but I generally consider nat loopback a kludgy, lazy hack and never the proper fix.

The other LE clients don’t use the Lescript.php library that FreePBX does. It’s a bad client, but its what we have for now. Certbot and other clients don’t have the self-test nonsense of Lescript.php.

You can take two minutes to edit the pbx hosts file to prove/disprove the fact, or continue circling around the problem.

@Stewart1 Thanks, I always appreciate your help.

Adding an entry in /etc/hosts is easy enough:

$ cat /etc/hosts
127.0.0.1 freepbx15vb localhost localhost.localdomain localhost4
127.0.0.1 www.fqdn.com
::1 freepbx15vb localhost localhost6
$

For all tests below, port 80/TCP was open and forwarded to FreePBX.

Note I am running the commands below on my FreePBX server at 172.16.0.175 SUCCESS:

$ pwd
/var/www/html/.well-known/acme-challenge
$ cat test.txt
Hello, hello. This is a test.
$ curl fqdn.com - This website is for sale! - fqdn Resources and Information.
Hello, hello. This is a test.
$

I have remotely logged in to a friend’s Windows 10 computer at another physical location 11.22.138.142 FAIL:

C:\Users\Angel>curl fqdn.com - This website is for sale! - fqdn Resources and Information.
curl: (7) Failed to connect to www.fqdn.com port 80: Timed out
C:\Users\Angel>

From pfSense on the local LAN at 172.16.0.1 FAIL:

# curl fqdn.com - This website is for sale! - fqdn Resources and Information.
<html>
<head>404 Not Found
<body>
<center><h1>404 Not Found
<hr>nginx
</body>
</html>
#

From a Mac Pro on my local LAN at 172.16.0.196, SUCCESS:

$ curl fqdn.com - This website is for sale! - fqdn Resources and Information.
Hello, hello. This is a test.
$

@jerrm Thanks.

When you say this is a DNS issue, what do you mean exactly? When I go to this website:

Network Tools: DNS,IP,Email

and type in my domain name, i.e., www.fqdn.com, the correct public IP address is displayed. I can ping my domain name from an external computer as well as a computer on the LAN. nslookup from an external computer shows as:

$ nslookup

www.fqdn.com
Server: 11.22.171.114
Address: 11.22.171.114#53

Non-authoritative answer:
Name: www.fqdn.com
Address: 11.22.15.72

I assume this was run after editing the hosts file?

From my prior posts:

@jerrm

I assume this was run after editing the hosts file?
Yes, after the hosts file was edited as follows:
$ cat /etc/hosts
127.0.0.1 freepbx15vb localhost localhost.localdomain localhost4
127.0.0.1 www.fqdn.com
::1 freepbx15vb localhost localhost6
$

@jerrm You wrote:

From the pbx, “www.fqdn.com” must resolve to the internal IP of the pbx.
This command is run on the PBX 172.16.0.175:
$ ip a

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 08:00:27:97:2f:36 brd ff:ff:ff:ff:ff:ff
inet 172.16.0.175/24 brd 172.16.0.255 scope global dynamic eth0
valid_lft 6768sec preferred_lft 6768sec
inet6 fe80::a00:27ff:fe97:2f36/64 scope link
valid_lft forever preferred_lft forever

$
After editing, the /etc/hosts file on the PBX:
$ cat /etc/hosts
127.0.0.1 freepbx15vb localhost localhost.localdomain localhost4
172.16.0.175 www.fqdn.com
::1 freepbx15vb localhost localhost6
$ curl fqdn.com - This website is for sale! - fqdn Resources and Information.
Hello, hello. This is a test.
$

For all tests below, port 80/TCP was open and forwarded to FreePBX 172.16.0.175:

On pfSense 172.16.0.1:

# curl http://www.hawkmcduck.com/.well-known/acme-challenge/test.txt
<html>
<head><title>404 Not Found
<body>
<center><h1>404 Not Found
<hr>nginx
</body>
</html>
#

On a remote computer running Windows 10:
C:\Users\Angel>nslookup
Default Server: node-2x7r9n24twqrzrke3a0v8lzp.ipv6.telus.net
Address: 2001:588:ff09:10a::66
> www.fqdn.com
Server: node-2x7jr9n24twqrzrke3a0v8lzp.ipv6.telus.net
Address: 2001:588:ff09:10a::66
Non-authoritative answer:
Name: www.fqdn.com
Address: 11.22.15.72
> quit
C:\Users\Angel>curl http://www.hawkmcduck.com/.well-known/acme-challenge/test.txt
curl: (7) Failed to connect to www.hawkmcduck.com port 80: Timed out
C:\Users\Angel>

When I attempt to update the LE www.fqdn.com cert, it’s the same, that is, in the FreePBX GUI it says, “Nothing to do, no changes made.”

Use the cli --updateall command.

The GUI doesn;t force an update if no changes to the certinfo.

@jerrm Many thanks for your perseverance and patience with this issue.
$ sudo fwconsole cert --updateall --force
Forced update enabled !!!
Processing: www.fqdn.com, Local IP: 172.16.0.175, Public IP: 11.22.15.72
Self test: trying http://www.fqdn.com/.freepbx-known/a30e8e7801dd3980de9b68a89e8ed4f6
Self test: received a30e8e7801dd3980de9b68a89e8ed4f6
Getting list of URLs for API
Requesting new nonce for client communication
Account already registered. Continuing.
Sending registration to letsencrypt server
Sending signed request to https://acme-v02.api.letsencrypt.org/acme/new-acct
Account: https://acme-v02.api.letsencrypt.org/acme/acct/79154396
Starting certificate generation process for domains
Requesting challenge for www.fqdn.com
Sending signed request to https://acme-v02.api.letsencrypt.org/acme/new-order
Sending signed request to https://acme-v02.api.letsencrypt.org/acme/authz-v3/11872356336
Got challenge token for www.fqdn.com
Token for www.fqdn.com saved at /var/www/html/.well-known/acme-challenge/e3ccSjvRZubxwi2HH68FJbHtEL0Fj9LNDMgfbEJWp4k and should be available at http://www.fqdn.com/.well-known/acme-challenge/e3ccSjvRZubxwi2HH68FJbHtEL0Fj9LNDMgfbEJWp4k
Sending request to challenge
Sending signed request to https://acme-v02.api.letsencrypt.org/acme/chall-v3/11872356336/_Fs37Q
Verification pending, sleeping 1s
Sending signed request to https://acme-v02.api.letsencrypt.org/acme/chall-v3/11872356336/_Fs37Q
Verification ended with status: valid
Sending signed request to https://acme-v02.api.letsencrypt.org/acme/finalize/79154396/8694593597
Sending signed request to https://acme-v02.api.letsencrypt.org/acme/cert/0349e7e586829e30906568608dc2387d94b9
Got certificate! YAY!
Saving fullchain.pem
Saving cert.pem
Saving chain.pem
Done !!§§!
Processing: www.anotherfqdn.com, Local IP: 11.22.15.72, Public IP: 11.22.15.72
Self test: trying http://www.anotherfqdn.com/.freepbx-known/976940aeb8276238e9489997bc53c280
Self test: received 976940aeb8276238e9489997bc53c280

Getting list of URLs for API
Requesting new nonce for client communication
Account already registered. Continuing.
Sending registration to letsencrypt server
Sending signed request to https://acme-v02.api.letsencrypt.org/acme/new-acct
Account: https://acme-v02.api.letsencrypt.org/acme/acct/79154396
Starting certificate generation process for domains
Requesting challenge for www.anotherfqdn.com
Sending signed request to https://acme-v02.api.letsencrypt.org/acme/new-order
Sending signed request to https://acme-v02.api.letsencrypt.org/acme/authz-v3/11872369065
Got challenge token for www.anotherfqdn.com
Token for www.anotherfqdn.com saved at /var/www/html/.well-known/acme-challenge/RHkYiF0Ll0fX9EV0ndhF4e4v7Y0nvyZ0_-F9NzTF1nU and should be available at http://www.anotherfqdn.com/.well-known/acme-challenge/RHkYiF0Ll0fX9EV0ndhF4e4v7Y0nvyZ0_-F9NzTF1nU
Sending request to challenge
Sending signed request to https://acme-v02.api.letsencrypt.org/acme/chall-v3/11872369065/4i2-UA
Verification pending, sleeping 1s
Sending signed request to https://acme-v02.api.letsencrypt.org/acme/chall-v3/11872369065/4i2-UA
Verification ended with status: valid
Sending signed request to https://acme-v02.api.letsencrypt.org/acme/finalize/79154396/8694603693
Sending signed request to https://acme-v02.api.letsencrypt.org/acme/cert/0433c4280baa46adabd3c4636492b3ee76ea
Got certificate! YAY!
Saving fullchain.pem
Saving cert.pem
Saving chain.pem
Done !!§§!

Certificate named “default” is going to expire in less than a month. Please update this certificate in Certificate Manager
Successfully updated certificate named "www.fqdn.com"
Successfully updated certificate named "www.anotherfqdn.com"
$

Certificate named “default” is going to expire in less than a month. Please update this certificate in Certificate Manager
Successfully updated certificate named “www.fqdn.com
Successfully updated certificate named “www.anotherfqdn.com

The log above states two certificates www.fqdn.com and www.anotherfqdn.com were both updated successfully. Just above those two lines in the log it says:

Certificate named “default” is going to expire in less than a month. Please update this certificate in Certificate Manager

  1. In the Certificate Manager menu in the FreePBX GUI the www.fqdn.com LE cert has a green checkmark as the default cert, and the Issued Certificate Details says the cert is valid for 89 days.
  2. The www.anotherfqdn.com LE cert is also valid for 89 days
  3. There is a self-signed certificate named “default” which is valid until 02/13/2028.

What cert is referred to as going to expire in less than a month?

default, the self signed one, delete it. It’s not useful

OK, will do. Thanks.

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.