Hi all - I have recently started having a problem renewing a cert on my Windows Server Essentials 2016 box. I know there is a current issue with new validation servers and GeoIP blocking, but I don’t think this is the issue - I’m using a TZ350 and have GeoIP blocking on, but I did try turning it off and still no joy.
I believe that somehow something happened to how IIS is handling extensionless URLs?
I can hit files in the C:\inetpub\wwwroot\.well-known\acme-challenge\ directory from outside via http://hostname.com/.well-known/acme-challenge/filename.ext but only if they have an extension - i.e. http://hostname.com/.well-known/acme-challenge/configcheck.txt works, but http://hostname.com/.well-known/acme-challenge/configcheck does not and throws a 404, even though both files are present. Any thoughts or pointers about how to fix this? Many thanks for all and any responses!
By default the app will try to host it’s own HTTP challenge listener in front of IIS (using http.sys), so normally the request won’t even reach IIS.
Occasionally things can happen that stop the http challenge server being effective, updates to AV (heuristics that block http listeners etc), or simply the machine needs a reboot. Make sure you’re definitely on the latest version of the app.
If the app is having to fallback to using IIS you will see requests in your IIS site log for /.well-known/acme-challenge and that’s when the fallback config has to be right. By default the app cycles through a collection of preset web.config configs to try to sense which one works for you, to get that to reset, delete the web.config file in the acme-challenge folder and run Test again (or run your Request Certificate again), but ideally you don’t want the request to touch IIS at all.
To properly diagnose http validation though, we’d need the specific error message as a 403 is very different to a timeout etc.
A timeout is universally always a firewall, other security feature blocking http requests, or a misconfigured IP address/routing. A 403 is the request passing through and being denied by IIS, a 404 is the request passing through and being not found by IIS.
Hi - thanks for your reply. This is the log after a renewal attempt just now - it’s showing a 403 error.
2024-04-21 21:56:14.478 -07:00 [INF] ---- Beginning Request [Default Web Site] ----
2024-04-21 21:56:14.478 -07:00 [INF] Certify/6.0.15.0 (Windows; Microsoft Windows NT 10.0.14393.0)
2024-04-21 21:56:14.478 -07:00 [INF] Beginning certificate request process: Default Web Site using ACME provider Anvil
2024-04-21 21:56:14.478 -07:00 [INF] The selected Certificate Authority is: Let's Encrypt
2024-04-21 21:56:14.478 -07:00 [INF] Requested identifiers to include on certificate: remote.domainname.com [dns]
2024-04-21 21:56:15.319 -07:00 [INF] Created ACME Order: https://acme-v02.api.letsencrypt.org/acme/order/126471156/263041585997
2024-04-21 21:56:15.866 -07:00 [INF] Got http-01 challenge https://acme-v02.api.letsencrypt.org/acme/chall-v3/341507431707/FS0Zpg
2024-04-21 21:56:15.993 -07:00 [INF] Got dns-01 challenge https://acme-v02.api.letsencrypt.org/acme/chall-v3/341507431707/obNKkw
2024-04-21 21:56:23.132 -07:00 [INF] Http Challenge Server process available.
2024-04-21 21:56:23.132 -07:00 [INF] Preparing automated challenge responses for: remote.domainname.com [dns]
2024-04-21 21:56:23.132 -07:00 [INF] Preparing challenge response for the issuing Certificate Authority to check at: http://remote.domainname.com/.well-known/acme-challenge/pUxihK0Bmm63AHpAYKxLlhfDSeVfqps1jjoCzxjbsW4 with content pUxihK0Bmm63AHpAYKxLlhfDSeVfqps1jjoCzxjbsW4.KQhgeAGalYrQLoDQcVlGqRmmsvjWm3S_EHMepOTrnzg
2024-04-21 21:56:23.132 -07:00 [INF] If the challenge response file is not accessible at this exact URL the validation will fail and a certificate will not be issued.
2024-04-21 21:56:23.144 -07:00 [INF] Using website path C:\inetpub\wwwroot
2024-04-21 21:56:23.145 -07:00 [INF] Checking URL is accessible: http://remote.domainname.com/.well-known/acme-challenge/pUxihK0Bmm63AHpAYKxLlhfDSeVfqps1jjoCzxjbsW4 [proxyAPI: True, timeout: 5000ms]
2024-04-21 21:56:23.361 -07:00 [INF] URL is accessible. Check passed.
2024-04-21 21:56:23.361 -07:00 [INF] Resuming certificate request using CA: Let's Encrypt
2024-04-21 21:56:23.361 -07:00 [INF] Attempting challenge response validation for: remote.domainname.com [dns]
2024-04-21 21:56:23.361 -07:00 [INF] [Progress] Checking automated challenge response for: remote.domainname.com [dns]
2024-04-21 21:56:23.361 -07:00 [INF] Submitting challenge for validation: remote.domainname.com [dns] http://remote.domainname.com/.well-known/acme-challenge/pUxihK0Bmm63AHpAYKxLlhfDSeVfqps1jjoCzxjbsW4
2024-04-21 21:56:28.742 -07:00 [ERR] [Progress] Validation failed: remote.domainname.com [dns]
Response from Certificate Authority: During secondary validation: 76.53.148.90: Invalid response from http://remote.domainname.com/.well-known/acme-challenge/pUxihK0Bmm63AHpAYKxLlhfDSeVfqps1jjoCzxjbsW4: 403 [Forbidden :: urn:ietf:params:acme:error:unauthorized]
2024-04-21 21:56:28.748 -07:00 [ERR] Validation of the required challenges did not complete successfully. Validation failed: remote.domainname.com [dns]
Response from Certificate Authority: During secondary validation: 76.53.148.90: Invalid response from http://remote.domainname.com/.well-known/acme-challenge/pUxihK0Bmm63AHpAYKxLlhfDSeVfqps1jjoCzxjbsW4: 403 [Forbidden :: urn:ietf:params:acme:error:unauthorized]
2024-04-21 21:56:28.754 -07:00 [INF] Performing Post-Request (Deployment) Tasks..
2024-04-21 21:56:28.755 -07:00 [INF] Task [Deploy to RDP Gateway Service] :: Task is enabled but will not run because primary request unsuccessful.
2024-04-21 21:56:28.755 -07:00 [ERR] Deploy to RDP Gateway Service :: Task is enabled but will not run because primary request unsuccessful.
Please let me know if I can provide any other information! Per previous post, I get a 404 when going to the https://remote.domainname.com/.well-known/acme-challenge/configcheck in a browser from outside the network, but having made a copy of the file called configcheck.txt, I can see that.
I think I see the problem, your Geo blocking is actually presenting the 403 itself, so it’s not IIS.
HTTP validation by Let’s Encrypt happens from multiple geographic locations (US, Europe, Switerland, Singapore etc). If you want to geo block I would suggest blocking very specific countries, alternatively you would need to switch to DNS domain validation instead: DNS Validation (dns-01) | Certify The Web Docs
Well, when you’re right, you’re right. Sorry to waste your time. I could swear I tested after turning off GeoIP blocking on the SonicWall and still failed, but I must not have done it correctly. I did try it again and this time it seems to have solved the problem. I guess the extensionless thing was a red herring. I’m blocking just the obvious ones now and it still works, before I was blocking by default and allowing what we needed. It would be nice if the Let’s Encrypt people would tell us what ranges they come from, but I guess there must be some good reason they don’t.
thanks again!
“As soon as we started programming, we found to our surprise that it wasn’t as easy to get programs right as we had thought. Debugging had to be discovered. I can remember the exact instant when I realized that a large part of my life from then on was going to be spent in finding my own mistakes.”
– Maurice Wilkes: “On Debugging”
1 Like
