HSTS TLDs failing the challenge for new certificates

Question
1

Is there an issue with obtaining certificates for TLDs that require https (eg .dev)?

For a first time request, the challenge is sent to the http protocol, but it can’t reach it because these domains are automatically redirected to https, and thus fail becuase there is no SSL certificate for this (yet) …

Kind of a catch-22 going on here… or am I barking up the wrong tree in wondering why it’s failing?

2

Hi, if you redirect all requests to https you must have an https website binding configured with at least an invalid certificate e.g. there has to be a TLS listener which will respond to port 443 - there’s likely some other problem - try your domain using https://letsdebug.net

HSTS tells the browser that after the first visit it should stick to using https but the Certificate Authority that’s trying to validate your domain (e.g. Let’s Encrypt) will still try to use http and then if it has to it will follow a redirection to https. It sounds to me like you don’t have an http listener on port 80 at all or you are redirecting before it reaches your server (e.g. with a front end proxy like Cloudflare).

If you really want to redirect to https before having https working (!) then I’d suggest manually setting up your https binding. assuming you’re using IIS this would be under Bindings for your site:

  • select https, IP (All unassigned), port 443, set the hostname to your domain, Require SNI, select any certificate just to get TLS working.
  • This will give you https for your domain but it will be invalid in a standard browser because the certificate names won’t match.

If you can’t get http validation to work you can consider DNS validation if your provider supports DNS automation via an API (or use a DNS challenge service like Certify DNS to answer the challenges for you).

Personally the easiest workflow I find is:

  • point your domain to your webserver with an A record in DNS
  • add an http port 80 binding in IIS, IP All Unassigned, hostname set - repeat binding for each hostname variation e.g. example.com and www.example.com
  • confirm your site works over http
  • Fire up Certify The Web, select your IIS site (domains will auto populate from the original http bindings), click Request Certificate.
  • Once you have your sites working and https://example.com etc works, then refine your settings according to your preferences (HSTS etc).

Note also that Certify The Web temporarily runs it’s own http challenge listener in front of IIS, so normally IIS itself is not involved in the challenge unless the listener could not start during validation. If you have something in front like cloudflare with strict https, the http request will never reach port 80 on your server and validation will fail.

3

FYI, I’ve used this CertifyTheWeb app for loads of websites without any issues. I know the process. The problem is that .dev domains always redirect to https no matter what - I even have HSTS disabled in IIS, but it still happens.

TO addres your points: I do have port 80 open (I can reach the site on http using an alternate, non-HSTS domain); and I am not behind a front-end proxy such as Cloudfare. Well, I say that: it is on AWS with an elastic (ie static) IP (ie the binding IP in IIS is not the same as the public server IP - though it’s set to “All unassigned”) - could that be the root of the issue, and if so, have you any ideas what to do about it?

Perhaps I’ll just have to use a different, non-HSTS domain.

4

Thanks, it’s not easy to tell exactly what people do or don’t know from first encounter.

What’s the exact error message you get when attempting validation? Including your domain if you are allowed to share it.

5

I’d rather not share it on a public forum, as it’s on behalf of a client. Can I email you the log file on your support address? (I have an “open source contributors license” so maybe that entitles me to…)

6

Sure, you can email support {at} certifytheweb.com - in which case if you can also include your log file for that managed certificate it would be useful.

Just to confirm, this probably has nothing to do with HSTS itself .dev domains are fully accessible by machines over normal http it’s just that they typically always have an https redirect because a normal web browser isn’t even going to attempt http for that domain. You can see an http only request to your domain using curl -I yourdomain.dev

It’s possible that your domain has an IPv6 AAAA record pointing to a different machine - you can check that using https://letsdebug.net

7

Thanks, have replied now - your domains are pointing to two different IP addresses currently.

1 Like
8

Yeah, I’m an idiot - got myself confused over what I had and hadn’t tried / done. My deepest apologies for the waste of time :frowning:

9

No problem at all! Feel free to get in touch if you ever have questions.