Letsencrypt error 400

[IceBlackz]

Hiya all,

I'm setting up a new, fresh Domoticz server because I moved, but I'm having some troubles getting a SSL certificate with letsencrypt.

I followed the instructions to install certbot from the letsencrypt website: https://certbot.eff.org/instructions?ws ... untubionic

After some trial and error I got to the following command to get a certificate:

Code: Select all

sudo certbot certonly --staging -v --webroot --email *my-email* -d *my_domain* -w /home/domoticz/domoticz/dev-domoticz/www/

But I''m getting the following error:

Code: Select all

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Requesting a certificate for *my_domain*
Performing the following challenges:
http-01 challenge for *my_domain*
Using the webroot path /home/domoticz/domoticz/dev-domoticz/www for all unmatched domains.
Waiting for verification...
Challenge failed for domain *my_domain*
http-01 challenge for *my_domain*

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
  Domain: *my_domain*
  Type:   unauthorized
  Detail: Invalid response from http://*my_domain*/.well-known/acme-challenge/w7X_gdfZJxMr1l-wRgbaJM-hNm1D34Hbj5S5qO6oYzg [*my_ip*]: 400

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

Cleaning up challenges
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

However, I placed a file (robots.txt) with the exact same permissions on that location and I am able to reach it through my domain

Code: Select all

-rw-r--r-- 1 root  root    87 dec 18 18:27 jUpjrjS5pkJAQntW1bR4WG8ukz_V-Vh_atdU7kI-lDI
-rw-r--r-- 1 root  root    35 dec 18 18:27 robots.txt

So, the folder and file is accessible through the Domoticz web hosting.

Has anyone got any ideas how to fix this?

Maybe relevant information:
I'm using duckdns
Port 443 and 80 are forwarded correctly (e.g. I can reach Domoticz web page both through http as https)
I'm running on a Nvidia Jetson Nano, on which it has worked before previously (back in my old house)

[IceBlackz]

As a workaround I got it working by using the DNS authenticator with DuckDNS as following:

Code: Select all

sudo snap install --beta certbot-dns-duckdns;
sudo snap set certbot trust-plugin-with-root=ok;
sudo snap connect certbot:plugin certbot-dns-duckdns
sudo certbot certonly -v --staging --authenticator dns-duckdns --preferred-challenges dns --dns-duckdns-token *duckdns token noted on website of duckdns* --email *my email* -d *my duckdns domain*

I also updated the wiki on that part

[fanabullunet]

Hi IceBlackz,

I upgraded my Domoticz installation yesterday from Build 13872 to 13949 and I'm currently facing the same issue than you do.

I will give it a try to downgrade my installation to 13872 and check if it works then (Trying to point out whether this issue is related to LetsEncrypt or to Domoticz).

I will let you know in a few hours (will do it this evening).

BR

[fanabullunet]

Hi there,

Just checked it having rolled back to build 13872 : Same issue.
The problem is not related to Domoticz or its latest builds.

BR

[Pete118]

I have the same issue (running build 13933)
I have checked my domain with https://letsdebug.net/
No errors. Does anyone have found a solution?

[IceBlackz]

I have the same issue (running build 13933)
I have checked my domain with https://letsdebug.net/
No errors. Does anyone have found a solution?

I also tried this and indeed no error shows with that webpage. I just tried again but still the same error. I'm thinking it might have something to do with certbot being installed through snapd instead of own compilation and install? But it seems to be more a problem of certbot than Domoticz indeed..

Oh and I'm running build 13949

[Pete118]

I could not find a solution. As a wrokaround I also use DNS authentication at the moment.

[fanabullunet]

Hi IceBlackz,

I did the same test as you did with the robots.txt and came to the same result... but :

If I rename the file generated by Certbot to something containing a dot (.) in the name, then the file is reachable!
(in my case, the generated file is named "cEWPEgwKKMG1k9KzSO2OOaScZk3Sglitrnzv60knm2A").

=> mv cEWPEgwKKMG1k9KzSO2OOaScZk3Sglitrnzv60knm2A test.txt : Works !
=> mv cEWPEgwKKMG1k9KzSO2OOaScZk3Sglitrnzv60knm2A test.t : Works !
=> mv cEWPEgwKKMG1k9KzSO2OOaScZk3Sglitrnzv60knm2A test. : Works !
=> mv cEWPEgwKKMG1k9KzSO2OOaScZk3Sglitrnzv60knm2A test : Does NOT Work !!!

Furthermore, in the /var/log/letsencrypt/letsencrypt.log file, if my understanding is correct, you can find something that looks like an HTTP 403 answer (meaning "Forbidden"), what fits to our issue...

Next step would be to understand why the Domoticz webserver doesn't allow the access to a file which doesn't contain a dot (.) in the name and where it can be change (if it is possible to parametrize it).

BR

[kiddigital]

The issue is that the Domoticz webserver always expects a file in the webroot to contain a . (dot) in the filename as it uses the files extension (the part after the last dot) to determine the mime-type it should use to serve the file.

The file generated by Certbot does not contain a dot so Domoticz can not determine the extension and thus the mime-type.

A code change would be needed to make it work.

[fanabullunet]

Hi kiddigital,

Is that rule (mandatory dot in the file name) also true for subfolders of the <webroot> folder ? Because Certbot doesn't create his challenge file in the <webroot> folder but in the <webroot>/.well-known/acme-challenge folder.

Furthermore, this would mean that something has been changed in that area in the latest versions of Domoticz which would be in contradiction with my previous mentioned rollback test :
=> The first time I ran into the issue was immediately after the update to build 13949.
=> Before upgrading to 13949 I ran Build 13872 where it worked without issue.
=> After rollback from Build 13949 to Build 13872 (and since then), it doesn't work anymore.

BR

[kiddigital]

Hi kiddigital,

Is that rule (mandatory dot in the file name) also true for subfolders of the <webroot> folder ? Because Certbot doesn't create his challenge file in the <webroot> folder but in the <webroot>/.well-known/acme-challenge folder.

Yes, true for all files in all (sub)folders.

Furthermore, this would mean that something has been changed in that area in the latest versions of Domoticz which would be in contradiction with my previous mentioned rollback test :
=> The first time I ran into the issue was immediately after the update to build 13949.
=> Before upgrading to 13949 I ran Build 13872 where it worked without issue.
=> After rollback from Build 13949 to Build 13872 (and since then), it doesn't work anymore.

BR

Hm.. not something I can explain easily. But the extra checking has been added a few months back. This could have broken a previously (by accident?) working situation.

I made a few small changes that hopefully make it into a Beta soon.

See PR #5084

[IceBlackz]

Thanks fanabullunet for diving deeper into this! Great to see that a cause has been found, hopefully kiddigital's fix gets released soon

[fanabullunet]

Hi IceBlackz,

Have you had a chance to check if it works now ?
kiddigital's change has been released in commit #13965.

On my side it still doesn't work. Would you be so nice and check it and let me know ?

BR

[kiddigital]

…
kiddigital's change has been released in commit #13965.

On my side it still doesn't work. …

Any ideas/information/errormessages, etc why it does not work (yet) on your end?

[fanabullunet]

Hi kiddigital ,

I'm currently running build 14050. In this version, we are still facing the same behaviour of the web server.

The letsecrypt challenge file that has been created when trying to generate a new certificate in my case is the following:
/home/pi/domoticz/www/.well-known/acme-challenge/xuyPP5g5v0uJEByqqhKyCd2lwVKUHRrQSe3sLJzH_v4

=> If I try to access it, I get an "HTTP 400 Bad request" answer.
=> If I rename or copy it to a new file having a dot(.) and at least one letter as extension, the file becomes accessible.

I have copied it to:
=> "test" : File is not accessible
=> "test." : File is not accessible
=> "test.x" : File IS accessible.

Could you please check once more and let us know ? Thank you

BR

[kiddigital]

Hi kiddigital ,

I'm currently running build 14050. In this version, we are still facing the same behaviour of the web server.

…

Could you please check once more and let us know ? Thank you

BR

Thx for testing and reporting.

Found the issue and made a PR for it. Which has already been merged and is available in Beta 14058 .

‘Problem’ was the dot in the ‘.well-known’ directory name. Old logic did not handle dots in the directory name(s) as it should. Should be ok now.

Let us know if everything works now.

[fanabullunet]

Hi kiddigital ,

Tested a few minutes ago : it works again

Thank you so much

[kiddigital]

Great

[IceBlackz]

After a load of trouble I managed to update (or rather, reinstall) domoticz to 14058 and I can confirm it now works!

[usky73]

I have the relaese version 2022.1 that supposed to include it but same issue : "ERR_EMPTY_RESPONSE" any idea ?