How to use api json with new security ??

[zicht]

Hi

Version: 2023.1 (build 15427)
Build Hash: 3c8e06c73
Compile Date: 2023-07-15 09:09:43
dzVents Version: 3.1.8
Python Version: 3.7.9 (tags/v3.7.9:13c94747c7, Aug 17 2020, 18:01:55) [MSC v.1900 32 bit (Intel)]

I want to use curl to have an external aplication set a switch or fill a variable.
This seems not to be possibe with current security changes. Json is not possible without having the other network device set in the trusted networks.
I do not want to set it in trusted networks as it grants all possible domoticz rigts without any autentication. ( In trusted networks it works without any authentication)

What i tried :

Code: Select all

curl --insecure --connect-timeout 5 --max-time 9 http://plainname:[email protected]:port/json.htm?type=command&param=switchlight&idx=1391&switchcmd=On"

result 401 unautorised

Code: Select all

curl --insecure --connect-timeout 5 --max-time 9 http://bas64name:[email protected]:port/json.htm?type=command&param=switchlight&idx=1391&switchcmd=On"

result 401 unautorised

Code: Select all

curl --insecure --connect-timeout 5 --max-time 9 https://plainname:[email protected]:port/json.htm?type=command&param=switchlight&idx=1391&switchcmd=On"

Error parsing http request address: ::ffff:192.168.0.200

Code: Select all

curl --insecure --connect-timeout 5 --max-time 9 https://bas64name:[email protected]:port/json.htm?type=command&param=switchlight&idx=1391&switchcmd=On"

Error parsing http request address: ::ffff:192.168.0.200

I have tried with and without having "Sta Basic-Auth authenticatie toe over HTTP (API aanroepen)" enabled.
Where is my mistake ? Or is the json api call not functional in the recent beta and only useable in trusted networks ?

I read in some change docs to use a temp token ? how to get that with curl ? An if get how to use it ?
I have limited coding skills, just trying to use domoticz to combine all things in one platform.

[waltervl]

What happens if you do the call from a browser from another system then your Domoticz server ? If this works just use curl without all those options.

Setting 'Sta Basic-Auth authenticatie toe over HTTP (API aanroepen)" has to be switched on as you are using http and not https.

[zicht]

What happens if you do the call from a browser from another system then your Domoticz server ? If this works just use curl without all those options.

Setting 'Sta Basic-Auth authenticatie toe over HTTP (API aanroepen)" has to be switched on as you are using http and not https.

Thank you for the fast reply

Extra detail: Domoticz that is called is running on windows 10.
- With no timeout option same result.
- From edge chrome firefox same results
- 'Sta Basic-Auth authenticatie toe over HTTP (API aanroepen)" on or off does not make a difference
- HTTPS and HTTP both same result (as shown above)

I have read that its working for others. In the past i have used base64 password and username in the url.
That always worked fine. So obvious i am missing something ?
Maybe somebody can post a working curl syntax so i have something to start with.
The 404 unautorised message is indicating something is quite ok ....
In the new security, does the password and username still need to be base64 encoded or just plain text ? I tried both without succes.
is there a dependency on new DLL with the new authentication ?

Only goal is to have a working curl to active a device ON/OFF and populating a uservar.

Edit : When i am on the domoticz system itself in edge,chrome or firefox it only works from the moment i am logged in as admin in the GUI on another tab, so the authentication is causing the problem. When i log out as admin it also does not work anymore from browser

[waltervl]

The way to use API calls with username/password is now

Code: Select all

https://username:password@IP:PORT/json.htm

[zicht]

The way to use API calls with username/password is now

Code: Select all

https://username:password@IP:PORT/json.htm

This keeps giving unauthorized.
I read in the wiki the login is always prefered over secured network, here also i make a different observation. In trusted networks the user is always the first admin account, regardless of the user specified in the api call.

As i am not able to find the solution at this moment, i will make a backup and reinstall, just to see if it makes any difference.

[zicht]

The way to use API calls with username/password is now

Code: Select all

https://username:password@IP:PORT/json.htm

tried over and over for weeks now, still no luck
Made a user:Lua with Password:Password*L

http://Lua:Password*[email protected]:808 ... itchcmd=On

Keeps returning unautorised via curl and in browser (latest chrome).
I can login into domoticz with these credentials
So i still must be missing something....
This command has always worked with the old user= &password= version of the URL

Edit : when i logon in another tab in chrome the command is accepted, so the command is ok, in this case it is using the session of the authenticated tab. So i am really sure it is only the authenication that is failing.

Edit2: yes the switch sta Basic-Auth authenticatie toe over HTTP (API aanroepen) is On

updated to different versions since it is implemented, current version :
Version: 2023.2 (build 15479)
Build Hash: a89b5a4e7
Compile Date: 2023-08-08 07:47:21
dzVents Version: 3.1.8

Is it possible this is just not working on windows with this new authentication ?

[waltervl]

Some browsers do not allow to send passwords over http anymore. Also a * would perhaps need to url encode to %2A

[Lokonli]

The JSON request needs a 'Basic Authentication' header. With curl you can do this as follows:

curl and http:

Code: Select all

curl "http://build:8080/json.htm?type=command&param=getauth" -u admin:domoticz 

curl and https:

Code: Select all

curl -k "https://build/json.htm?type=command&param=getauth" -u admin:domoticz 

Most browsers (incl Chrome) cannot be used for the json request, since they remove the basicAuth user credentials from the url.
So if you want to see the response in a browser, you must add the IP of the browser in the Trusted Networks setting in Domoticz.

or:
First login into Domoticz (http://build:8080), and accept the cookie.

Then open the following url:

Code: Select all

http://build:8080/json.htm?type=command&param=getauth

[zicht]

Some browsers do not allow to send passwords over http anymore. Also a * would perhaps need to url encode to %2A

Tried with just plain text password, same result. Tried to urlencode the complete request, also no luck

[zicht]

The JSON request needs a 'Basic Authentication' header. With curl you can do this as follows:

Thnx will try this. But that makes calling it from other programs ( another hardware) impossible without having its ip set as trusted.
Think like ispy events and a device used for facerecognition or other software/hardware where i can set an event url to interact with domotics.
Previous this was possible with password protection, now all that is running has unlimited acces to domoticz, thus less secure. Shouldn't we have the option to choose for these kind of interaction, making it up to the user to decide. (My domoticz is not connected to internet, some other devices that onteract with domoticz have to be connected outside the subnet)

Edit: curl is working now! Thats a big win allready

[Lokonli]

Thnx will try this. But that makes calling it from other programs ( another hardware) impossible without having its ip set as trusted.
Think like ispy events and a device used for facerecognition or other software/hardware where i can set an event url to interact with domotics.
Previous this was possible with password protection, now all that is running has unlimited acces to domoticz, thus less secure. Shouldn't we have the option to choose for these kind of interaction, making it up to the user to decide. (My domoticz is not connected to internet, some other devices that onteract with domoticz have to be connected outside the subnet)

I would expect that calling the following url from another application still should work:

Code: Select all

http://admin:domoticz@domoticz:8080/json.htm?type=command&param=getauth

Some notes:
* if the calling application is a webapp, and started via https, then the url probably will be rejected (mixed content of http and https)
* It only works when calling the domoticz api. (json.htm must be part of the url)
* 'Allow basicauth on http' must be checked in Domoticz settings

You can also use the https version of Domoticz:

Code: Select all

https://admin:domoticz@domoticz/json.htm?type=command&param=getauth

Additional notes:
* Most applications will reject self-signed ssl cetificates. The default domoticz SSL certificate is self-signed, meaning the https call will be rejected.

Further, I noticed you have to restart Domoticz after adding a new user, otherwise BasicAuth will fail.

[Lokonli]

I tested with iSpy (Agent DVR).

You can configure the action URL as in the picture below:

ispy.jpg (44.23 KiB) Viewed 4198 times

This will switch the Domoticz device to On when motion is detected.

The full URL is:

Code: Select all

http://build:8080/json.htm?type=command&param=switchlight&idx=174&switchcmd=On&level=0

You have to use your own domoticz server and Domoticz device idx of course.


You can generate the Authorization Header at:

Code: Select all

https://www.ispyconnect.com/userguide-agent-actions.aspx

[kiddigital]

…

Further, I noticed you have to restart Domoticz after adding a new user, otherwise BasicAuth will fail.

Hmm.. that should not be needed.

Did you add the user over HTTPS and than tested Basic Auth over HTTP (or vice versa)?

If you test this with the ‘auth’ debugflag enabled, what info do you get?

[Lokonli]

1) If I add the user via HTTPS, and test with HTTP the user is not found.

2) If I then test the same user with HTTPS the user is found.

3) If I restart Domoticz, and test with HTTP the user is found.

log1:

Spoiler: show

Code: Select all

2023-08-14 18:00:30.069  Debug: [web:8080] Host:127.0.0.1 Uri:/json.htm?type=command&param=getauth
2023-08-14 18:00:30.069  Debug: [web:8080] Request Headers:
Host: build:8080
Authorization: Basic dXNlcjQ6dXNlcjQ=
User-Agent: curl/7.74.0
Accept: */*

2023-08-14 18:00:30.069  Debug: [Basic] Found a Basic Auth Header (user4)
2023-08-14 18:00:30.069  Debug: [Auth Check] Username admin, Password 59515f6b193071e263f14bfa94be..., rights ...
2023-08-14 18:00:30.069  Debug: [Auth Check] Username user, Password ee11cbb19052e40b07aac0ca060c2..., rights ...
2023-08-14 18:00:30.069  Debug: [Auth Check] Username user2, Password 5ebe2294ecd0e0f08eab7690d2a6e..., rights ...
2023-08-14 18:00:30.069  Debug: [Auth Check] Username user3, Password 92877af70a45fd6a2ed7fe81e1236..., rights ...
2023-08-14 18:00:30.069  Debug: [Auth Check] Username domoticzUI, Password d41d8cd98f00b204e9800998ecf84..., rights ...
2023-08-14 18:00:30.069  Debug: [Auth Check] Username dashticz, Password b155b97b7c8b06992fd649317ae74..., rights ...
2023-08-14 18:00:30.069  Debug: [Auth Check] Invalid Basic Authorization for API call!
2023-08-14 18:00:30.069  Debug: [web:8080] isPage 1 isAction 0 isUpgrade 0 needsAuthentication 0 isAuthenticated 0 ()
2023-08-14 18:00:30.069  Debug: CWebServer::GetJSonPage :getauth :/json.htm?type=command&param=getauth 
2023-08-14 18:00:30.069  Debug: Web ACLF: 127.0.0.1 - - [14/Aug/2023:18:00:30.069 +0200] "GET /json.htm?type=command&param=getauth HTTP/1.1" 200 43 - "curl/7.74.0"

log2:

Spoiler: show

Code: Select all

2023-08-14 18:06:42.200  Debug: [web:443] Host:127.0.0.1 Uri:/json.htm?type=command&param=getauth
2023-08-14 18:06:42.200  Debug: [web:443] Request Headers:
Host: build
Authorization: Basic dXNlcjQ6dXNlcjQ=
User-Agent: curl/7.74.0
Accept: */*

2023-08-14 18:06:42.200  Debug: [Basic] Found a Basic Auth Header (user4)
2023-08-14 18:06:42.200  Debug: [Auth Check] Username admin, Password 59515f6b193071e263f14bfa94bef..., rights ...
2023-08-14 18:06:42.200  Debug: [Auth Check] Username user, Password ee11cbb19052e40b07aac0ca060c2..., rights ...
2023-08-14 18:06:42.200  Debug: [Auth Check] Username user2, Password 5ebe2294ecd0e0f08eab7690d2a6e..., rights ...
2023-08-14 18:06:42.200  Debug: [Auth Check] Username user3, Password 92877af70a45fd6a2ed7fe81e1236..., rights ...
2023-08-14 18:06:42.200  Debug: [Auth Check] Username user4, Password 3f02ebe3d7929b091e3d8ccfde2f3..., rights ...
2023-08-14 18:06:42.200  Debug: [Auth Check] Found Basic Authorization for API call: Method BASIC, Userdata user4, rights 1
2023-08-14 18:06:42.200  Debug: [web:443] isPage 1 isAction 0 isUpgrade 0 needsAuthentication 0 isAuthenticated 1 (user4)
2023-08-14 18:06:42.200  Debug: CWebServer::GetJSonPage :getauth :/json.htm?type=command&param=getauth 
2023-08-14 18:06:42.200  Debug: Web ACLF: 127.0.0.1 - - [14/Aug/2023:18:06:42.200 +0200] "GET /json.htm?type=command&param=getauth HTTP/1.1" 200 114 - "curl/7.74.0"

log3:

Spoiler: show

Code: Select all

023-08-14 20:01:41.199  Debug: [web:8080] Host:127.0.0.1 Uri:/json.htm?type=command&param=getauth
2023-08-14 20:01:41.200  Debug: [web:8080] Request Headers:
Host: build:8080
Authorization: Basic dXNlcjQ6dXNlcjQ=
User-Agent: curl/7.74.0
Accept: */*

2023-08-14 20:01:41.200  Debug: [Basic] Found a Basic Auth Header (user4)
2023-08-14 20:01:41.200  Debug: [Auth Check] Username admin, Password 59515f6b193071e263f14bfa94bef..., rights ...
2023-08-14 20:01:41.200  Debug: [Auth Check] Username user, Password ee11cbb19052e40b07aac0ca060c2..., rights ...
2023-08-14 20:01:41.200  Debug: [Auth Check] Username user2, Password 5ebe2294ecd0e0f08eab7690d2a6e..., rights ...
2023-08-14 20:01:41.200  Debug: [Auth Check] Username user3, Password 92877af70a45fd6a2ed7fe81e1236..., rights ...
2023-08-14 20:01:41.200  Debug: [Auth Check] Username user4, Password 3f02ebe3d7929b091e3d8ccfde2f3..., rights ...
2023-08-14 20:01:41.200  Debug: [Auth Check] Found Basic Authorization for API call: Method BASIC, Userdata user4, rights 1
2023-08-14 20:01:41.200  Debug: [web:8080] isPage 1 isAction 0 isUpgrade 0 needsAuthentication 0 isAuthenticated 1 (user4)
2023-08-14 20:01:41.200  Debug: CWebServer::GetJSonPage :getauth :/json.htm?type=command&param=getauth 
2023-08-14 20:01:41.200  Debug: Web ACLF: 127.0.0.1 - - [14/Aug/2023:20:01:41.199 +0200] "GET /json.htm?type=command&param=getauth HTTP/1.1" 200 114 - "curl/7.74.0"

I've added some logging, starting with '[Auth Check] Username ' in my local build.

[zozo]

Hello,

In 2023*2, I have a switch with an action "On" (to add +1 to a water counter) and the command :

Code: Select all

http://localhost:8080/json*htm?type=command&param=udevice&idx=3712&nvalue=0&svalue=1

I know there is some evolutions, and there is no way to resolve "Error opening the URL"...
I've tried new commands for 2023*2, but no way this is not working :

Code: Select all

http://admin:domoticz@localhost:8080/json*htm?type=command&param=getdevices&rid=3712&nvalue=0&svalue=1

Can you help me ?

Note : json* is not a mistake "*" is "."

[kiddigital]

You are using HTTP and not HTTPS, so an unsecured connection.

You need to enable Basic Authentication over HTTP in the security settings otherwise domoticz will not allow/accept Basic authentication over an insecure connection. (Make sure you consider the security risks it poses).

More information and answer can be found in the top of this thread and other forum threads.

[habahabahaba]

Hi.
I'm using dzvents script for switches with action like this:

domoticz.openURL('http://127.0.0.1:8080/json.htm?username ... 0&svalue=1')

works fine.

and as kiddigital said you need to enable Basic Authentication over HTTP in the security settings

[ajarzyn]

For everyone strugling with authentication to Domoticz's REST API https://www.domoticz.com/wiki/Domoticz_API/JSON_URL%27s

1. Do not test this functionality with Firefox (121.0 (64-bit)) or Brave (v1.61.109 (Dec 21, 2023))
Why? This browsers do not add Authentication header. No authentication credentials are passed to Domoticz server.
Additionally Firefox detect login as potential risk with popup warning.
2. Use curl
3. Wrap URI in quotes

Code: Select all

curl "https://user:password@domoticzip:domoticzport/json.htm?type=command&param=customevent&event=gate_open"

4. Do not webescape the parameters part of the URI eg.

Code: Select all

&?=