Nested HTTPResponse

[elmortero]

Hi all,

Our national Weather Service has a rather complicated way of fetching info.
FIrst you access the API with a URL constructed of several parameters (type of data you want, stationID and API key):
https://opendata.aemet.es/opendata/api/ ... key=XXXXXX
That results in a JSON response like:

Code: Select all

{
descripcion: "éxito",
estado: 200,
datos: "https://opendata.aemet.es/opendata/sh/ff3de32d",
metadatos: "https://opendata.aemet.es/opendata/sh/8ca2e7e3",
}

Where a temporary URL is generated, based on you request. It is valid for 5 minutes. The URL that leads to the actual info is in the field called "Datos".

I have tried several approaches to fetch the data I want but so far all fail..
One method I used was running 2 separate scripts: 1 to fetch the "Datos" field and write it to a variable (both dzvents persisten and domoticz user variable) and then 2 minutes later a second one that calls the generated URL. With that one I always get nil data or invalid JSON result. When the URL has been fetched and I open it by copy/past from the variable I get a valid JSON with the data available.

Second method I have been trying is combining the URLs in one script (like below) but that does not give the results either.
It does get the "datos" field, which I know by both printing it to the log and writing it to the variable.
But I do not get it to trigger the part where the final data are fetched.
There are no errors shown, it just does not trigger.

Hope the many eyes here spot the (probably) silly mistake I am making.
The current script I have is

Code: Select all

return {
	on = {
		timer = { 'at *:26', 'at *:52' },
		httpResponses = { 'aemeturl', 'aemetInfo' } -- matches callback string below
	},
	
	execute = function(domoticz, triggerItem)

		if (triggerItem.isTimer) then
			domoticz.openURL({
				url = 'https://opendata.aemet.es/opendata/api/prediccion/especifica/playa/0301408/?api_key=MYKEY',
				method = 'GET',
				callback = 'aemeturl'
			})
			
		elseif (triggerItem.isHTTPResponse) then

	local response = triggerItem
		if (response.ok and response.isJSON and response.trigger == 'aemeturl') then
		 local Aemetlink = tostring(response.json.datos)
			--processing
			print('Aemetlink = '..Aemetlink)
			domoticz.variables('Aemetlink').set(Aemetlink)
				domoticz.openURL({
					url = Aemetlink,
					method = 'GET',
					callback = 'aemetInfo'
				})
			--end processing
			end
			
			
		if (response.ok and response.isJSON and response.trigger == 'aemetInfo') then			
			local city = tostring(response.json.nombre)  	--location
			print('City = '..city)
			local tagua = domoticz.round((tonumber(response.json.prediccion.dia[1].tagua.valor1)),1)		--visibility
			print('Agua = '..tagua)
			local wave = tostring(response.json.prediccion.dia[1].oleaje.descripcion2)
			print('Oleaje = '..wave)
		end
	
		end
	end
}

[dannybloe]

Have you tried to set it in debug mode and see what is being logged? Also try to log the response headers and the raw data.

[elmortero]

Hi,

Seems that the headers say it is text/plain, even though a copy and paste to jsonlint.com says is a valid JSON table.

Code: Select all

dzVents: Info: beach: {["Accept"]="*/*", ["Content-Length"]="2666", ["Server"]="Apache/2.2.15 (CentOS)", ["Connection"]="close", ["Date"]="Sun, 06 May 2018 20:40:14 GMT", ["Content-Type"]="text/plain;charset=ISO-8859-15"}

With that knowledge, I altered HTTPResponse.lua (I know, shouldn't mess there ) to validate as JSON also when the headers say that the content-type is text/plain.
And guess what: It works!

[dannybloe]

You don’t need to do that. You can convert the raw data to json in your script using domoticz.utils. And report a bug that they don’t set their content headers properly.

[dannybloe]

Besides. Your change will throw errors if the data is text and not json.

[elmortero]

You don’t need to do that. You can convert the raw data to json in your script using domoticz.utils. And report a bug that they don’t set their content headers properly.

hi Daniel,

You are talking about fromJSON(json) I assume? Already been looking at that but honestly I am having a hard time figuring out how to use it
About reporting it to the provider, done. Let's see how they respond.

[dannybloe]

This should do the trick (it is exactly the same as in HTTPResponse.lua that you adapted):

Code: Select all

domoticz.utils.fromJSON(response.data)

And response is the same object as in your script.

You see, the reason why the json is available on the response object in dzVents is more of a convenience. IF the response type is application/json THEN I do the conversion to Lua automatically because I can be sure that it can be converted and it is likely you will need it in your script. On the other hand, if the response type is not like this (as in your situation) but you KNOW that it is json, then you can do it yourself with the code I gave you above.

[elmortero]

Great!
It works now.