Upload gas usage to mindergas.nl [Solved]

[roblom]

So this is my first dzvents script ever, what do you think of it? Please give some feedback.

It collects the gas usage from the smartmeter and uploads it to the mindergas.nl database.

Code: Select all

--[[
 /home/pi/domoticz/scripts/dzVents/scripts/upload_gas_usage_to_minderGas.lua
 Author 		: Roblom
 Description 	: This script collects the gas values from Domoticz (for example from the smart meter) and uploads the values to a MinderGas account.
			For more information about MinderGas, see the API instructions on their website on https://mindergas.nl/member/api
]]--

---------------------------------------------------------------------------------------------------------------
local AUTH_TOKEN = 'xxxxxxxxxxxxxx'	 -- Fill in here your Mindergas authentication token
---------------------------------------------------------------------------------------------------------------

return {
	active = true,

	on = {
		--timer 			= {'at 00:05'},
		timer 			= {'every minute'},
		httpResponses 	= {'UploadToMindergas'}
	},
	
	logging =	{
		level	=	domoticz.LOG_DEBUG,
		marker	=	'Mindergas'
	},
	
	execute = function(domoticz, item)

		
        if item.isTimer then
		
			local GasUsageCounter 	= domoticz.devices('Gas (P1)').counter
			local TodaysDate 		= tostring(domoticz.time.rawDate)

			domoticz.log('The gas usage is ' .. GasUsageCounter, domoticz.LOG_INFO)
			domoticz.log('The date is ' .. TodaysDate, domoticz.LOG_INFO )
		
			domoticz.openURL({
				url = 'https://www.mindergas.nl/api/gas_meter_readings',
				method = 'POST',
				headers = {
					['Content-Type']	= 'application/json',
					['AUTH-TOKEN']		= AUTH_TOKEN
				},
				callback = 'UploadToMindergas',
				postData = {
					['date']			= TodaysDate,
					['reading']			= GasUsageCounter
				},
			})
			
        elseif (item.isHTTPResponse) then
			if (item.statusCode == 201) then
				domoticz.log('Gas usage data is sucessfully upoaded to Mindergas.nl.', domoticz.LOG_INFO)
			else
				if (item.statusCode == 401) then
					domoticz.log('There was an authorisation problem with the Mindergas.nl database.', domoticz.LOG_ERROR)
				end
				if (item.statusCode == 422) then
					domoticz.log('There was an unprocessable enrty while trying to upload the gas usage data to Mindergas.nl', domoticz.LOG_ERROR)
				end
				domoticz.notify('Domoticz error', 'An error occured while trying to upload the gas usage data to Mindergas.nl', PRIORITY_NORMAL)
			end
        end
	end
}

[waaren]

So this is my first dzvents script ever, what do you think of it? Please give some feedback.

@robblom, my compliments; looking good.

some small not important remarks:

You included your Mindergas authentication token as plain text in your script. What you could do is to put the token in a domoticz uservariable and read that during the execution of the script. In that way the token is not direct readable n the script. You could even make it really fancy if you store the key encoded en decode it during execution. Many examples to encode / decode in Lua can be found on the internet.

Code: Select all

if (item.statusCode == 201) then

is excellent. Could be replaced by

Code: Select all

if item.ok then -- self.statusCode >= 200 and self.statusCode <= 299

to test for a more generic statsusCode to reflect success.

You use the string "trigger" to trigger the script on the openURL callback. If you develop more scripts you might want to use a more distinctive name.

[roblom]

Thanks for the compliments, it's not that complicated script but was a good example to learn dzVents. As I'm no programmer at all I'm learning through trial and error but have good autodidact skills and dzVents is quite easy to learn.

About the authentication token, I removed it from the above example, thanks for pointing out.
but putting it in a user variable is of course an option, but would this be a real advantage in security purpose? Because a user variable is also readable in domoticz GUI itself.

The statusCode that are described in the mindergas API are only 201, 401 and 422 so therefore I only put in those. If the status is not 201, 401 or 422 there will be send a notification for extra attention.

The 'trigger' name is indeed not that distinctive, changed that to 'UploadToMindergas'.

Again, thanks for the feedback, quite useful!

[waaren]

but putting it in a user variable is of course an option, but would this be a real advantage in security purpose? Because a user variable is also readable in domoticz GUI itself.

True if you edit and save your scripts via the internal event editor. I am used to an external editor and do not store my scripts in the database. So if you encrypt / decrypt the value in the uservariable, users with access to the gui and without access to the OS file system would not be able to see the decrypted api-key; but I doubt if there is a really secure way to handle a situation where the api-key need to be part of the url.

[roblom]

I also use a external editor (Notepad++). As I don't find it a big security issue, I leave it as it is for now. Thanks for your feedaback.

[EdwinK]

Looks great, but

2019-05-21 08:36:01.238 Status: dzVents: Info: Handling httpResponse-events for: "UploadToMindergas
2019-05-21 08:36:01.238 Status: dzVents: Info: Mindergas: ------ Start internal script: Minder Gas: HTTPResponse: "UploadToMindergas"
2019-05-21 08:36:01.238 Status: dzVents: Error (2.4.21): Mindergas: HTTP/1.1 response: 422 ==>> Unprocessable Entity
2019-05-21 08:36:01.238 Status: dzVents: Error (2.4.21): Mindergas: There was an unprocessable enrty while trying to upload the gas usage data to Mindergas.nl
2019-05-21 08:36:01.238 Status: dzVents: Info: Mindergas: ------ Finished Minder Gas
2019-05-21 08:37:00.582 Status: dzVents: Info: Mindergas: ------ Start internal script: Minder Gas:, trigger: every minute
2019-05-21 08:37:00.582 Status: dzVents: Info: Mindergas: The gas usage is 1184.658
2019-05-21 08:37:00.582 Status: dzVents: Info: Mindergas: The date is 2019-05-21
2019-05-21 08:37:00.587 Status: dzVents: Debug: Mindergas: OpenURL: url = https://www.mindergas.nl/api/gas_meter_readings
2019-05-21 08:37:00.587 Status: dzVents: Debug: Mindergas: OpenURL: method = POST
2019-05-21 08:37:00.587 Status: dzVents: Debug: Mindergas: OpenURL: post data = {"date":"2019-05-21","reading":1184.658}
2019-05-21 08:37:00.587 Status: dzVents: Debug: Mindergas: OpenURL: headers = {["Content-Type"]="application/json", ["AUTH-TOKEN"]="WdC_oA8VqqLjB6ijYmzC"}
2019-05-21 08:37:00.587 Status: dzVents: Debug: Mindergas: OpenURL: callback = UploadToMindergas
2019-05-21 08:37:00.587 Status: dzVents: Info: Mindergas: ------ Finished Minder Gas

[EdwinK]

It seems the meter is updated at mindergas.nl, just throwing an error in the logs.

[waaren]

It seems the meter is updated at mindergas.nl, just throwing an error in the logs.

Could it be that this entry is already at mindergas ? That would also generate this returnCode

[roblom]

It seems the meter is updated at mindergas.nl, just throwing an error in the logs.

Could it be that this entry is already at mindergas ? That would also generate this returnCode

That is correct, the 422 error is

422 Unprocessable Entity
Er is een validatiefout opgetreden. Dit kan komen door:

Er is al een meterstand voor de opgegeven datum.
De meterstand is geen getal.
De meterstand is kleiner dan de vorige meterstand.
De meterstand is groter dan de volgende meterstand.
De datum ligt in de toekomst.
De datum ligt voor 31 december 2005.

So if you say the data is uploaded in the mindergas database then the 422 error is because you try to put in the same values that are already in. So when you change the script to that it is only running once a day, you have no errors anymore.

[EdwinK]

Okay. Will change the script to run less often. Thanks

[HvdW]

And can you upload the finished script too?
It's very interesting.
To get results that can be compared over the years (I'm on mindergas since 2013) I will upload it once a moth and as from now it'll be automagic upload!

[maxtrash]

Thanks, works like a charm.

[BOverdevest]

Thanks, works like a charm.

indeed and thanks for pushing this topic to the top again, or I would have missed this one...

[jandahmen]

pls publish the last good working script, so we can learn too. regards jan dahmen

[BOverdevest]

pls publish the last good working script, so we can learn too. regards jan dahmen

See post #1, worked for me, with some adjustment to make it work on my system.

Code: Select all

--[[
 /home/pi/domoticz/scripts/dzVents/scripts/upload_gas_usage_to_minderGas.lua
 Author 		: Roblom
 Description 	: This script collects the gas values from Domoticz (for example from the smart meter) and uploads the values to a MinderGas account.
			      For more information about MinderGas, see the API instructions on their website on https://mindergas.nl/member/api
]]--

---------------------------------------------------------------------------------------------------------------
local AUTH_TOKEN = 'xxxxxxxxxxxxxx'	 -- Fill in here your Mindergas authentication token
---------------------------------------------------------------------------------------------------------------

return {
	active = true,
	on = {
		timer 			= {'at 00:05'},
		--timer 			= {'every minute'},
		httpResponses 	= {'UploadToMindergas'}},
	logging =	{	level	=	domoticz.LOG_ERROR, marker	=	'Mindergas'	},
	execute = function(domoticz, item)
		
        if item.isTimer then
		
			local GasUsageCounter 	= domoticz.devices('P1GasMeterStand').counter
			local TodaysDate 		= tostring(domoticz.time.rawDate)

			domoticz.log('The gas usage is ' .. GasUsageCounter, domoticz.LOG_INFO)
			domoticz.log('The date is ' .. TodaysDate, domoticz.LOG_INFO )
		
			domoticz.openURL({
				url = 'https://www.mindergas.nl/api/gas_meter_readings',
				method = 'POST',
				headers = {
					['Content-Type']	= 'application/json',
					['AUTH-TOKEN']		= AUTH_TOKEN
				},
				callback = 'UploadToMindergas',
				postData = {
					['date']			= TodaysDate,
					['reading']			= GasUsageCounter
				},
			})
			
        elseif (item.isHTTPResponse) then
            local SubSystem   = domoticz.NSS_TELEGRAM
            local Priority      = domoticz.PRIORITY_NORMAL
            local Sound         = domoticz.SOUND_DEFAULT
            local Tittle        = "MinderGas - "
			local Message       = "Geen idee :" .. item.statusCode
			
			if (item.statusCode == 201) then Message ='Gas usage data is sucessfully upoaded.'
			    elseif (item.statusCode == 401) then Message = 'There was an authorisation problem with the Mindergas.nl database.'
			    elseif (item.statusCode == 422) then Message = 'There was an unprocessable enrty while trying to upload the gas usage data to Mindergas.nl'
			end 	
			domoticz.log(Message, domoticz.LOG_INFO)
			domoticz.notify(Tittle,Message,Priority,Sound,"",SubSystem)
        end
	end
}

--[[
De volgende HTTP status codes worden geretourneerd:

201 Created 
De meterstand is succesvol verwerkt en opgeslagen.

401 Unauthorized 
Het authenticatietoken is ongeldig of zit niet in de request.

422 Unprocessable Entity 
Er is een validatiefout opgetreden. Dit kan komen door:

Er is al een meterstand voor de opgegeven datum.
De meterstand is geen getal.
De meterstand is kleiner dan de vorige meterstand.
De meterstand is groter dan de volgende meterstand.
De datum ligt in de toekomst.
De datum ligt voor 31 december 2005.
]]--

[Akerboom]

thanks for the script!

[dutchronnie]

I do use this script some time without any problems.

But since a few days the readings are not uploaded anymore, and do i get the this error

An error occured while trying to upload the gas usage data to Mindergas.nl',


I didn't change anything, but is is not working anymore.
Does sombody know what the problem could be?

Code: Select all

--[[
 /home/pi/domoticz/scripts/dzVents/scripts/upload_gas_usage_to_minderGas.lua
 Author 		: Roblom
 Description 	: This script collects the gas values from Domoticz (for example from the smart meter) and uploads the values to a MinderGas account.
			For more information about MinderGas, see the API instructions on their website on https://mindergas.nl/member/api
]]--

---------------------------------------------------------------------------------------------------------------
local AUTH_TOKEN = 'KKgjpoiJJeJPG'kjhgf'	 -- Fill in here your Mindergas authentication token
---------------------------------------------------------------------------------------------------------------

return {
	active = true,

	on = {
		timer 			= {'at 00:05'},
		--timer 			= {'every minute'},
		httpResponses 	= {'UploadToMindergas'}
	},
	
	logging =	{
		level	=	domoticz.LOG_DEBUG,
		marker	=	'Mindergas'
	},
	
	execute = function(domoticz, item)

		
        if item.isTimer then
		
			local GasUsageCounter 	= domoticz.devices('Gas (P1)').counter
			local TodaysDate 		= tostring(domoticz.time.rawDate)

			domoticz.log('The gas usage is ' .. GasUsageCounter, domoticz.LOG_INFO)
			domoticz.log('The date is ' .. TodaysDate, domoticz.LOG_INFO )
		
			domoticz.openURL({
				url = 'https://www.mindergas.nl/api/gas_meter_readings',
				method = 'POST',
				headers = {
					['Content-Type']	= 'application/json',
					['AUTH-TOKEN']		= AUTH_TOKEN
				},
				callback = 'UploadToMindergas',
				postData = {
					['date']			= TodaysDate,
					['reading']			= GasUsageCounter
				},
			})
			
        elseif (item.isHTTPResponse) then
			if (item.statusCode == 201) then
				domoticz.log('Gas usage data is sucessfully upoaded to Mindergas.nl.', domoticz.LOG_INFO)
			else
				if (item.statusCode == 401) then
					domoticz.log('There was an authorisation problem with the Mindergas.nl database.', domoticz.LOG_ERROR)
				end
				if (item.statusCode == 422) then
					domoticz.log('There was an unprocessable enrty while trying to upload the gas usage data to Mindergas.nl', domoticz.LOG_ERROR)
				end
				domoticz.notify('Domoticz error', 'An error occured while trying to upload the gas usage data to Mindergas.nl', PRIORITY_NORMAL)
			end
        end
	end
}

[waaren]

"An error occured while trying to upload the gas usage data to Mindergas.nl',"

If you insert below line directly after line 61 it will show some potential useful information

Code: Select all

domoticz.log(item, domoticz.LOG_FORCE)

[dutchronnie]

I added the line in the script, i hope i can see it tommorow.

I found the following rules in the log file from domoticz

2020-11-15 00:03:01.237 Error: dzVents: Error: (3.0.16) Mindergas: HTTP/1.1 response: 401 ==>> Unauthorized
2020-11-15 00:03:01.238 Error: dzVents: Error: (3.0.16) Mindergas: There was an authorisation problem with the Mindergas.nl database.
2020-11-16 00:05:01.755 Error: dzVents: Error: (3.0.16) Mindergas: HTTP/1.1 response: 401 ==>> Unauthorized
2020-11-16 00:05:01.755 Error: dzVents: Error: (3.0.16) Mindergas: There was an authorisation problem with the Mindergas.nl database.
2020-11-17 00:05:01.694 Error: dzVents: Error: (3.0.16) Mindergas: HTTP/1.1 response: 401 ==>> Unauthorized
2020-11-17 00:05:01.694 Error: dzVents: Error: (3.0.16) Mindergas: There was an authorisation problem with the Mindergas.nl database.

Authorisation problem, strange. i checked the api, but it is alright.
On mindergas i can add the values to the database by hand, so there is nothing wrong with the database.

[alpafranken]

I added the line in the script, i hope i can see it tommorow.

I found the following rules in the log file from domoticz

2020-11-15 00:03:01.237 Error: dzVents: Error: (3.0.16) Mindergas: HTTP/1.1 response: 401 ==>> Unauthorized
2020-11-15 00:03:01.238 Error: dzVents: Error: (3.0.16) Mindergas: There was an authorisation problem with the Mindergas.nl database.
2020-11-16 00:05:01.755 Error: dzVents: Error: (3.0.16) Mindergas: HTTP/1.1 response: 401 ==>> Unauthorized
2020-11-16 00:05:01.755 Error: dzVents: Error: (3.0.16) Mindergas: There was an authorisation problem with the Mindergas.nl database.
2020-11-17 00:05:01.694 Error: dzVents: Error: (3.0.16) Mindergas: HTTP/1.1 response: 401 ==>> Unauthorized
2020-11-17 00:05:01.694 Error: dzVents: Error: (3.0.16) Mindergas: There was an authorisation problem with the Mindergas.nl database.

Authorisation problem, strange. i checked the api, but it is alright.
On mindergas i can add the values to the database by hand, so there is nothing wrong with the database.

The problem is your local AUTH_TOKEN = 'KKgjpoiJJeJPG'kjhgf' your token has an ' in it which causes the variable to malfunction.