Domoticz

Posted: **Wednesday 13 May 2020 21:01**

Waaren,

Thank you for al the info.

I can decode my vbus file with manual codes. But automatic doenst work yet.

Which codes have you added in crontab?

0/1 * * * * wget -O /home/pi/resol-vbus.rs/vbus/current_packets.vbus http://192.168.2.19/current/current_packets.vbus
0/1 * * * * /home/pi/resol-vbus.rs/examples/formatter/target/release/formatter csv /home/pi/resol-vbus.rs/vbus/current_packets.vbus

the second code doesnt work, only when you are in the formatter folder. Do you have an solution?

The next step. How have you uploaded your .lua script in domoticz. I have added it to the dzvents folder but nnothing happens.

Thanks Sander

waaren wrote: ↑Monday 17 June 2019 17:41 Almost forgot to post the final working version here.

Code: Select all

--[[ 

    get values from csv file and store in virtual sensors
    (dzVents >= 2.4.19)

Based on <TAB> separated data file: HR Solar BASIC controller [Regler] HR Solar BASIC controller [WMZ 1]

[01] Date / Time
[02] Systemdatum
[03] Temperatur Sensor 1
[04] Temperatur Sensor 2
[05] Temperatur Sensor 3
[06] Temperatur Sensor 4
[07] TAGE
[08] Volumenstrom Sensor V40
[09] Volumenstrom Sensor VFS
[10] Temperatur Sensor VFS
[11] Drehzahl Relais 1
[12] Drehzahl Relais 2
[13] Regler Ausgang 1
[14] Wmz1_Wert_Wh
[15] SW-Version
[16] Betriebsstunden Relais1
[17] Betriebsstunden Relais2
[18] Urlaubsfunktion
[19] Blockierschutz 1
[20] Initalisieren
[21] Befüllung
[22] Stabilisieren
[23] Pumpenverzögerung
[24] Überwärmeabfuhr
[25] Nachlauf
[26] Speicherkühlung
[27] Frostschutz
[28] Kollektorkühlung
[29] Einheit Temperatur
[30] Speichermaximaltemperatur
[31] Neustarts
[32] Fehlermaske
[33] Wmz1_Wert_Wh
[34] Wmz1_Leistung_W
[35] Wmz1_WertHeute_Wh
[36] Wmz1_WertWoche_Wh

]]--

return {
        on = 
        {
            timer     = { 'every minute' }, -- change to the interval suited for your situation
        },

        logging = 
        {
            level = domoticz.LOG_DEBUG, -- set to LOG_ERROR when script is working as expected
            marker = 'read CSV data' 
        },

    execute = function( dz )
        local maxAge = 300             -- max Age of data file in seconds
        local round = dz.utils.round   -- shorthand

        local dataFile = '/home/pi/resol-vbus.rs/examples/formatter/Output.csv'  -- location/csfFile 
        local temperatureSensor1 = dz.devices('temperature Solar collector')  -- create as virtual temperature sensor
        local temperatureSensor2 = dz.devices('temperature Boilervat') -- create as virtual temperature sensor
        local temperatureSensorVFS = dz.devices('temperature outlet')  -- create as virtual temperature sensor
        local drehzahlRelais1 = dz.devices('pump-speed Solar collector')  -- create as virtual percentage sensor
        local Wmz1_Wert = dz.devices('total power Solar collector') -- create as Usage Electric
        local VolumenstromSensorVFS = dz.devices('Boiler outlet') -- create as waterflow (l/m)
        local Wmz1_WertHeute = dz.devices('daily total Solar collector') -- create as Usage Electric

        local function readCSVLine(OSfile)
            local f = assert(io.popen('tail -1 '.. OSfile)) -- OS command to read last line of file
            local line = assert(f:read('*a'))               -- read the complete result into string variable
            f:close() -- close the file
            dz.log('line: ' .. line,dz.LOG_DEBUG)
            return dz.utils.stringSplit(line:gsub(',','.'),'%\t') -- replace decimal commas with . ; use TAB as delimiter and split line into columns (table) 
        end

        local function isRecent(csvData)
            if csvData == nil or #csvData < 15 then 
                dz.log("Problem with reading and converting to table of the data" ,dz.LOG_ERROR)
                return
            end 
            local Time = require('Time')
            local csvTime = Time(csvData[1]:gsub('T',' '))  -- convert timestamp in csv file to time object
            local now = dz.time                             -- Get current time into time object

            if now.compare(csvTime).secs > maxAge then 
                dz.log('Timestamp in ' .. dataFile .. ' is ' .. now.compare(csvTime).secs .. ' seconds old. Please check!',dz.LOG_ERROR)
                return 
            end    
            return true
        end

        local function updateTemperatureCheckFirst(device, temperature)
            if round(device.temperature,1) ~= round(temperature,1) then
                dz.log('---------' .. temperature ..' ==>> ' .. device.temperature  ,dz.LOG_DEBUG)
                device.updateTemperature(round(temperature,1))
            end
        end

        local function updateSensorsCheckFirst(csvData)
          -- Sensors will only be updated when old value ~= new value
            updateTemperatureCheckFirst(temperatureSensor1,csvData[3]) 
            updateTemperatureCheckFirst(temperatureSensor2,csvData[4])
            updateTemperatureCheckFirst(temperatureSensorVFS,csvData[10])

            if round(drehzahlRelais1.percentage,1) ~= round(csvData[11],1) then drehzahlRelais1.updatePercentage(round(csvData[11],1)) end
            if round(Wmz1_Wert.WhActual,1) ~= round(csvData[14],1) then Wmz1_Wert.updateEnergy(round(csvData[14],1)) end
            if round(VolumenstromSensorVFS.flow,1) ~= round(csvData[9],1) then VolumenstromSensorVFS.updateWaterflow(round(csvData[9],1)) end
            if round(Wmz1_WertHeute.WhActual,1) ~= round(csvData[35],1) then WertHeute.updateEnergy(round(csvData[35],1)) end
        end
        
        -- main
        csvData = readCSVLine(dataFile)

        if isRecent(csvData) then
            updateSensorsCheckFirst(csvData)
        else
            dz.log("Something was not OK. Skipping device updates",dz.LOG_ERROR)
        end
    end
}

Posted: **Wednesday 13 May 2020 21:14**

sanderj1985 wrote: ↑Wednesday 13 May 2020 21:01 I can decode my vbus file with manual codes. But automatic doenst work yet.
The next step. How have you uploaded your .lua script in domoticz. I have added it to the dzvents folder but nnothing happens.

@sanderj1985

I did not do anything on the crontab side. I just process the csv file that was presented by the OP using a dzVents script.

You can find how to get the dzVents script active in domoticz in the wiki Quickstart chapter

When not yet familiar with dzVents please start with reading Get started Before implementing (~ 5 minutes). Special attention please for "In Domoticz go to Setup > Settings > Other and in the section EventSystem make sure the checkbox 'dzVents enabled' is checked. Also make sure that in the Security section in the settings you allow 127.0.0.1 to not need a password. dzVents uses that port to send certain commands to Domoticz. Finally make sure you have set your current location in Setup > Settings > System > Location, otherwise there is no way to determine nighttime/daytime state."

Posted: **Friday 15 May 2020 14:21**

Waaren,

Thank you for the information.

I have dzVents working. The only problem is one command.

domoticz.utils.osExecute('/home/pi/resol-vbus.rs/examples/formatter/formatter -- simple-json /tmp/current_packets.vbus')

When I put the command ('') in Putty it works and the vbus file is decoded.

But dzVents does not execute the command. I think it is a autorisation question.

Any ideas?

Posted: **Friday 15 May 2020 16:29**

sanderj1985 wrote: ↑Friday 15 May 2020 14:21 domoticz.utils.osExecute('/home/pi/resol-vbus.rs/examples/formatter/formatter -- simple-json /tmp/current_packets.vbus')
Any ideas?

A couple of posts back you stated that this only works when in the formatter directory. Maybe if you change this to

os.execute('cd <formatter directory> ;/home/pi/resol-vbus.rs/examples/formatter/formatter -- simple-json /tmp/current_packets.vbus')

Posted: **Friday 15 May 2020 20:04**

Waaren,

Thx the script is now working perfect.

Have a nice weekend!

Posted: **Sunday 13 November 2022 20:25**

Hey my friends,

as I am new in posting because all I looked for I figured out I am now stuck somewhere related to this topic and hope you can help.

I have a python script which provides by regulare speedtests a csv (speedtest.csv) with 6 values like the following:

Date,Time,Ping (ms),Jitter (ms),Download (Mbps),Upload (Mbps)
11/09/22,07:40,9.93,3.37,925.84,48.61
...

As the script runs every 15 minutes there are several lines and I am interested in the last one to be transfered into domoticz. Here I only care about Download, Upload and Ping.

Due to this I tried to transfer the here provided dzVents to my needs but got stuck with the following:

Code: Select all

--[[ 

    get values from csv file and store in virtual sensors
    (dzVents >= 2.4.19)

[01] Date
[02] Time
[03] Ping (ms)
[04] Jitter (ms)
[05] Download (Mbps)
[06] Upload (Mbps)

]]--

return {
        on = 
        {
            timer     = { 'every minute' }, -- change to the interval suited for your situation
        },

        logging = 
        {
            level = domoticz.LOG_DEBUG, -- set to LOG_ERROR when script is working as expected
            marker = 'read CSV data' 
        },

    execute = function( dz )
        local csvDataFile = '/home/pi/speedtest/speedtest.csv'  -- location of csv file on raspberry
		local sep = "," -- define seperated as comma
        local Dld = dz.devices('Download')  -- created as virtual Custom sensor
        local Uld = dz.devices('Upload') -- created as virtual Custom sensor
        local Ping = dz.devices('Ping')  -- created as virtual Custom sensor

        local function readCSVLine(OSfile)
            local f = assert(io.popen('tail -1 '.. OSfile)) -- OS command to read last line of file
            local line = assert(f:read('*a'))               -- read the complete result into string variable
            f:close() -- close the file
            dz.log('line: ' .. line,dz.LOG_DEBUG)
			local csvDataTable = line:split(sep)
        end

        local function updateData(csvData)
            Dld.updateCustomSensor(csvData[5])  -- Update custom sensor with value from 5th column
            Uld.updateCustomSensor(csvData[6]) -- Update custom sensor with value from 6th column
            Ping.updateCustomSensor(csvData[3]) -- Update custom sensor with value from 3rd column
        end
    end
}

Currently I dont get any errors but unfortunately nothing got transfered into domoticz as the dummy devices do not get any data.

My goal is to store every 15 minutes the last line in speedtest.csv to the devices so I get an overview of my speed data into domoticz.

I think I am not that far away but do something wrong by transformation of values and am missing timestamp as well but am not sure if domoticz registers the change with timestamp anyway?

Hope anyone can help me solve this!

Thanks and best

Markus

Posted: **Monday 14 November 2022 10:24**

Use dzvents .stringSplit function:
stringSplit(string [,separator ] [,convertNumber ][,convertNil ]3.0.17):Function. Helper function to split a line in separate words. Default separator is space. Return is a table with separated words. Default convertNumber is false when set to true it will convert strings to number where possible. Word is ignored when nil, unless convertNil is set; in that case word will be set to the convertNil value.

eg

Code: Select all

local csvDataTable = domoticz.utils.stringSplit(line , ',')

Posted: **Monday 14 November 2022 11:57**

If I got you right, the mentioned statement is missing or do I have to exchange something from my code? Maybe you can add the line where you would recommend to integrate it into my code?

Posted: **Monday 14 November 2022 13:08**

I think there is a lot missing. You for example miss the main program to start all your sub functions. That is why it is not doing anything only declaring local functions.

You should for example add the main section to the end of your script, see also the previous scripts in this thread:

Code: Select all

       -- main
        csvData = readCSVLine(dataFile)
	updateData(csvData)

You should change line

Code: Select all

local csvDataTable = line:split(sep)

with

Code: Select all

local csvDataTable = domoticz.utils.stringSplit(line , ',')

This changes will probably give some errors but when adding some logging you will get there fixing those.

Posted: **Monday 14 November 2022 13:26**

Damn, I removed the line by mistake because previously there was a time and date comparison which I didnt need - thanks for pointing this out, I will come back as soon as I checked your lines.

Thanks up to now!

Posted: **Monday 14 November 2022 22:30**

I now end up at least with the prepared error messages, but do not understand why - from my point of view it should work now, where am I wrong?

Code: Select all

return {
        on = 
        {
            timer     = { 'every minute' }, 					-- change to the interval suited for your situation
        },

        logging = 
        {
            level = domoticz.LOG_ERROR, 				-- set to LOG_ERROR when script is working as expected
            marker = 'read CSV data' 
        },

    execute = function( dz )
        local csvDataFile = '/home/pi/speedtest/speedtest.csv'  		-- location of csv file on raspberry
        local Dld = dz.devices(64)  							-- created as virtual Custom sensor
        local Uld = dz.devices(65) 								-- created as virtual Custom sensor
        local Ping = dz.devices(66)  							-- created as virtual Custom sensor

        local function readCSVLine(OSfile)
	local f = assert(io.popen('tail -1 '.. OSfile)) 				-- OS command to read last line of file
	local line = assert(f:read('*a'))               					-- read the complete result into string variable
	local csvDataTable = dz.utils.stringSplit(line ,',')				-- Separate by line and comma
	f:close() 											-- close the file
	dz.log('line: ' .. line,dz.LOG_ERROR)
	
        end
		
        local function isRecent(csvData)
            if csvData == nil then 
                dz.log("Problem with reading and converting to table of the data" ,dz.LOG_ERROR)
                return
            end
		end
			local function updateData(csvData)
            Dld.updateCustomSensor(csvData[5])  		-- Update custom sensor with value from 5th column
            Uld.updateCustomSensor(csvData[6]) 		-- Update custom sensor with value from 6th column
            Ping.updateCustomSensor(csvData[3]) 		-- Update custom sensor with value from 3rd column
			end
		-- main
		csvData = readCSVLine(csvDataFile)
		
		if isRecent(csvData) then
		updateData(csvData)
		else
            dz.log("Something was not OK. Skipping device updates",dz.LOG_ERROR)
        end
	end
}

The error I get in domoticz log is - I can see the data read from the file correctly, but it is not going to be separated and converted to table as I think?!

Code: Select all

2022-11-14 22:15:00.201 Error: dzVents: Error: (3.1.8) read CSV data: line: 11/14/22,22:00,178.88,249.85,98.35,0.28
2022-11-14 22:15:00.201
2022-11-14 22:15:00.201 Error: dzVents: Error: (3.1.8) read CSV data: Problem with reading and converting to table of the data
2022-11-14 22:15:00.201 Error: dzVents: Error: (3.1.8) read CSV data: Something was not OK. Skipping device updates

Thanks for any advice!

Posted: **Tuesday 15 November 2022 0:17**

I suppose you better start with setting the right indentation so you know what lines belongs to what function.

I think you miss or have to much 'end' statements.

Posted: **Tuesday 15 November 2022 23:10**

Actually I dont make it through. I tried to transfer the needed "ends" from code above but always end up in "unexpected symbol near '}' " in last line which means for me I have some end missing.

I resorted the code a bit, can anyone help?

Code: Select all

return {
		on = 
		{
		timer = { 'every minute' }, 								-- change to the interval suited for your situation
		},

		logging = 
		{
		level = domoticz.LOG_ERROR, 								-- set to LOG_ERROR when script is working as expected
		marker = 'read CSV data' 
		},

		execute = function( dz )
		local csvDataFile = '/home/pi/speedtest/speedtest.csv'		-- location of csv file on raspberry
		local Dld = dz.devices(64)									-- created as virtual Custom sensor
		local Uld = dz.devices(65) 									-- created as virtual Custom sensor
		local Ping = dz.devices(66)									-- created as virtual Custom sensor

		local function readCSVLine(OSfile)
		local f = assert(io.popen('tail -1 '.. OSfile)) 			-- OS command to read last line of file
		local line = assert(f:read('*a')) 							-- read the complete result into string variable
		local csvDataTable = dz.utils.stringSplit(line ,',')		-- Separate by line and comma
		f:close() 													-- close the file
		dz.log('line: ' .. line,dz.LOG_ERROR)
		end
		
		local function isRecent(csvData)
		if csvData == nil then 
		dz.log("Problem with reading and converting to table of the data" ,dz.LOG_ERROR)
		end
		
		local function updateData(csvData)
		Dld.updateCustomSensor(csvData[5])							-- Update custom sensor with value from 5th column
		Uld.updateCustomSensor(csvData[6]) 							-- Update custom sensor with value from 6th column
		Ping.updateCustomSensor(csvData[3]) 						-- Update custom sensor with value from 3rd column
		end
		
		-- main
		csvData = readCSVLine(csvDataFile)
		
		if isRecent(csvData) then
		updateData(csvData)
		
		else
		dz.log("Something was not OK. Skipping device updates",dz.LOG_ERROR)
		end
}

Hope we can solve this! Thanks!

Posted: **Wednesday 16 November 2022 16:02**

I think you were missing some ends
Please try this script (did not test it!)

Code: Select all

return {
		on = 
		{
			timer = { 'every minute' }, 								-- change to the interval suited for your situation
		},

		logging = 
		{
			level = domoticz.LOG_ERROR, 								-- set to LOG_ERROR when script is working as expected
			marker = 'read CSV data' 
		},

		execute = function( dz )
			local csvDataFile = '/home/pi/speedtest/speedtest.csv'		-- location of csv file on raspberry
			local Dld = dz.devices(64)									-- created as virtual Custom sensor
			local Uld = dz.devices(65) 									-- created as virtual Custom sensor
			local Ping = dz.devices(66)									-- created as virtual Custom sensor

			local function readCSVLine(OSfile)
				local f = assert(io.popen('tail -1 '.. OSfile)) 			-- OS command to read last line of file
				local line = assert(f:read('*a')) 							-- read the complete result into string variable
				local csvDataTable = dz.utils.stringSplit(line ,',')		-- Separate by line and comma
				f:close() 													-- close the file
				dz.log('line: ' .. line,dz.LOG_ERROR)
			end
		
			local function isRecent(csvData)
				if csvData == nil then 
					dz.log("Problem with reading and converting to table of the data" ,dz.LOG_ERROR)
				end
			end
		
			local function updateData(csvData)
				Dld.updateCustomSensor(csvData[5])							-- Update custom sensor with value from 5th column
				Uld.updateCustomSensor(csvData[6]) 							-- Update custom sensor with value from 6th column
				Ping.updateCustomSensor(csvData[3]) 						-- Update custom sensor with value from 3rd column
			end
		
			-- main
			csvData = readCSVLine(csvDataFile)
		
			if isRecent(csvData) then
				updateData(csvData)
			else
				dz.log("Something was not OK. Skipping device updates",dz.LOG_ERROR)
			end
		end
}

Posted: **Wednesday 16 November 2022 22:47**

I finally did it! Thanks to your help - in fact I was nearer than I thought.

Today I went through line by line as you mentioned to check the "ends" but this wasnt actually the problem or at least not the only thing missing. Especially I missed some returns and return true statements which where necessary for the last statement to get executed.

Here is the finally working code:

Code: Select all

return {
		on = 
		{
			timer = { 'every minute' }, 								-- change to the interval suited for your situation
		},

		logging = 
		{
			level = domoticz.LOG_DEBUG, 								-- set to LOG_ERROR when script is working as expected
			marker = 'read CSV data' 
		},

		execute = function( dz )
			local csvDataFile = '/home/pi/speedtest/speedtest.csv'		-- location of csv file on raspberry
			local Dld = dz.devices(64)									-- created as virtual Custom sensor
			local Uld = dz.devices(65) 									-- created as virtual Custom sensor
			local Ping = dz.devices(66)									-- created as virtual Custom sensor

			local function readCSVLine(OSfile)
				local f = assert(io.popen('tail -1 '.. OSfile)) 			-- OS command to read last line of file
				local line = assert(f:read('*a')) 							-- read the complete result into string variable
				f:close() 													-- close the file
				dz.log('line: ' .. line,dz.LOG_DEBUG)
				return dz.utils.stringSplit(line ,',')		-- Separate by line and comma
			end
		
			local function isRecent(csvData)
				if csvData == nil then 
					dz.log("Problem with reading and converting to table of the data" ,dz.LOG_ERROR)
				return
				end
				return true
			end
		
			local function updateData(csvData)
				Dld.updateCustomSensor(csvData[5])							-- Update custom sensor with value from 5th column
				Uld.updateCustomSensor(csvData[6]) 							-- Update custom sensor with value from 6th column
				Ping.updateCustomSensor(csvData[3]) 						-- Update custom sensor with value from 3rd column
			end
		
			-- main
			csvData = readCSVLine(csvDataFile)
		
			if isRecent(csvData) then
				updateData(csvData)
			else
				dz.log("Something was not OK. Skipping device updates",dz.LOG_ERROR)
			end
		end
}

Result looks good as I now can track my speedtest results in domoticz, see attachement. And as we also can see this will help to proof for my provider that this is not a 1mbit connection

)

Thanks for your suggestions and help!

Domoticz

CVS file (on pi) to virtual sensors

Re: CVS file (on pi) to virtual sensors

Re: CVS file (on pi) to virtual sensors

Re: CVS file (on pi) to virtual sensors

Re: CVS file (on pi) to virtual sensors

Re: CVS file (on pi) to virtual sensors

Re: CVS file (on pi) to virtual sensors

Re: CVS file (on pi) to virtual sensors

Re: CVS file (on pi) to virtual sensors

Re: CVS file (on pi) to virtual sensors

Re: CVS file (on pi) to virtual sensors

Re: CVS file (on pi) to virtual sensors

Re: CVS file (on pi) to virtual sensors

Re: CVS file (on pi) to virtual sensors

Re: CVS file (on pi) to virtual sensors

Re: CVS file (on pi) to virtual sensors