Scraping only the temperature from WeatherUnderground sensor

Moderator: leecollings

myozone
Posts: 6
Joined: Monday 19 January 2015 13:03
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: Scraping only the temperature from WeatherUnderground se

Post by myozone »

If this the correct way to control via the temperature
Attachments
blockly-newtemp.jpg
blockly-newtemp.jpg (24.67 KiB) Viewed 6698 times
TallGuy212
Posts: 2
Joined: Monday 09 February 2015 20:21
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: Scraping only the temperature from WeatherUnderground se

Post by TallGuy212 »

I am wondering what the syntax is to scrape the humidity from the same device. Getting the temperature was no problem but it proves difficult to scrape the next value. I guess it has to do with being a % instead of a temperature.

What is correct syntax?
DutchHans
Posts: 230
Joined: Friday 03 April 2015 20:44
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: Germany (near dutch border)
Contact:

Re: Scraping only the temperature from WeatherUnderground se

Post by DutchHans »

Hi Tallguy,

Did you figure it out? I am very interested and struggling with the same.
Regards,
Hans
User avatar
jcjames13009
Posts: 33
Joined: Monday 09 March 2015 15:24
Target OS: Raspberry Pi / ODroid
Domoticz version: 2023.2
Location: France / Marseille
Contact:

Re: Scraping only the temperature from WeatherUnderground se

Post by jcjames13009 »

Hi Hans,

I used the following command in my lua script to scrap Temperature and Hulidity from my WU sensor

Code: Select all

sWeatherTemp, sWeatherHumidity, sWeatherHumidityStat, sWeatherBarometer, sWeatherBarometerFor = otherdevices_svalues[capteurwu1]:match("([^;]+);([^;]+);([^;]+);([^;]+);([^;]+)")
commandArray[1]={['UpdateDevice'] = idx1 .. '|0|' .. tostring(sWeatherTemp)}
commandArray[2]={['UpdateDevice'] = idx2 .. '|' .. tostring(sWeatherHumidity) .. '|' .. '|' .. tostring(sWeatherHumidityStat) .. '|'}
And I have this in my Domoticz
Capture8.PNG
Capture8.PNG (23.18 KiB) Viewed 6617 times
Hope it'll help

JC
DutchHans
Posts: 230
Joined: Friday 03 April 2015 20:44
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: Germany (near dutch border)
Contact:

Re: Scraping only the temperature from WeatherUnderground se

Post by DutchHans »

Hello all,

There is something strange.. when I look in the Logfile the lua script is not triggered.
I am not a programmer, I try to read code and understand it. I have tried to change the WU script for my needs... but I have no idea

But am I missing something?
The WU script is triggered, thats what the log says, but my Cresta lua file isnt.
What am i doing wrong?

Code: Select all

--script_device_tempCresta.lua
local sensorLR = 'ThermoRFXCOM' --Original sensor of wunderground carrying the 0001 ID 
local idx = 102 --idx of your manual created virtual temp sensor
commandArray = {}
if devicechanged[sensorLR] then
sWeatherTemp, sWeatherHumidity = otherdevices_svalues[sensorLR]:match("([^;]+);([^;]+)")
--sWeatherTemp = tonumber(sWeatherTemp)
commandArray['UpdateDevice'] = idx .. '|0|' .. tostring(sWeatherTemp)
end

return commandArray
Hope someone can help me out.
Kind regards, Hans
User avatar
Domosapiens
Posts: 232
Joined: Wednesday 20 August 2014 12:08
Target OS: Windows
Domoticz version: V3.5981
Location: NL
Contact:

Re: Scraping only the temperature from WeatherUnderground se

Post by Domosapiens »

Are you sure that this is the correct IDX of your dummy temp sensor ?
It looks exact like the number from the Wiki
local idx = 102 --idx of the virtual temperature sensor you created above
:?
Domosapiens
Win Vista&7; 1#Aeon Z-Stick S2; 1#Aeotec Z-Sick Gen5, 6#Fib.FGBS001; 24#DS18B20; 8#Everspr.AN158-2; 3#Philio PAN04; 1#Philio PAN06, 1#YouLess El; 1#Fib.FGWPE; 1#ZME_RC2; 2#FAK_ZWS230, 2#Quib.ZMNHCDx, 1#Quib.ZMNHDD1, 7#EM6555
User avatar
Siewert308SW
Posts: 290
Joined: Monday 29 December 2014 15:47
Target OS: Raspberry Pi / ODroid
Domoticz version: Stable
Location: The Netherlands
Contact:

Re: Scraping only the temperature from WeatherUnderground se

Post by Siewert308SW »

DutchHans wrote:Hello all,

There is something strange.. when I look in the Logfile the lua script is not triggered.
I am not a programmer, I try to read code and understand it. I have tried to change the WU script for my needs... but I have no idea

But am I missing something?
The WU script is triggered, thats what the log says, but my Cresta lua file isnt.
What am i doing wrong?

Code: Select all

--script_device_tempCresta.lua
local sensorLR = 'ThermoRFXCOM' --Original sensor of wunderground carrying the 0001 ID 
local idx = 102 --idx of your manual created virtual temp sensor
commandArray = {}
if devicechanged[sensorLR] then
sWeatherTemp, sWeatherHumidity = otherdevices_svalues[sensorLR]:match("([^;]+);([^;]+)")
--sWeatherTemp = tonumber(sWeatherTemp)
commandArray['UpdateDevice'] = idx .. '|0|' .. tostring(sWeatherTemp)
end

return commandArray
Hope someone can help me out.
Kind regards, Hans
Added a cresta sensor from my neighbors to Domoticz
Added a new virtual temp sensor
and adjusted the current names in de SensorWU script for testing.
Seems it just works fine, you could test it.

Code: Select all


--script_device_sensorCR.lua
local sensorcr = 'Weerstation' --name of the sensor that gets created when you add the Cresta device (and that contains multiple values like temperature, humidity, barometer etc)
local idx = 327 --idx of the virtual temperature sensor you created for scrapping the temperature
commandArray = {}
 
if devicechanged[sensorcr] then
	sCrestaTemp, sCrestaHumidity, sCrestaPressure = otherdevices_svalues[sensorcr]:match("([^;]+);([^;]+);([^;]+)")
	sCrestaTemp = tonumber(sCrestaTemp)
	commandArray['UpdateDevice'] = idx .. '|0|' .. tostring(sCrestaTemp)
end
return commandArray
Setup:
- RPi4 - Domo Stable / Aeotec Z-stick7 / PiHole Unbound Gemini
- RPi4 - PiHole / PiVPN Unbound Gemini
- Synology DS923+ / DS218j
- P1 Gas/Power, SmartGateway watermeter
- Fibaro switches, contacts, plugs, smoke/Co2 ect
- rootfs @ USB HDD
DutchHans
Posts: 230
Joined: Friday 03 April 2015 20:44
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: Germany (near dutch border)
Contact:

Re: Scraping only the temperature from WeatherUnderground se

Post by DutchHans »

Hello Siewert308SW

FANTASTIC!
It works perfectly


Im starting to understand how it works now.
Thank you very much.
h143
Posts: 27
Joined: Tuesday 28 October 2014 22:26
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: Scraping only the temperature from WeatherUnderground se

Post by h143 »

how can i scraping the humidity to a virtual humidity sensor?
the value is always 0 i have the following script:

Code: Select all

--script_device_sensorCR.lua
local sensorcr = 'Weerstation' --name of the sensor that gets created when you add the Cresta device (and that contains multiple values like temperature, humidity, barometer etc)
local idx = 435 --idx of the virtual temperature sensor you created for scrapping the temperature
commandArray = {}
 
if devicechanged[sensorcr] then
   sCrestaTemp, sCrestaHumidity, sCrestaPressure = otherdevices_svalues[sensorcr]:match("([^;]+);([^;]+);([^;]+)")
   sCrestaHumidity = tonumber(sCrestaHumidity)
   commandArray['UpdateDevice'] = idx .. '|0|' .. tostring(sCrestaHumidity)
end
return commandArray
tlpeter
Posts: 191
Joined: Wednesday 26 November 2014 18:43
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: Netherlands
Contact:

Re: Scraping only the temperature from WeatherUnderground se

Post by tlpeter »

Why define a new name instead of using the real device name in the script?
jannl
Posts: 666
Joined: Thursday 02 October 2014 6:36
Target OS: Raspberry Pi / ODroid
Domoticz version: 2022.2
Location: Geleen
Contact:

Re: Scraping only the temperature from WeatherUnderground se

Post by jannl »

Like Peter says.
Start using the actual device name.
Print the value to the log in domoticz and check if it works first. (should work)
spudgunman
Posts: 32
Joined: Tuesday 26 May 2015 18:59
Target OS: Raspberry Pi / ODroid
Domoticz version: beta
Location: Seattle WA, USA
Contact:

Re: Scraping only the temperature from WeatherUnderground se

Post by spudgunman »

note sure what I am missing here - new to this whole project and wondering where to troubleshoot next, I have also tried this on the SVN build with same results (the results that my virtual temp device wont update)

version 2.2492

Code: Select all

2015-05-27 21:10:00.430 EventSystem: Script event triggered: /home/pi/domoticz/scripts/lua/script_device_sensorWU.lua
2015-05-27 21:10:00.420 (WeatherUnderground) Temp + Humidity + Baro (WUCombo)
2015-05-27 21:10:00.440 Hardware Monitor: Fetching data (System sensors)
2015-05-27 21:10:00.437 (WeatherUnderground) Wind (Wind)
2015-05-27 21:10:00.450 (WeatherUnderground) UV (UV)
2015-05-27 21:10:00.476 (WeatherUnderground) Rain (Rain)
Capture.JPG
Capture.JPG (71.85 KiB) Viewed 6457 times

Code: Select all

local sensorwu = 'WUCombo' --name of the sensor that gets created when you add the WU device (and that contains multiple values like $
local idx = 14065 --idx of the virtual temperature sensor you created above
commandArray = {}

if devicechanged[sensorwu] then
        sWeatherTemp, sWeatherHumidity, sWeatherPressure = otherdevices_svalues[sensorwu]:match("([^;]+);([^;]+);([^;]+)")
        sWeatherTemp = tonumber(sWeatherTemp)
        commandArray['UpdateDevice'] = idx .. '|0|' .. tostring(sWeatherTemp)
end
return commandArray
spudgunman
Posts: 32
Joined: Tuesday 26 May 2015 18:59
Target OS: Raspberry Pi / ODroid
Domoticz version: beta
Location: Seattle WA, USA
Contact:

Re: Scraping only the temperature from WeatherUnderground se

Post by spudgunman »

AI hacked up the scrip to pull all data.
updated May 28

Code: Select all

--Script To Parse WeatherUnderground Multi-Value Sensor, Additionally using PWS: system from WU with a new output format
--This script assumes the output (which can be viewed in events show current state button) is like this 19.5;79;3;1019;3 (temp;humidity;null;pressure;null)
--more details at this wiki http://www.domoticz.com/wiki/Virtual_weather_devices
--
--The following need updated for your enviroment get the 'Idx' or 'Name' off the Device tab. By default only the Temp is 'uncommented or enabled' in this script.
local sensorwu = 'ChangeME' --name of the sensor that gets created when you add the WU device (and that contains multiple values like temperature, humidity, barometer etc)
local idxt = 999 --idx of the virtual temperature sensor you need to change this to your own Device ID
local idxh = 999 --idx of the virtual humidity sensor you need to change this to your own Device ID
local idxp = 999 --idx of the virtual pressure sensor you need to change this to your own Device ID
--

commandArray = {}

if devicechanged[sensorwu] then
        sWeatherTemp, sWeatherHumidity, sHumFeelsLike, sWeatherPressure = otherdevices_svalues[sensorwu]:match("([^;]+);([^;]+);([^;]+);([^;]+);([^;]+)")
        sWeatherTemp = tonumber(sWeatherTemp)
        sWeatherHumidity = tonumber(sWeatherHumidity)
        sWeatherPressure = tonumber(sWeatherPressure)
        --parseDebug = ('WU Script Parsed Temp=' .. sWeatherTemp .. ' Humidity=' .. sWeatherHumidity .. ' Pressure=' .. sWeatherPressure)
        --print(parseDebug)

        commandArray[1] = {['UpdateDevice'] = idxt .. '|0|' .. sWeatherTemp}
        --commandArray[2] = {['UpdateDevice'] = idxh .. '|' .. tostring(sWeatherHumidity) .. '|' .. tostring(sHumFeelsLike)}
        --commandArray[3] = {['UpdateDevice'] = idxp .. '|0|' .. sWeatherPressure}
end

return commandArray
Last edited by spudgunman on Tuesday 02 June 2015 2:40, edited 1 time in total.
spudgunman
Posts: 32
Joined: Tuesday 26 May 2015 18:59
Target OS: Raspberry Pi / ODroid
Domoticz version: beta
Location: Seattle WA, USA
Contact:

Re: Scraping only the temperature from WeatherUnderground se

Post by spudgunman »

spudgunman wrote:note sure what I am missing here - new to this whole project and wondering where to troubleshoot next, I have also tried this on the SVN build with same results (the results that my virtual temp device wont update)
found my error I was using the ID and not the idx
spudgunman
Posts: 32
Joined: Tuesday 26 May 2015 18:59
Target OS: Raspberry Pi / ODroid
Domoticz version: beta
Location: Seattle WA, USA
Contact:

Re: Scraping only the temperature from WeatherUnderground se

Post by spudgunman »

So my script 2 posts up to parse all the data (humidity and pressure) the script is working but I think there is a bug now..

I see all values in the events show current states screen
sensor-issues.JPG
sensor-issues.JPG (30.97 KiB) Viewed 6439 times
but the virtual humidity and pressure are at 0 on the device tab
sensor-issues-2.JPG
sensor-issues-2.JPG (31.16 KiB) Viewed 6439 times
and wacky on the dash
sensor-issues-3.JPG
sensor-issues-3.JPG (18.94 KiB) Viewed 6439 times
Jem101
Posts: 66
Joined: Saturday 02 May 2015 21:26
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: Bexleyheath, SE London, UK
Contact:

Re: Scraping only the temperature from WeatherUnderground se

Post by Jem101 »

I have to say, I've tried many variations of these scripts and finally managed to get one to run properly without logging errors, but I get the same results as spudgunman above. The temperature scrapes and works OK, the virtual humidity sensor gets reset to 0 (and stays at 0), the pressure appears to work OK but i have the same dashboard problem as above.

I've tried deleting and recreating the virtual sensors many times but I always get the same results.
I firmly believe in having an open mind - although not so open that your brain falls out!

- sometimes attributed to Carl Sagan, but the original quote is much older than that
User avatar
G3rard
Posts: 669
Joined: Wednesday 04 March 2015 22:15
Target OS: -
Domoticz version: No
Location: The Netherlands
Contact:

Re: Scraping only the temperature from WeatherUnderground se

Post by G3rard »

You have to change to script a bit for the humidity, see the last line from (a part of) my code below. I am not using it for the pressure, so I can't help you with that, but maybe it works the same way.

Code: Select all

sWeatherTempwu, sWeatherHumiditywu, sWeatherPressurewu = otherdevices_svalues[sensorwu]:match("([^;]+);([^;]+);([^;]+)")
sWeatherTempwu = tonumber(sWeatherTempwu)
sWeatherHumiditywu = tonumber(sWeatherHumiditywu)	
if (sWeatherHumiditywu > 60 ) then
   verw = 3 -->60 = 3 wet
elseif (sWeatherHumiditywu  < 25 ) then
   verw = 2 --<25 = 2 dry
else
   verw = 1 -->=25 of <=60 = 1 comfortable
end
commandArray[2]={['UpdateDevice'] = idxwut .. '|0|' .. tostring(sWeatherTempwu)}
commandArray[3]={['UpdateDevice'] = idxwuh .. '|' .. tostring(sWeatherHumiditywu) .. '|' .. tostring(verw)}
Not using Domoticz anymore
spudgunman
Posts: 32
Joined: Tuesday 26 May 2015 18:59
Target OS: Raspberry Pi / ODroid
Domoticz version: beta
Location: Seattle WA, USA
Contact:

Re: Scraping only the temperature from WeatherUnderground se

Post by spudgunman »

ok that helps a ton - i updated my script to work with humidity now.. all that is remaining is to figure out pressure and what it expects.. so far the same trick of using 1,2,3 gave funny results

Code: Select all

--Script To Parse WeatherUnderground Multi-Value Sensor, Additionally using PWS: system from WU with a new output format
--This script assumes the output (which can be viewed in events show current state button) is like this 19.5;79;3;1019;3 (temp;humidity;null;pressure;null)
--more details at this wiki http://www.domoticz.com/wiki/Virtual_weather_devices
--
--The following need updated for your enviroment get the 'Idx' or 'Name' off the Device tab. By default only the Temp is 'uncommented or enabled' in this script.
local sensorwu = 'ChangeME' --name of the sensor that gets created when you add the WU device (and that contains multiple values like temperature, humidity, barometer etc)
local idxt = 999 --idx of the virtual temperature sensor you need to change this to your own Device ID
local idxh = 999 --idx of the virtual humidity sensor you need to change this to your own Device ID
local idxp = 999 --idx of the virtual pressure sensor you need to change this to your own Device ID
--

commandArray = {}

if devicechanged[sensorwu] then
        sWeatherTemp, sWeatherHumidity, sHumFeelsLike, sWeatherPressure = otherdevices_svalues[sensorwu]:match("([^;]+);([^;]+);([^;]+);([^;]+);([^;]+)")
        sWeatherTemp = tonumber(sWeatherTemp)
        sWeatherHumidity = tonumber(sWeatherHumidity)
        sWeatherPressure = tonumber(sWeatherPressure)
        --parseDebug = ('WU Script Parsed Temp=' .. sWeatherTemp .. ' Humidity=' .. sWeatherHumidity .. ' Pressure=' .. sWeatherPressure)
        --print(parseDebug)

        commandArray[1] = {['UpdateDevice'] = idxt .. '|0|' .. sWeatherTemp}
        --commandArray[2] = {['UpdateDevice'] = idxh .. '|' .. tostring(sWeatherHumidity) .. '|' .. tostring(sHumFeelsLike)}
        --commandArray[3] = {['UpdateDevice'] = idxp .. '|0|' .. sWeatherPressure}
end

return commandArray
Jem101
Posts: 66
Joined: Saturday 02 May 2015 21:26
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: Bexleyheath, SE London, UK
Contact:

Re: Scraping only the temperature from WeatherUnderground se

Post by Jem101 »

Yes indeed using the code above, I've also managed to get the virtual Humidity sensor working and giving the proper values but just like spudgunman the Pressure value is still giving problems.

So far, I've created a virtual 'Barometer' sensor which appears in devices, I've setup the script to scrape the value for 'pressure' from WU and this runs without showing any errors. In the 'devices' tab, I can see that the value has updated and does show the correct figure, which, I assume, means that the scraping is working OK. However I get the exact same odd value in the Dashboard as spudgunman posted earlier.

It looks as if the dashboard code is expecting different data (or a different format) to what it is receiving. Oddly enough I tried creating a virtual generic 'Pressure' sensor, plugged the idx for this into the script and this worked properly. The only problem is that this sensor appears in the 'Utilities' tab rather than the Weather one.

I'm wondering if the issue is something to do with the forecast field. I've notice that the output from WU will be something like 1004 (the pressure value) followed by a forecast, wet for example. So might the Barometer virtual sensor be expecting additional data (which isn't being scraped properly so it just doesn't display correctly) - or maybe I'm just talking complete rubbish!

Any help or suggestions would be gratefully received.
I firmly believe in having an open mind - although not so open that your brain falls out!

- sometimes attributed to Carl Sagan, but the original quote is much older than that
spudgunman
Posts: 32
Joined: Tuesday 26 May 2015 18:59
Target OS: Raspberry Pi / ODroid
Domoticz version: beta
Location: Seattle WA, USA
Contact:

Re: Scraping only the temperature from WeatherUnderground se

Post by spudgunman »

I have the device updating with the following but the graphic is still messed up and wont display on the dash at all, I tried a ton of different combos' according to the git-source the prediction text should not matter in the web.js it should take it either way but its not?

Code: Select all

sPressiureFeelsLike = 3 --basically just a placeholder I dont know what this should be
commandArray[3] = {['UpdateDevice'] = idxp .. '|' .. sPressiureFeelsLike .. '|' .. sWeatherPressure}
void SendBaroSensor(const int NodeID, const int ChildID, const int BatteryLevel, const float pressure, const int forecast);

But this isnt decoding still...
Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest