Weather Underground Upload script Topic is solved

[Toulon7559]

@jms3700

If you have correctly set ID + Password for Wunderground, with the selected interval you should see a 'green' section in the log, similar to the Example log-screenshot from my actual script-version script_time_wu101_nexus_upload6a.lua is below (with my WU_Password replaced by XXXXX)
The first timed line is result of a print-command somewhere halfway in my script-version.
The second timed line is result of the last print-command in the script.
'Green' meaning successful execution & upload.
If this 'green' section is never shown in your log, there is an error in the script, probably in the insertion of WU_ID , WU_Password, or an error in a reference to sensor-data needed to fill the WU_URL. But for the latter error you probably will see a specific error-report referring to a line earlier in the script.

Code: Select all

 
2016-05-29 14:25:03.257 LUA: WU_URL = http://weatherstation.wunderground.com/weatherstation/updateweatherstation.php?ID=IOVERIJS101&PASSWORD=XXXXX&dateutc=2016-05-29+12%3A25%3A03&tempf=68.7&humidity=76&dewptf=60.8&baromin=29.78
2016-05-29 14:25:03.257 LUA: http://weatherstation.wunderground.com/weatherstation/updateweatherstation.php?ID=IOVERIJS101&PASSWORD=XXXXX&dateutc=2016-05-29+12%3A25%3A03&tempf=68.7&humidity=76&dewptf=60.8&baromin=29.78&dailyrainin=0.00&rainin=0.00&winddir=101&windspeedmph=2&windgustmph=0&softwaretype=Domoticz&action=updateraw
2016-05-29 14:25:03.258 EventSystem: Fetching url...
2016-05-29 14:25:03.258 EventSystem: Script event triggered: /home/pi/domoticz/scripts/lua/script_time_wu101_nexus_upload6a.lua 

[jms3700]

here a extract of my logging

2016-05-29 15:20:00.237 LUA: http://weatherstation.wunderground.com/ ... =updateraw
2016-05-29 15:20:00.237 EventSystem: Fetching url...
2016-05-29 15:20:00.237 EventSystem: Script event triggered: /home/pi/domoticz/scripts/lua/script_time_ws.lua

the lines Eventsystem are also in green in my log. The update is done to weatherstation underground by the script.
So parameters ID and password are correct, also the devices data update.
REMARK: BUT i must assign myself the value in the line commandArray['OpenURL']='http://......................................
This string is the value of the command print(WU_URL), That I copy from my logging file

the last lines of the script

print(WU_URL)

commandArray = {}

--remove the line below to actualy upload it

commandArray['OpenURL']=WU_URL

commandArray['OpenURL'] = 'http://weatherstation.wunderground.com/weatherstatio$.............................

return commandArray

[Toulon7559]

@jms3700

It looks as if an error exists in the last lines which proceed the transmission of WU_URL.

Please try this for the last 5 lines in your script:

WU_URL = WU_URL .. "&softwaretype=" .. SoftwareType .. "&action=updateraw"

print (WU_URL)

commandArray = {}

commandArray['OpenURL']=WU_URL

return commandArray

[jms3700]

Hi,

I deleted all the lines after WU_URL = WU_URL ..
and pasted from your reply the last 4 lines.
the WU_URL line must be correct, when I copy the result from the logging in Chrome: Weather station return with "success"

nothing changed for the update.

again I created the hardcoded line with

commandArray['OpenURL']='http:// ..................

and every minute weatherstation is updated obviously with te same value.

for your information: initial I started the script by pasting it from a post of this forum

very strange?

[Toulon7559]

Puzzled, because indeed very strange that apparently correct script-lines do not result in a correct upload.

[jms3700]

Hi Toulon 7559

I started again from scrap the script and now it is working fine!
Probably there was a control or hidden character in the script
Thanks for helping

jms3700

[MikeF]

I've just successfully set up the script and uploaded data to a WU PWS. I'm using an Adafruit BME380 in MySensors, which reports temp + humidity + baro all on the same node. This is defined in the script as:

Code: Select all

Outside_Temp_Hum = 'Node 3'
Barometer = 'Node 3'

and I'm using the original barometer section:

Code: Select all

if Barometer ~= '' then
      WU_URL = WU_URL .. "&baromin=" .. string.format("%2.2f", hPatoInches(otherdevices_barometer[Barometer]))
end

(i.e., I didn't need to do the svalues mod for pressure).

[henkkay]

I did not know anything about scripting until i tried to get this script to work. A few days later it works after i found out some rookie errors (the rookie being me of course). It was a Nice journey though.

one tip you which you may find useful getting the script to work:

do not use your password for WU but the station key you get with your ID for your PWS and set your station key as the password in the script.

[Toulon7559]

@henkkay
What you now have learned 'the hard way' is called 'tuning of a script to your personal configuration', as clearly hinted in this thread as a requirement in the first message and in various later messages.
The red line from 2) and 3) of your message is correct that you have to be very consequent when tuning, not forgetting any later link or any later reference: that is called programming and code testing, and there are many ways to achieve a working software .........
In that perspective, please delete your message, because it is in no way complete and therefore may only confuse.

[IanDury]

With some slight changes this script can also be used for Met Office WOW.
For more information see: https://wow.metoffice.gov.uk/ and (in Dutch) https://wow.knmi.nl/

Code: Select all

-- Met Office WeatherObservationsWebsite (WOW) AWS upload script
-- Based on (C)2013 GizMoCuz script for Weatherunderground
-- 2016 IanDury

date = os.date("*t")
if (date.min % 15 == 7) then
-- start script every 15 minutes with an offset of 7 minutes

Outside_Temp_Hum = 'MyTempHumBaro'
Barometer = 'MyTempHumBaro'
RainMeter = 'MyRain'
WindMeter = 'MyWind'

-- WOW Settings
baseurl = "http://wow.metoffice.gov.uk/automaticreading?"
SiteID = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
AWSPin = "yyyyyy"

local function CelciusToFarenheit(C)
   return (C * (9/5)) + 32
end

local function hPatoInches(hpa)
   return hpa * 0.0295301
end

local function mmtoInches(mm)
   return mm * 0.039370
end

utc_dtime = os.date("!%m-%d-%y %H:%M:%S",os.time())

month = string.sub(utc_dtime, 1, 2)
day = string.sub(utc_dtime, 4, 5)
year = "20" .. string.sub(utc_dtime, 7, 8)
hour = string.sub(utc_dtime, 10, 11)
minutes = string.sub(utc_dtime, 13, 14)
seconds = string.sub(utc_dtime, 16, 17)

timestring = year .. "-" .. month .. "-" .. day .. "+" .. hour .. "%3A" .. minutes .. "%3A" .. seconds

SoftwareType="Domoticz"

WU_URL= baseurl .. "siteid=" .. SiteID .. "&siteAuthenticationKey=" .. AWSPin .. "&dateutc=" .. timestring

if Outside_Temp_Hum ~= '' then
   WU_URL = WU_URL .. "&tempf=" .. string.format("%3.1f", CelciusToFarenheit(otherdevices_temperature[Outside_Temp_Hum]))
   WU_URL = WU_URL .. "&humidity=" .. otherdevices_humidity[Outside_Temp_Hum]
   WU_URL = WU_URL .. "&dewptf=" .. string.format("%3.1f", CelciusToFarenheit(otherdevices_dewpoint[Outside_Temp_Hum]))
end

if Barometer ~= '' then
   WU_URL = WU_URL .. "&baromin=" .. string.format("%2.2f", hPatoInches(otherdevices_barometer[Barometer]))
end

if RainMeter ~= '' then
   WU_URL = WU_URL .. "&dailyrainin=" .. string.format("%2.2f", mmtoInches(otherdevices_rain[RainMeter]))
   WU_URL = WU_URL .. "&rainin=" .. string.format("%2.2f", mmtoInches(otherdevices_rain_lasthour[RainMeter]))
end

if WindMeter ~= '' then
   WU_URL = WU_URL .. "&winddir=" .. string.format("%.0f", otherdevices_winddir[WindMeter])
   WU_URL = WU_URL .. "&windspeedmph=" .. string.format("%.0f", (otherdevices_windspeed[WindMeter]/0.1)*0.223693629205)
   WU_URL = WU_URL .. "&windgustmph=" .. string.format("%.0f", (otherdevices_windgust[WindMeter]/0.1)*0.223693629205)
end

WU_URL = WU_URL .. "&softwaretype=" .. SoftwareType

-- print (WU_URL)

commandArray = {}

-- remove the comment in the line below to actually upload it
commandArray['OpenURL']=WU_URL

return commandArray

-- end of script

else
  commandArray = {}
  return commandArray
end

[henkkay]

@Toulon7559
Thnx for your comment. Much appreciated! The last thing i want is to confuse people so I changed my first post on this forum (!) and left only the text about the password because it has Nothing to do with scripting and can take people on a search for a long time.

[henkkay]

@IanDury thnx. Just what is was searching for!

[Toulon7559]

@IanDury

Thanks!
A nice solution for upload to WOW, especially because it is 'more-of-the-same', which makes 'tuning & maintenance' easier .......

Addition 13NOV2016, updated/modified 14NOV2016 by (text):
At the WOW-Group at Google it seemed that both AWS-Pin and System-ID no longer are number-strings, but text-strings. Which is contrary to the info at http://wow.metoffice.gov.uk/support/dat ... #automatic
In response to a direct question WOW_UK responded: that after Login and selection of 'My Sites',
the (first) ID string under the map is the unique Site ID for that particular station, and is the one you need to use.
The second User ID string is your personal user account ID (= NOT the Authentication Key!). While you can have many sites with different Site ID's managed by you, your User ID does not change.
(The AWS Pin and the Authentication Key are one and the same thing, and consist of a 6-digit number of your own choice, to be inserted in the related field at 'Edit Site')

Status 15NOV2016:
PWS is visible on the WOW-Map, which proves that settings and script are OK.

[IanDury]

My (still) working WoW upload script uses a 6 digit number as the AWS pin that I selected myself on the "Edit a Site" page in the Authentication Key field.
The Site ID for a station is indeed the string right below the station's map and site name. In my case it has the hexadecimal form:
xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

Don't forget to remove the brackets.

Edit 15 Nov. 2016:
@Toulon7559 - Congratulations with your working WoW weather station!

[Toulon7559]

@IanDury

Meanwhile from MetOffice a clear response exactly describing your info:
will update my earlier message by alignment and replacing the 'question-marks' by 'positive' information.

[IanDury]

The Weather Underground Upload script can also be used for PWSweather.com. Basically al you need to change is the baseurl and probably the ID and PASSWORD unless they are chosen exactly the same.

Code: Select all

-- PWSweather.com upload script
-- Based on (C)2013 GizMoCuz script for Weatherunderground
-- 2016 IanDury

date = os.date("*t")
if (date.min % 10 == 9) then
-- start script every 10 minutes with an offset of 9 minutes

Outside_Temp_Hum = 'MyTempHumBaro'
Barometer = 'MyTempHumBaro'
RainMeter = 'MyRain'
WindMeter = 'MyWind'
UVMeter = 'MyUV'

-- PWSweather.com Settings
baseurl = "http://www.pwsweather.com/pwsupdate/pwsupdate.php?"
ID = "yourstationid"
PASSWORD = "yourpassword"

local function CelciusToFarenheit(C)
   return (C * (9/5)) + 32
end

local function hPatoInches(hpa)
   return hpa * 0.0295301
end

local function mmtoInches(mm)
   return mm * 0.039370
end

utc_dtime = os.date("!%m-%d-%y %H:%M:%S",os.time())

month = string.sub(utc_dtime, 1, 2)
day = string.sub(utc_dtime, 4, 5)
year = "20" .. string.sub(utc_dtime, 7, 8)
hour = string.sub(utc_dtime, 10, 11)
minutes = string.sub(utc_dtime, 13, 14)
seconds = string.sub(utc_dtime, 16, 17)

timestring = year .. "-" .. month .. "-" .. day .. "+" .. hour .. "%3A" .. minutes .. "%3A" .. seconds

SoftwareType="Domoticz"

WU_URL= baseurl .. "ID=" .. ID .. "&PASSWORD=" .. PASSWORD .. "&dateutc=" .. timestring

if Outside_Temp_Hum ~= '' then
   WU_URL = WU_URL .. "&tempf=" .. string.format("%3.1f", CelciusToFarenheit(otherdevices_temperature[Outside_Temp_Hum]))
   WU_URL = WU_URL .. "&humidity=" .. otherdevices_humidity[Outside_Temp_Hum]
   WU_URL = WU_URL .. "&dewptf=" .. string.format("%3.1f", CelciusToFarenheit(otherdevices_dewpoint[Outside_Temp_Hum]))
end

if Barometer ~= '' then
   WU_URL = WU_URL .. "&baromin=" .. string.format("%2.2f", hPatoInches(otherdevices_barometer[Barometer]))
end

if RainMeter ~= '' then
   WU_URL = WU_URL .. "&dailyrainin=" .. string.format("%2.2f", mmtoInches(otherdevices_rain[RainMeter]))
   WU_URL = WU_URL .. "&rainin=" .. string.format("%2.2f", mmtoInches(otherdevices_rain_lasthour[RainMeter]))
end

if WindMeter ~= '' then
   WU_URL = WU_URL .. "&winddir=" .. string.format("%.0f", otherdevices_winddir[
WindMeter])
   WU_URL = WU_URL .. "&windspeedmph=" .. string.format("%.0f", (otherdevices_wi
ndspeed[WindMeter]/0.1)*0.223693629205)
   WU_URL = WU_URL .. "&windgustmph=" .. string.format("%.0f", (otherdevices_win
dgust[WindMeter]/0.1)*0.223693629205)
end

if UVMeter ~= '' then
   WU_URL = WU_URL .. "&UV=" .. string.format("%.1f", mmtoInches(otherdevices_uv
[UVMeter]))
end

--&weather=
--&clouds=

WU_URL = WU_URL .. "&softwaretype=" .. SoftwareType .. "&action=updateraw"

-- print (WU_URL)

commandArray = {}

-- remove the comment in the line below to actualy upload it
commandArray['OpenURL']=WU_URL

return commandArray

-- end of script

else
  commandArray = {}
  return commandArray
end

[Toulon7559]

@IanDury

Easy enough.
Those different baseURLs, IDs and Passwords just spoil the fun to make just one common script for the combination of WUnderground, WOW and PWSWeather .....
Perhaps still better to keep them separate, also to have easy control on different upload-times and different upload-rates.

[IanDury]

@Ton

It should be possible to combine all into a single script and still use different rates/offsets for the different upload sites. Maybe I will work on that on a rainy sunday afternoon

[Toulon7559]

A user of the script observed an offset in his measurements of rain, as read through an RFXCom from the wireless signals of his PWS.
Correction of the value at the rain-widget of Domoticz does not seem to affect the value taken for upload.
Therefore requested in the script a facility to correct the rain-values to be uploaded to WUnderground.

A primitive solution, but the following adapted scriptlines should do the tric.
The relative correction values a and b have to be determined empirically, and in my opinion should be the same value, but you never know how WUnderground reacts, and by applying them separately, you have freedom to experiment. It means that by multiplication you arrive at the assumed correct value.
The absolute correction of 0 has only been added for completeness to show the principle of the correction. The reason that this correction for rain is not implemented as a variable, is explained in the 3rd suggested script-line.
Look for the original 4 lines of script-code dealing with making the 'Rainmeter-WU_URL', and replace them by these 7 lines.

Code: Select all

a = 1 -- relative correction for rain_day in mm, default is 1, range from 0.9 to 1.1 for  max.10% offset
b = 1 -- relative correction for rain_lasthour in mm, default is 1, range from 0.9 to 1.1 for max.10% offset
-- absolute correction of 0 for rain_day and for rain_lasthour, because otherwise strange bias each day at 00:00
if RainMeter ~= '' then
   WU_URL = WU_URL .. "&dailyrainin=" .. string.format("%2.2f", mmtoInches((otherdevices_rain[RainMeter])*a + 0))
   WU_URL = WU_URL .. "&rainin=" .. string.format("%2.2f", mmtoInches((otherdevices_rain_lasthour[RainMeter])*b + 0))
end

In a similar way you might 'correct' all other uploaded values, derived from reading from the wireless sensors.
Sometimes quite useful, because I have seen thermosensors and barometersensors with both relative and absolute offsets.
Although it is difficult to determine which correction values should be implemented if you desire max. absolute accuracy ........
At least for barometer-readings the values reported by a near 'official' meteostation or airport may be a good reference.

[Toulon7559]

The adaptations of the script for upload to PWS Weather and for upload to WOW made me look around for a comparable possibility for upload to AWEKAS. Perhaps found a method, described as MyAWEKAS DirectLink.
Except for generation of the md5 hash-code, a Lua-script for this push_API-protocol for MyAWEKAS_DirectLink could be an offspring of the upload-script for WUnderground, PWSWeather and WOW.
The github-reference is very specific in relation to a request to AWEKAS in 2012: for making an own upload-script, advised to first contact AWEKAS to get the latest protocol-description applicable for your purpose. The latest, more extended issue I got is from 2015 and divides in a 'basic section' of 23 positions, 2 positions for geoposition for mobile application, and 86 positions in an 'API2-extension' which deals with statistic data.
Filling the 'API2-extension' is only possible if you have database or datatables at hand which contain more than a year of information .........
Fortunately the 'extension' is optional and may be neglected/deleted.
Besides other layout of the upload-URL also rework for the AWEKAS_script to apply local time instead of UTC (as applied for WU, PWSWeather and WOW).
Generation of the md5 hashcode for your password is a once-only effort for script's settings, for which this tool is suitable.

After some testing, now operational for my configuration as shown at this weblink