Get a temperature value from webpage/txt file

[WhiWhi]

I'm wondering how to import a temperature from a webpage / txt file.

http://www.temperatur.nu/termo/abro/temp.txt

And be able to display it as a sensor in domoticz. I think you could do it with a lua or python script but do not have the knowledge to implement it.

I need help.

Edit. Fix the url

[zicht]

the only way i see is by dzevents or pure lua.

example for local text file.

local path="blabla.txt"
local open = io.open
local file = open(path, "rb") -- r read mode and b binary mode
local content = file:read "*a" -- *a or *all reads the whole file
file:close()
--print(content)

but this only works for local files.

then you need to find out how to get the right value, or strip all that is not needed.
The found value can bed put in a virtual device with commandArray["blabla"]=value

I am not a coder so you need to investigate how to do it best, the above is copied from the forum.
Maybe it points you in a direction that is helpfull otherwise i hope someone has a working solution for you

[FireWizard]

Hi, @WhiWhi,

This is possible with Node Red and MQTT or HTTP.

1. The provided link does not exist, so my first question is:
Is this URL real or just fake and used as a possible example?
As long as we cannot see the real URL and the way the temperature is presented on the webpage or in a text file, it is difficult to say, whether it is doable.

2. You do not indicate how often the temperature should be read from the webpage and pushed to Domoticz.

Regards

[WhiWhi]

http://www.temperatur.nu/termo/abro/temp.txt

I would like to update the value every five minutes

[FireWizard]

HI, @WhiWhi,

Questions:

1. Is this a local owned website, as it only indicates 25.8 (degrees Celsius, I assume)?

2. Do you have Node Red installed?

If not, install it on your Raspberry Pi, which runs Domoticz.
See: https://nodered.org/docs/getting-started/raspberrypi

3.Do you have a MQTT server (broker) installed?

If not, install it on your Raspberry Pi, which runs Domoticz.
See: https://pimylifeup.com/raspberry-pi-mos ... tt-server/

Let me know,

Regards

[WhiWhi]

Well there is a public temperature page on the web. As I report my outside temperature to.
Here is the real page you see when you visit the page. https://www.temperatur.nu/abro The second link is just to use to type to implement in other applications.

No I have not installed Node Red

And not MQTT server

I may well install and test it. Not because I know exactly how to proceed, but it may work when I try a little.

[FireWizard]

Hi @WhiWhi

You wrote:

No I have not installed Node Red

I suggest to install it, as I think it is an excellent companion of Domoticz (or any other Home Automation system).
Make yourself familiar with the interface.

And not MQTT server

Install this as well, as it will extend your possibilities.
Add the hardware in Domoticz as well.

I also suggest to install MQTT Explorer on your laptop/desktop. Excellent tool.
See: http://mqtt-explorer.com/

If you have installed both Node Red and MQTT (also in Domoticz) the flow is ready.

(It is really the most simple web scraping, I ever did)

See the screenshot below:

Screenshot_Temp1.png (35.49 KiB) Viewed 2354 times

And this will result in:

Screenshot_Temp2.png (43.6 KiB) Viewed 2354 times

If you have installed both Node Red and Mosquitto, you may want to import this flow:

Code: Select all

[{"id":"53864ab3.6d5304","type":"tab","label":"Temperature2Domoticz","disabled":false,"info":""},{"id":"5490d5ff.8a5774","type":"inject","z":"53864ab3.6d5304","name":"Inject 5 Minutes","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"300","crontab":"","once":true,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":570,"y":160,"wires":[["eaf2f3b6.01dd48"]]},{"id":"eaf2f3b6.01dd48","type":"http request","z":"53864ab3.6d5304","name":"Get Temperature","method":"GET","ret":"obj","paytoqs":"ignore","url":"http://www.temperatur.nu/termo/abro/temp.txt","tls":"","persist":false,"proxy":"","authType":"","x":810,"y":160,"wires":[["66d5458.1a526bc","4fbd99e.7a945e8"]]},{"id":"66d5458.1a526bc","type":"debug","z":"53864ab3.6d5304","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1030,"y":220,"wires":[]},{"id":"4fbd99e.7a945e8","type":"function","z":"53864ab3.6d5304","name":"Prepare for Domoticz","func":"msg.payload = {\"command\":\"udevice\",\"idx\":11,\"nvalue\":0,\"svalue\":msg.payload.toString()};\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1060,"y":160,"wires":[["1ccce439.04623c","844c19c8.4eba3"]]},{"id":"1ccce439.04623c","type":"debug","z":"53864ab3.6d5304","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1290,"y":220,"wires":[]},{"id":"844c19c8.4eba3","type":"mqtt out","z":"53864ab3.6d5304","name":"","topic":"domoticz/in","qos":"","retain":"","respTopic":"","contentType":"","userProps":"","correl":"","expiry":"","broker":"f9f13036.e28b58","x":1290,"y":160,"wires":[]},{"id":"f9f13036.e28b58","type":"mqtt-broker","name":"localhost","broker":"127.0.0.1","port":"1883","clientid":"","usetls":false,"compatmode":true,"keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthRetain":"false","birthPayload":"","closeTopic":"","closeQos":"0","closePayload":"","willTopic":"","willQos":"0","willPayload":""}]

Do not forget to replace the IDX of the virtual temperature sensor with your IDX (currently 11).

Regards

[jjcarron]

Hi @WhiWhi

I have solve a similar problem with dsVents

My Situation:
I have a device in my Flat sendig Temp and Humidity to a server. I can get the information from an internet URL. This information is not structured but just a page displaying my information.

The two following links show the available pages
https://live.ecco2.ch/Data?devId=10CE45FFFE0006DC
https://live.ecco2.ch/More?devId=10CE45FFFE0006DC

My Solution:
I have create a dummy hardware and then:

• one virtual Temp+Humidity device

• one virtual devices for the Battery

• one virtual device for the signal strength

Then I just create a dzVents event handler internally with the following code.

Code: Select all

utils = require("Utils") 

function os.capture(cmd, raw)
  local f = assert(io.popen(cmd, 'r'))
  local s = assert(f:read('*a'))
  f:close()
  if raw then return s end
  s = string.gsub(s, '^%s+', '')
  s = string.gsub(s, '%s+$', '')
  s = string.gsub(s, '[\n\r]+', ' ')
  return s
end
 

return {
	on = {
		timer = {
			'every 15 minutes' -- just an example to trigger the request
		},
		httpResponses = {
			'trigger' -- must match with the callback passed to the openURL command
		}
	},
	logging = {
		level = domoticz.LOG_INFO,
		marker = 'ECorridor',
	},
	execute = function(dz, item)
        local temperatureHumidityDevice = dz.devices('Ecco2 Corridor')
        local batteryDevice = dz.devices('Ecco2 Battery')
        local signalDevice = dz.devices('Ecco2 Signal')        
        local dataUrl = 'https://live.ecco2.ch/data?devId=10CE45FFFE0006DC'
        local moreUrl = 'https://live.ecco2.ch/more?devId=10CE45FFFE0006DC'

        local function humidityStatus(temperature, humidity) -- as used indz source
            if humidity <= 30 then return dz.HUM_DRY
            elseif humidity >= 70 then return dz.HUM_WET
            elseif  humidity >= 35 and humidity <= 65 and temperature >= 22 and temperature <= 26 then return dz.HUM_COMFORTABLE end
            return dz.HUM_NORMAL
        end
		
		if (item.isTimer) then
			dz.openURL({
				url = dataUrl,
				method = 'GET',
				callback = 'trigger', -- see httpResponses above.
			})
		end

		if (item.isHTTPResponse) then

			if (item.ok) then
			    --dz.log(item['data'], dz.LOG_INFO)
			    local strTemp =  [[ echo "]] .. item['data'] .. [[" | grep  "&#176" - | sed 's/.*>\(.*\)<.*/\1/' - | awk '{print $1}' - ]]
                local strHygro = [[ echo "]] .. item['data'] .. [[" | grep  "%</span>" - | sed 's/.*>\(.*\)<.*/\1/' - | awk '{print $1}' - ]]
                local t = tonumber(os.capture(strTemp, false));
                local h = tonumber(os.capture(strHygro, false));
                dz.log(t, h,  dz.LOG_INFO)
                temperatureHumidityDevice.updateTempHum( t, h, humidityStatus( t, h ) )
			else
				dz.log('There was a problem handling the temperature and humidity request', dz.LOG_ERROR)
				dz.log(item, dz.LOG_ERROR)
			end
			
			local strMore = os.capture('curl '..  moreUrl, true)
			local strSig =  [[ echo "]] .. strMore .. [["| grep  "dBm</span>" - | sed 's/.*>\(.*\)<.*/\1/' - | awk '{print $1}' - ]]
			local strBat =  [[ echo "]] .. strMore .. [["| grep  "uAh</span>" - | sed 's/.*>\(.*\)<.*/\1/' - | awk '{print $1}' - ]]
			

			local battery = tonumber(os.capture(strBat, false));
            local signal = tonumber(os.capture(strSig, false));
            
            
            dz.log('Battery =' .. battery .. ' uAh', dz.LOG_INFO)
            dz.log('Signal =' .. signal .. ' dBm', dz.LOG_INFO)
            
            signalDevice.updateCustomSensor(signal)
            batteryDevice.updateCustomSensor(battery)
        end

	end

The first link is called by a timer every 15 minutes.
The data are parsed using linux commands.

• First echo the data (Full HTML page) and

• pipe it to grep function that will detect the line where the temperature is displayed &#176 = °

• pipe it to sed function that extract the parts between tags delimiters > and <

• pipe it to awk function that retrive the string containing the value

The same process is repeated for the Humidity but grep % instead of °

Then I repeat the process for the second link, but as it was not call by the script, I just do a curl first to retrieve the second page in a variable.

Note: curl should be installed on linux

After that the devices should be updated with their native functions

[mgugu]

Why not simply using openUrl in dzVents since web page is in text format:

domoticz.openURL({
url = 'http://www.temperatur.nu/termo/abro/temp.txt',
method = 'GET',
callback = 'testopenurl'
})

and for the callback:

if (item.isHTTPResponse) then
if (item.ok) then
print(item.data)
end
end

gives

15.0

This value can be used to update a virtual sensor

[WhiWhi]

Thanks for the help. Now I've got it working.
I managed to make a dzVents script that "mgugu" suggested.

I have another problem that I can not solve. I have a nodemcu that I try to send a temperature to domoticz but do not succeed.

I have used "http://192.168.1.50:8080/json.htm?type= ... value=24.9" in both a web browser and in a simple python script and got domoticz to update the temperature but not via nodemcu with same string. Any suggestions?

[jjcarron]

@WhiWhi

I use a similar technics from a Pi to domoticz. Your string seems correct but as you are on another device, you maybe have to set a proper header to get the autorisation to access the domoticz server.

I use the following Python code to generate my string. (APICall = the part after json.htm? = type=...24.9)

Ex: res = domoticzAPI('type=command&param=udevice&idx=20&nvalue=0&svalue=23.75')
My config.domoticz_url='https://192.168.1.50:8080/json.htm?'

Code: Select all

def DomoticzAPI(APICall):
    resultJson = None
    url = "{}{}".format(config.domoticz_url,  parse.quote(APICall, safe="&="))
    print("Calling domoticz API: {}".format(url))
    resultJson = None
    try:
        req = request.Request(url)
        if config.user != "":
            credentials = ('%s:%s' % (config.user, config.pw))
            encoded_credentials = base64.b64encode(credentials.encode('ascii'))
            req.add_header('Authorization', 'Basic %s' % encoded_credentials.decode("ascii"))

        response = request.urlopen(req)
        if response.status == 200:
            resultJson = json.loads(response.read().decode('utf-8'))
            if resultJson["status"] != "OK":
                status ="Domoticz API returned an error: status = {}".format(resultJson["status"])
                resultJson = None
            else:
                status ="Domoticz API returned {}".format(resultJson["status"])
        else:
            status="Domoticz API: http error = {}".format(response.status)
    except:
        status="Error '{}'".format(url)
    
    print (status)
    return resultJson