Page 1 of 5
Getting data from LoRa - TTN network
Posted: Friday 16 March 2018 16:12
by costo
I'm trying to extract data from a node that is communicating with TheThingsNetwork over LoRaWan.
In a terminal window I can sent this command:
curl -X GET --header 'Accept: application/json' --header 'Authorization: MyKey' 'MyURL'
where MyKey is a long private key for this particular node and MyURL is on .data.thethingsnetwork.org/etc
I get a response between [ ] brackets which contain several strings separated by comma's:
each line is like this:
{"device_id":"costonode0","hum":55,"raw":"3x4gGw==","temp":21.9375,"time":"2018-03-16T13:50:23.154294105Z","volt":3227},
So I want to write a script either in BASH or LUA which analyses the strings and with JSON returns the time, hum(idity) , temp(erature) and voltage of my battery (in mV). Ofcourse this node can generate other data so the script needs to be a little configurable.
I am not a very good programmer, I have a little experience with LUA so I ask here for some help and clues how to write this script, either in LUA or another language.
Re: Getting data from LoRa - TTN network
Posted: Saturday 17 March 2018 11:46
by costo
I made this LUA script but apparently there is something wrong in the formatting of the curl string , i get this error:
-- 2018-03-17 11:32:00.391 Error: EventSystem: in /home/pi/domoticz/scripts/lua/script_time_lora.lua: /home/pi/domoticz/scripts/lua/script_time_lora.lua:14: ')' expected near 'Accept'
Code: Select all
local idxHum = 644
local idxTemp = 645
local idxVolt = 643
local hum = ' '
local temp = ' '
local volt = ' '
commandArray = {}
time = os.date("*t")
if ((time.min % 5)==0) then -- Run every 5 minutes.
json = (loadfile "/home/pi/domoticz/scripts/lua/JSON.lua")()
local jsondata = assert(io.popen('curl -X GET --header 'Accept: application/json' --header 'Authorization: key ttn-account-v2.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' 'https://costonode100.data.thethingsnetwork.org/api/v2/query/costonode0?last=5m'))
local jsondevices = jsondata:read('*all')
jsondata:close()
local jsonLoRa = json:decode(jsondevices)
hum = jsonLoRa.hum
temp = jsonLoRa.temp
volt = jsonLoRa.volt
commandArray[1] = {['UpdateDevice'] = idxHum .. '|0|' .. hum}
commandArray[2] = {['UpdateDevice'] = idxTemp .. '|0|' .. temp}
commandArray[3] = {['UpdateDevice'] = idxVolt .. '|0|' .. volt}
end
return commandArray
Re: Getting data from LoRa - TTN network
Posted: Saturday 17 March 2018 20:21
by costo
Does someone see the error ?
I cannot figure it out what is going wrong.
I get error in line 23: attempt to index local 'jsonLoRa' (a nil value)
So maybe the json does not execute well or maybe there is an error in the Curl string.
I am out of options.
Can someone who understands LUA better than me help me here a bit please. ?
Code: Select all
local idxHum = 644
local idxTemp = 645
local idxVolt = 643
local hum = ''
local temp = ''
local volt = ''
local accessKey = 'ttn-account-v2.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
local urlTTN = 'https://costonode100.data.thethingsnetwork.org/api/v2/query/costonode0?last=5m'
commandArray = {}
-- time = os.date("*t")
-- if ((time.min % 5)==0) then -- Run every 5 minutes.
json = (loadfile "/home/pi/domoticz/scripts/lua/JSON.lua")()
local config = assert(io.popen("curl -X GET -H \'Accept: application/json\' -H \'Authorization: ..'accessKey'..\' \'..'urlTTN'..\'"))
local devices = config:read('*all')
config:close()
print(output)
local jsonLoRa = json:decode(devices)
hum = jsonLoRa.hum
temp = jsonLoRa.temp
volt = jsonLoRa.volt
commandArray[1] = {['UpdateDevice'] = idxHum .. '|0|' .. hum}
commandArray[2] = {['UpdateDevice'] = idxTemp .. '|0|' .. temp}
commandArray[3] = {['UpdateDevice'] = idxVolt .. '|0|' .. volt}
-- end
return commandArray
Re: Getting data from LoRa - TTN network
Posted: Sunday 18 March 2018 1:16
by costo
I found out that the problem is in the square brackets.
ttn server returns something like this:
[{"device_id":"costonode0","hum":50,"raw":"3RogEg==","temp":21.8125,"time":"2018-03-17T23:52:14.248591862Z","volt":3218}]
or
[{"device_id":"costonode0","hum":50,"raw":"3RogEg==","temp":21.8125,"time":"2018-03-17T23:52:14.248591862Z","volt":3218},{"device_id":"costonode0","hum":50,"raw":"3hogEg==","temp":21.875,"time":"2018-03-17T23:55:56.381884587Z","volt":3218}]
data for the last 5 minutes usually returns one json object and sometimes two.
look like local jsonLoRa = json:decode(devices) gives an error (nil value) because of the square brackets .
tried to remove the square brackets with this:
x = string.gsub(devices,"[{","{")
devices = string.gsub(x,"}]","}")
that was not working.
someone has a clue for me ?
Re: Getting data from LoRa - TTN network
Posted: Saturday 05 May 2018 10:14
by alexsh1
costo wrote: ↑Sunday 18 March 2018 1:16
someone has a clue for me ?
I am afraid you are on your own.
I have just started to looking into LoraWAN/TTN
Re: Getting data from LoRa - TTN network
Posted: Sunday 26 August 2018 9:51
by gizmocuz
Hi,
I just finished the first implementation of a hardware class (available in current beta, 9946 and up) that connects to the TTN MQTT broker and is able to work with devices that use CayenneLPP
"The Things Network (MQTT/CayenneLPP) with LAN interface"
First try to get yourself connected via a commandline MQTT client to test if everything works, then you can add this hardware
(Do not forget to enable 'accept new hardware' in the Domoticz settings)
Hope all works well , tested here with a temp/hum/baro sensor (BME 280), but separate should also work, as well as the other sensors.
Currently it is not possible to 'send' data (like send a switch ON/OFF command)
If you use the same channel for your temp,humidity,baro sensor, domoticz will combine them into 1 sensor, same for temp+hum, or temp+baro
In the Domoticz github repo you can find a class to encode to CayenneLPP in C++ (folder hardware/CayenneLPP)
Re: Getting data from LoRa - TTN network
Posted: Thursday 29 November 2018 18:59
by Toulon7559
Re: Getting data from LoRa - TTN network
Posted: Thursday 29 November 2018 21:19
by gizmocuz
Sure... thats what i implemented... the CayenneLPP protocol
Re: Getting data from LoRa - TTN network
Posted: Saturday 15 December 2018 14:28
by Toulon7559
Any hints for application with KPN LoRa?
Re: Getting data from LoRa - TTN network
Posted: Thursday 27 December 2018 20:05
by QSKONE
Hi , Can you make tutorial?
My log:
Code: Select all
Status: TTN_MQTT: Connecting to 192.168.1.5:1883
Error: TTN_MQTT: Failed enabling TLS mode, return code: 3 (CA certificate: 'ttn-account-xxx')
2018-12-27 19:58:28.404 Status: TTN_MQTT: Worker stopped...
Re: Getting data from LoRa - TTN network
Posted: Tuesday 01 January 2019 13:21
by gizmocuz
You need to use the same privilege settings as you would when using MQTT to connect to the TTN network.
Did you try using a MQTT client ?
Re: Getting data from LoRa - TTN network
Posted: Friday 04 January 2019 12:55
by costo
I have been puzzling for some time but I cannot make this plugin working. It is not clear to me what data i need to put in the fields of the TTN plugin.
If I look at my 'MQTT Client Gateway with LAN Interface' plugin that works with a IP in the 192.168.178.xx range, and port 1883. The Username and Password are the ones to connect to the RasPi server where Domoticz also resides.
I imaging that I need to use the TTN credentials for accessing my TTN-node . That access key looks like 'ttn-account-vs.ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!@#$%&*'. I do not know of any Username/Password/CA_Filename combination to access the TTN network.
Re: Getting data from LoRa - TTN network
Posted: Sunday 13 January 2019 9:00
by jeroenkl
costo wrote: ↑Friday 04 January 2019 12:55
I have been puzzling for some time but I cannot make this plugin working. It is not clear to me what data i need to put in the fields of the TTN plugin.
If I look at my 'MQTT Client Gateway with LAN Interface' plugin that works with a IP in the 192.168.178.xx range, and port 1883. The Username and Password are the ones to connect to the RasPi server where Domoticz also resides.
I imaging that I need to use the TTN credentials for accessing my TTN-node . That access key looks like 'ttn-account-vs.ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!@#$%&*'. I do not know of any Username/Password/CA_Filename combination to access the TTN network.
Hi Costo,
I'm dealing with the same issue, do you have already the solution?
BR, Jeroen
Re: Getting data from LoRa - TTN network
Posted: Sunday 13 January 2019 9:48
by QSKONE
Integration TTN to Domoticz it's possible ,but nobody knows how do it.
Re: Getting data from LoRa - TTN network
Posted: Sunday 13 January 2019 20:01
by jeroenkl
costo wrote: ↑Friday 04 January 2019 12:55
I have been puzzling for some time but I cannot make this plugin working. It is not clear to me what data i need to put in the fields of the TTN plugin.
If I look at my 'MQTT Client Gateway with LAN Interface' plugin that works with a IP in the 192.168.178.xx range, and port 1883. The Username and Password are the ones to connect to the RasPi server where Domoticz also resides.
I imaging that I need to use the TTN credentials for accessing my TTN-node . That access key looks like 'ttn-account-vs.ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!@#$%&*'. I do not know of any Username/Password/CA_Filename combination to access the TTN network.
I'm able to connect:
2019-01-13 19:49:27.925 Status: TTN_MQTT: Connecting to eu.thethings.network:1883
2019-01-13 19:49:28.053 Status: TTN_MQTT: connected to: eu.thethings.network:1883
2019-01-13 19:49:28.158 Status: TTN_MQTT: Subscribed
2019-01-13 19:57:07.336 TTN_MQTT: Topic: xxxxx/devices/xxxxxx/up
2019-01-13 19:57:07.337 Error: TTN_MQTT:
Invalid data received!
with this:
mosquitto_sub -h <Region>.thethings.network -t '+/devices/+/up' -u '<AppID>' -P '<AppKey>' -v
I've got the data from my devices, so something wrong with the domoticz plugin?
pi@raspberrypi:~ $ mosquitto_sub -h eu.thethings.network -t '+/devices/+/up' -u 'xxxxxxx' -P 'ttn-account-v2.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-v
xxxxxx/devices/xxxxxxxxxxx/up"app_id":"xxxxx","dev_id":"xxxxxxxxxxxxx","hardware_serial":"xxxxxxxxxxxxxxxxxx","port":1,"counter":1124,"payload_raw":"A+gDDA==","payload_fields":{"humidity":10,"temperature":7.8},"metadata":{"time":"2019-01-13T19:19:59.379375846Z","frequency":867.7,"modulation":"LORA","data_rate":"SF7BW125","airtime":51456000,"coding_rate":"4/5","gateways":[{"gtw_id":"xxxxxxxxx","gtw_trusted":true,"timestamp":2412230724,"time":"2019-01-13T19:20:02Z","channel":6,"rssi":-106,"snr":2,"rf_chain":0,"latitude":52.26572,"longitude":5.183096,"altitude":12,"location_source":"registry"}]}}
^C
pi@raspberrypi:~ $
Re: Getting data from LoRa - TTN network
Posted: Tuesday 15 January 2019 17:24
by jeroenkl
gizmocuz wrote: ↑Tuesday 01 January 2019 13:21
You need to use the same privilege settings as you would when using MQTT to connect to the TTN network.
Did you try using a MQTT client ?
Pfff finally all is working. I'll provide the info later this week.
Re: Getting data from LoRa - TTN network
Posted: Friday 18 January 2019 17:44
by Toulon7559
As start of my LoRaWAN learning curve, now experimenting with a
Marvin Development Board, with related, temporary 'Free' subscriptions to the KPN LoRa-Developer Portal (6 months) and to the Mendix Dashboard (70 days).
'New Land' and therefore looking around at Internet, to
sites like this one with related experiences.
So far got my Marvin-board with connected SHT31-sensor periodically uploading T/H-data over the KPN LoRa-Network towards the Mendix Dashboard:
uplink is quick, but storage and read-out is not very frequent, because the free subscription limits the handling to 6messages/hour.
For more direct & quicker read-out, the next hurdle/challenge is how to get online hold of the JSON-file which is told to be 'there', containing the uploaded data.
Also trying data collection by means of other Endpoint_URLs like Hookbin.com, but not yet with results.
The example scripts at the
home page of hookbin.com probably are good starting-material for an extraction script, but first have to know which link&key-information to insert and how .....
For 'data grabbing' from the KPN Network somebody having a tested, (relatively) simple example setup for the applicable (C)URL-call, etc.?
Re: Getting data from LoRa - TTN network
Posted: Tuesday 22 January 2019 20:32
by Toulon7559
@Gizmocuz
Reading your remark that Domoticz now supports Cayenne linking to TTN's MQTT broker, looked around and found a contribution by Hans Boksem for a
Meteo-System linked through TTN and NodeRed to Domoticz, later extended[?] with a variant
linking to Cayenne.
Question related to last line of your statement
I just finished the first implementation of a hardware class (available in current beta, 9946 and up) that connects to the TTN MQTT broker and is able to work with devices that use CayenneLPP
"The Things Network (MQTT/CayenneLPP) with LAN interface"
First try to get yourself connected via a commandline MQTT client to test if everything works, then you can add this hardware
;-( Possible to give example-for-dummies for
connect via a commandline MQTT client
?
Just wondering how to fit my Marvin in that picture, presently operating at the KPN LoRa-Network:
apparently have to mimick the TTN-'scheme' created by Hans van Boksem and Gizmocuz, with a KPN-replacement for TTN's MTTQ-broker.
Re: Getting data from LoRa - TTN network
Posted: Wednesday 23 January 2019 20:07
by robert13b
jeroenkl wrote: ↑Tuesday 15 January 2019 17:24
gizmocuz wrote: ↑Tuesday 01 January 2019 13:21
You need to use the same privilege settings as you would when using MQTT to connect to the TTN network.
Did you try using a MQTT client ?
Pfff finally all is working. I'll provide the info later this week.
Looking forward to it. I get the same kind of data from the command line with de mosquito client. Bout how do i get it into Domoticz referring to an IDX?
Re: Getting data from LoRa - TTN network
Posted: Sunday 27 January 2019 10:25
by jeroenkl
1; First ensure you got the Lora required application data in cayenne visibel (tutorial on the web, in case you need help let me know)
2: Next add new hardware in domoticz: The Things Network (MQTT/CayenneLPP) with LAN interface, eu.thethings.network:1883, login: Lora application and passwd
3: Discovered devices are automatic added in domoticz
Need help? Let me know.
BR, Jeroen