Ookla Speedtest CLI integration

[FireWizard]

Hi @SavioThecnic,

Of course I can share the flow, but I already did.
See my post March 29, 2020 at 18:38, in this thread.

Regards

[SavioThecnic]

Yes, I saw that you posted something in the previous messages

Unfortunately my English is not very strong
and with google translate I got a bit lost
I would have preferred an attached json export
In a way to imprint without the risk of making mistakes
for having misunderstood
and of course then tweak it for my setup
What palette do you need to have installed in the Red Node
to use your flow

[SavioThecnic]

Hello @SavioThecnic

From time to time Ookla Speedtest requires that you accept their license.
You will not see this, until you open a terminal and activate the speedtest-cli manually..
However it is indicated in Domoticz with a red bar above your widget, that it has exceed the maximum time for that sensor.

In the first version of the script, the first part of the command was as follows:

Code: Select all

/usr/bin/speedtest --format=json 

Currently I use:

Code: Select all

/usr/bin/speedtest --accept-gdpr --format=json

This works fine for me, so far.
But you may want to add --accept-licence as well. See the @grutter post.

You can test from the command line in a terminal.Regards

Sorry about the ooklaspeed.sh
I do not find the item you mention I am attaching my ooklaspeed.sh
that I did by seeing your example
Where should I add the entry to accept the license
In fact, via script it does not work by launching it manually it works As you can see from the attached screenshots
Hope by fixing your script or with the red knot
with one two two systems can solve

my ooklaspeed.sh

Code: Select all

#!/bin/bash
# make sure jq is installed
# sudo apt-get install jq

# define domoticz connection
host=192.168.1.20
port=8080

# define ids for domoticz sensors
idxpng=64
idxul=63
idxdl=62
idxjit=65

# define location for file
file=/home/domoticz/domoticz/ookla.json

# execute the speedtest and store result.
speedtest -f json-pretty > $file

downraw=$(jq -r '.download.bandwidth' $file)
download=$(printf %.2f\\n "$((downraw * 8))e-6")

upraw=$(jq -r '.upload.bandwidth' $file)
upload=$(printf %.2f\\n "$((upraw * 8))e-6")

png=$(jq -r '.ping.latency' $file)

jit=$(jq -r '.ping.jitter' $file)

#output if you run it manually
echo "download = $download Mbps"
echo "upload =  $upload Mbps"
echo "ping =  $png ms"
echo "jitter = $jit ms"

# upload result to Domiticz
curl "http://$host:$port/json.htm?type=command&param=udevice&idx=$idxdl&svalue=$download" > /dev/null 2>&1
curl "http://$host:$port/json.htm?type=command&param=udevice&idx=$idxul&svalue=$upload" > /dev/null 2>&1
curl "http://$host:$port/json.htm?type=command&param=udevice&idx=$idxpng&svalue=$png" > /dev/null 2>&1
curl "http://$host:$port/json.htm?type=command&param=udevice&idx=$idxjit&svalue=$jit" > /dev/null 2>&1

[FireWizard]

Hello @SavioThecnic,

You wrote:

Sorry about the ooklaspeed.sh

This shell script is unknown for me. I never wrote it and did not see it before.

My command line consist in fact on a single line:

Code: Select all

/usr/bin/speedtest --accept-gdpr --accept-license --format=json | /usr/bin/mosquitto_pub -L mqtt://user:pass@<IP Address>:1883/speedtest/status -s

This command is executed on my router with a direct connection to the Internet and that is a different machine than where Domoticz, Node RED and Mosquitto runs on.

It pipes the command into mosquitto_pub and so it publishes the data to the MQTT Server.
Node RED does the translation to Domoticz. Do not forget to install mosquitto-clients.

Find below my Node RED flow.

Code: Select all

[{"id":"71ee0102.5abb38","type":"tab","label":"Speedtest","disabled":false,"info":""},{"id":"80d3d6f7.287248","type":"mqtt in","z":"71ee0102.5abb38","name":"Server Speedtest In","topic":"speedtest/status","qos":"0","datatype":"json","broker":"f8a872af.a6898","inputs":0,"x":190,"y":100,"wires":[["f99521a7.91a89"]]},{"id":"f99521a7.91a89","type":"function","z":"71ee0102.5abb38","name":"To Domoticz","func":"var msg1 ={};\nvar msg2 ={};\nvar msg3 ={};\nvar msg4 ={};\nvar msg5 ={};\nmsg1.payload = {\"command\":\"udevice\",\"idx\":119,\"nvalue\":0,\"svalue\":msg.payload.ping.latency.toString()};\nmsg2.payload = {\"command\":\"udevice\",\"idx\":120,\"nvalue\":0,\"svalue\":(parseFloat((msg.payload.download.bytes*8/msg.payload.download.elapsed)/1000).toFixed(3)).toString()};\nmsg3.payload = {\"command\":\"udevice\",\"idx\":121,\"nvalue\":0,\"svalue\":(parseFloat((msg.payload.upload.bytes*8/msg.payload.upload.elapsed)/1000).toFixed(3)).toString()};\nmsg4.payload = {\"command\":\"udevice\",\"idx\":402,\"nvalue\":0,\"svalue\":msg.payload.ping.jitter.toString()};\nmsg5.payload = {\"command\":\"udevice\",\"idx\":403,\"nvalue\":0,\"svalue\":msg.payload.packetLoss.toString()};\nreturn [[msg1,msg2,msg3,msg4,msg5]];","outputs":1,"noerr":0,"x":410,"y":100,"wires":[["64834ac6.75a034"]]},{"id":"64834ac6.75a034","type":"mqtt out","z":"71ee0102.5abb38","name":"Domoticz In","topic":"domoticz/in","qos":"2","retain":"false","broker":"f8a872af.a6898","x":610,"y":100,"wires":[]},{"id":"b47c88ca.5804d","type":"mqtt in","z":"71ee0102.5abb38","name":"Domoticz Out","topic":"domoticz/out","qos":"2","datatype":"json","broker":"f8a872af.a6898","inputs":0,"x":170,"y":220,"wires":[["6f6b8107.956448"]]},{"id":"6f6b8107.956448","type":"filter","z":"71ee0102.5abb38","name":"Filter Button","property":"payload.idx","propertyType":"msg","asArray":false,"itemProperty":"","itemPropertyType":"item","rules":[{"t":"eq","v":"122","vt":"num","output":1}],"checkall":"true","outputs":1,"x":370,"y":220,"wires":[["965aceab.7ea108"]]},{"id":"965aceab.7ea108","type":"function","z":"71ee0102.5abb38","name":"To Speedtest","func":"if (msg.payload.nvalue === 1) {\n    msg.payload = \"on\";\n}\nreturn msg;","outputs":1,"noerr":0,"x":570,"y":220,"wires":[["93d16ba1.e71948"]]},{"id":"93d16ba1.e71948","type":"mqtt out","z":"71ee0102.5abb38","name":"Server Speedtest Out","topic":"speedtest/cmd","qos":"0","retain":"false","broker":"f8a872af.a6898","x":800,"y":220,"wires":[]},{"id":"d6399787.e39ee","type":"inject","z":"71ee0102.5abb38","name":"Hourly Speedtest ","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"3600","crontab":"","once":true,"onceDelay":0.1,"topic":"","payload":"on","payloadType":"str","x":550,"y":160,"wires":[["93d16ba1.e71948"]]},{"id":"f8a872af.a6898","type":"mqtt-broker","name":"Jonas_MQTT_Server","broker":"192.168.10.24","port":"1883","clientid":"","usetls":false,"protocolVersion":"4","keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","birthMsg":{},"closeTopic":"","closeQos":"0","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willPayload":"","willMsg":{},"sessionExpiry":""}]

The only extra non default node you need is:

Code: Select all

node-red-contrib-filter

One remark.

If you run speedtest, domoticz and Node RED on the same device, you can also use an "Exec" node.
Even a dedicated speedtest node exists. See: https://flows.nodered.org/node/node-red ... -speedtest

I never used that, so I cannot tell you, if it works well

Regards

[SavioThecnic]

Hello @SavioThecnic,
You wrote:

Sorry about the ooklaspeed.sh

This shell script is unknown for me. I never wrote it and did not see it before.
My command line consist in fact on a single line:

Code: Select all

/usr/bin/speedtest --accept-gdpr --accept-license --format=json | /usr/bin/mosquitto_pub -L mqtt://user:pass@<IP Address>:1883/speedtest/status -s

Sorry the script I use now is the one at the beginning of the topic
which, however, has not worked for many months now
I thought you had modified that better

I have a Raspberry PI4 8Gb connected with LAN Copper
So I shouldn't have any problems with the speed on the network
and on the same raspberry I have Domoticz NodeRed and Mosquitto

[jberinga]

Does anyone know if there has been a change at Ookla for cutting down the times you can do the test? Because around the 20th of May I have a bigger difference between download values:

ZiggoJBE.jpg (318.88 KiB) Viewed 2376 times

I asked someone who also use the CLI script in Domoticz and it seems he also experiencing the same, although his difference is not as big as mine:

ZiggoWK.jpg (224.18 KiB) Viewed 2376 times

PS: I also asked my internet provider the question if it might be an issue with my connection, this is still in investigation

[jberinga]

Does anyone know if there has been a change at Ookla for cutting down the times you can do the test? Because around the 20th of May I have a bigger difference between download values:

ZiggoJBE.jpg

I asked someone who also use the CLI script in Domoticz and it seems he also experiencing the same, although his difference is not as big as mine:

ZiggoWK.jpg


PS: I also asked my internet provider the question if it might be an issue with my connection, this is still in investigation

It seems that at Ookla there is a server that not responds correctly on the download. Because after editing the script with a specific server
(# execute the speedtest and store result.
speedtest --server-id=13130 --accept-license --accept-gdpr -f json-pretty > $file)
it all looks much better :

Ziggo.jpg (349.83 KiB) Viewed 2321 times

[zicht]

Hi

Two(2) things to mention in addition to be aware off :
1. When using this Ookla Cli with schedule -> it is causing Microsoft Teams to clutter and even freeze for a short moment when it is running ( bandwidth use issue i suppose) so do not set it to frequent when working at home with a lot of teams meetings.
2. When When using this Ookla Cli to frequent in x time -> you will trigger an ookla defense mechanisme redirection you to a "special anti tamper" server giving different results and some restrictions. This will only trigger after some time, trying to reach this with manual testing has for me never succeed, but with scripting somewhere during the week i was redirected.