Page 1 of 13
How to monitor your internet connection with virtual custom sensors
Posted: Saturday 22 October 2016 17:38
by safi78
Hi all,
I was having some troubles with my internet connection and couldn't figure out where they were coming from. I really like the graphing functionality of domoticz and I have implemented a script that notifies me with Telegram, so I wanted to monitor my download speed with domoticz, thought I'd share:
1. Install speedtest
Code: Select all
sudo apt-get install speedtest-cli
2. Create 3
Virtual Sensors of the type 'Custom Sensors' (for ping, download, upload-values) and remember the idx's (you need that later on)
3. Create a script (I called it st2domo.sh) with:
Code: Select all
#!/bin/bash
#setup
host=xxx.xxx.xxx.xxx
port=xxxx
username=xxx
password=xxx
pingidx=xxx
downloadidx=xxx
uploadidx=xxx
# no need to edit
speedtest-cli --simple > output.txt
ping=$(cat output.txt | sed -ne 's/^Ping: \([0-9]*\.[0-9]*\).*/\1/p')
download=$(cat output.txt | sed -ne 's/^Download: \([0-9]*\.[0-9]*\).*/\1/p')
upload=$(cat output.txt | sed -ne 's/^Upload: \([0-9]*\.[0-9]*\).*/\1/p')
#output if you run it manually
echo "ping = $ping ms"
echo "download = $download Mbps"
echo "upload = $upload Mbps"
curl -s -i -H "Accept: application/json" "http://$username:$password@$host:$port/json.htm?type=command¶m=udevice&idx=$pingidx&svalue=$ping"
curl -s -i -H "Accept: application/json" "http://$username:$password@$host:$port/json.htm?type=command¶m=udevice&idx=$downloadidx&svalue=$download"
curl -s -i -H "Accept: application/json" "http://$username:$password@$host:$port/json.htm?type=command¶m=udevice&idx=$uploadidx&svalue=$upload"
and fill in all the details after #setup:
host = your domoticz server (e.g. localhost or 192.168.x.x)
port = your domoticz port (e.g. 8080)
username = your domoticz username
password = your domoticz password
pingidx = the idx of the virtual sensor for accepting the ping-value
downloadidx = the idx of the virtual sensor for accepting the download-value
uploadidx = the idx of the virtual sensor for accepting the upload-value
Don't forget to make it executable (I always do):
4. edit your crontab (I scheduled this @hourly):
And put in the command:
If setup correctly, domoticz will receive your results from speedtest every hour:

- Screen Shot 2016-10-22 at 19.57.29.png (124.73 KiB) Viewed 22367 times
And you can do 'useful' stuff with it
Enjoy!
Re: monitor internet download speed with virtual sensor
Posted: Saturday 22 October 2016 18:20
by woody4165
Hi and thanks for sharing your script.
I have not found speedtest as package, but only speedtest-cli and probably the usage is different.
Should I add some source to download this package?
Thanks
Re: monitor internet download speed with virtual sensor
Posted: Saturday 22 October 2016 18:22
by safi78
my bad, you install 'speedtest-cli' but the command you use is 'speedtest' (so the script is ok). I'll update my post accordingly.
Thanks for the heads up!
Re: monitor internet download speed with virtual sensor
Posted: Saturday 22 October 2016 18:36
by manjh
This is great for the R-Pi, but is there an alternative for a Windows based server?
Re: monitor internet download speed with virtual sensor
Posted: Saturday 22 October 2016 18:37
by safi78
manjh wrote:This is great for the R-Pi, but is there an alternative for a Windows based server?
Sorry manjh, I do not use Windows ever.
Re: monitor internet download speed with virtual sensor
Posted: Saturday 22 October 2016 19:10
by woody4165
@safi78
it's working like a charm, just replaced
with
and it's ok.
I've also added --server xxxx where xxx is the server number near to myself so that test is more accurate.
You can find your test server with
and you will find the number associated to the server name.
So my test is:
Code: Select all
speedtest-cli --server xxxx | sed -ne 's/^Download: \([0-9]*\.[0-9]*\).*/\1/p')
Do you know how can I get also ping value?
the return is like this:
......[15.27 km]: 12.356 ms
Thanks!
Re: monitor internet download speed with virtual sensor
Posted: Saturday 22 October 2016 19:32
by safi78
changing the speedtest command to speedtest --simple will just produce this output:
Code: Select all
Ping: 27.302 ms
Download: 36.26 Mbits/s
Upload: 5.03 Mbits/s
Then you can use:
Code: Select all
sed -ne 's/^Ping: \([0-9]*.[0-9]*\).*/\1/p'
results to:
Re: monitor internet download speed with virtual sensor
Posted: Saturday 22 October 2016 19:36
by woody4165
thanks again!!
what i would like to understand is if I have to run three times the speedtest to get DL,UP and Ping, or if there is a way to get all the three numbers with one speedtest.
Re: monitor internet download speed with virtual sensor
Posted: Saturday 22 October 2016 19:38
by safi78
Sure, this was just a quick hack
just run speedtest --simple > output.txt
then cat output.txt | sed (etc) ?
edit: I'll change the script to that, more efficient (I was only chasing after the Download value

)
Re: monitor internet download speed with virtual sensor
Posted: Saturday 22 October 2016 19:49
by woody4165
Fantastic
so now my script is like this:
Code: Select all
#!/bin/bash
#setup
host=xxx.xxx.xxx.xxx
port=xxxx
username=xxx
password=xxx
#idx for download, upload and ping
idxdl=xxx
idxul=xxx
idxpng=xxx
# speedtest server number
serverst=xxxx
# no need to edit
speedtest-cli --simple --server $serverst > outst.txt
download=$(cat outst.txt | sed -ne 's/^Download: \([0-9]*\.[0-9]*\).*/\1/p')
upload=$(cat outst.txt | sed -ne 's/^Upload: \([0-9]*\.[0-9]*\).*/\1/p')
png=$(cat outst.txt | sed -ne 's/^Ping: \([0-9]*\.[0-9]*\).*/\1/p')
#output if you run it manually
echo "download = $download Mbps"
echo "upload = $upload Mbps"
echo "ping = $png ms"
# Updating download, upload and ping ..
wget -q "http://$host:$port/json.htm?type=command¶m=udevice&idx=$idxdl&svalue=$download" >/dev/null 2>&1
wget -q "http://$host:$port/json.htm?type=command¶m=udevice&idx=$idxul&svalue=$upload" >/dev/null 2>&1
wget -q "http://$host:$port/json.htm?type=command¶m=udevice&idx=$idxpng&svalue=$png" >/dev/null 2>&1
working and updating.
Perfect!
Thanks again
Re: monitor internet download speed with virtual sensor
Posted: Saturday 22 October 2016 19:52
by safi78
you're welcome & thanks for your input as well

Re: monitor internet download speed with virtual sensor
Posted: Saturday 22 October 2016 20:20
by jumbotroll
manjh wrote:This is great for the R-Pi, but is there an alternative for a Windows based server?
On windows 10 it's working here.
Install python 3.5.2 from
https://www.python.org/downloads/
Open cmd as Administrator and type: pip install speedtest-cli
Then you can type: speedtest
Re: monitor internet connection with virtual sensors
Posted: Sunday 23 October 2016 1:01
by simonrg
Nice approach, a couple of comments.
I only discovered that speedtest appears to download 30MB per test, when I used up my 50GB limit in 7 days by testing every 5 minutes

(~4GB per day

).
I took a slightly different approach based on a mash up of ideas from this forum.
A simple bash script runs speedtest:
Code: Select all
#!/bin/sh
/usr/bin/speed-test --json >/var/tmp/speed-test.json 2>&1
set to run by cron every hour on the hour (min=00). The script could also run via a timer in Domoticz to keep more of the setup inside Domoticz.
A Lua script runs at 3 minutes past the hour to read the file created by speedtest, decode the json, send the value to Domoticz via commandArray and send a notification if download value is below 3.0Mbps.
Code: Select all
-- ~/domoticz/scripts/lua/script_time_speed.lua
commandArray = {}
JSON = require('JSON')
date = os.date("*t")
function dospeed(filename, devdown, devup, devping, name, index)
-- filename - full path and filename where results from speedtest are stored
-- devdown, deveup, devping - Domoticz virtual sensor indexes
-- name - used in notification to identify isp
-- index - normal 1, increment by 5 for each additional internet connection tested
file = io.open(filename, "r")
json_text = file:read()
file:close()
speeds = JSON:decode(json_text)
print(name..": Ping="..speeds.ping.." Download=".. speeds.download.." Upload=".. speeds.upload)
commandArray[index]={['UpdateDevice']=devdown..'|0|'..speeds.download}
commandArray[index+1]={['UpdateDevice']=devup..'|0|'..speeds.upload}
commandArray[index+2]={['UpdateDevice']=devping..'|0|'..speeds.ping}
if speeds.download <3.0 then
commandArray[index+3]={['SendNotification']=name..' internet download speed alert#'..name..' download '..speeds.download..'Mb/s'}
end
if speeds.upload <1.0 then
commandArray[index+4]={['SendNotification']=name..' internet upload speed alert#'..name..' upload '..speeds.upload..'Mb/s'}
end
end
-- Once an hour
if (date.min == 3) then
-- os.execute('wget http://ipinfo.io/ip -qO - > /var/tmp/ip.txt')
-- 1st internet connection
dospeed("/var/tmp/speed-test.json", 958, 959, 960, "Eclipse", 1)
-- 2nd internet connection
dospeed("/var/tmp/speed-test-lnc.json", 961, 962, 963, "Boundless", 6)
end
return commandArray
Re: monitor internet connection with virtual sensors
Posted: Sunday 23 October 2016 7:54
by woody4165
Hi simonrg
one question is why you should test the line every five mins...
In any case, nice approach.
with speedtest-cli there is no --json option
where can I install from speed-test command? I tried sudo apt-get install speed-test
Thanks
Re: monitor internet connection with virtual sensors
Posted: Sunday 23 October 2016 10:23
by safi78
Nice example with JSON, like it.
But same here, the --json doesn't seem to be included in the debian package? What kind of speedtest are you using?
Regards,
Sander
Re: monitor internet connection with virtual sensors
Posted: Sunday 23 October 2016 12:46
by simonrg
woody4165 wrote:one question is why you should test the line every five mins...
In any case, nice approach.
with speedtest-cli there is no --json option
where can I install from speed-test command? I tried sudo apt-get install speed-test
Why every 5 minutes? Why not? Ok because it uses up my bandwith, but a realtime monitor would allow me to spot issues early and sort them out.
I have both a fibre and an over the air long range wifi-connection. While speed at one time can appear to be fast, I still to see stuttering on streaming services etc., so I was trying to see if there was anything systematic - farmer moving tractor into line of sight, wind blowing trees into line of sight etc..
So I am not sure once an hour is really useful to track long term trends, but not really useful to me for intermittant faults, as fault could already have happened 55minutes ago, and more annoylingly I could leave home after fault has occured without realising, so could have fixed it if I had known. Basically speed test is not the not the way to do it, I really need access to ISP network infrastructure as this will have the information I need on quality of connection with no unnecessary bandwidth.
For example, this morning

- 161023Speed.jpeg (39.31 KiB) Viewed 22223 times
my speed has dropped from 10 to <2Mbps, I have no idea why, no wind no rain no obvious obstructions, but lets see what ISP says.
I am running on Raspberry Pi, I got speed-test from
https://www.npmjs.com/package/speed-test.
I had forgotten my code is largely copied from
viewtopic.php?t=11970#p86162 which has a lot more discussion of measuring network speed.
Re: monitor internet connection with virtual sensors
Posted: Sunday 23 October 2016 15:38
by safi78
simonrg wrote:
Why every 5 minutes? Why not?
Because it will try to suck the shit out of your connection every 5 minutes and that will affect everything else you do?
Even if it is just 30MB every 5 minutes you'll have major lag on everything else right.
Re: monitor internet connection with virtual sensors
Posted: Sunday 23 October 2016 17:08
by Derik
@Safi78
Try it:
Code: Select all
pi@raspberrypi ~ $ sudo apt-get install speedtest-cli
Pakketlijsten worden ingelezen... Klaar
Boom van vereisten wordt opgebouwd
De status informatie wordt gelezen... Klaar
E: Kan pakket speedtest-cli niet vinden
pi@raspberrypi ~ $
Edit:
Did some work:
Then:
Then try the script :
cd pad to the script:
then:
./scriptname.sh
And working...
Only what is the time when i use:
@hourley
Edit 2:
I do not like to run this script exactly @ x.00 hour.
So i made the crondtab:
Code: Select all
11 0 * * * /home/pi/domoticz/scripts/speedtest.sh
11 minutes past a hour..
Hope this is going to work, and not kill my internet speed every hour...
Thanks guys.. for this script!!
Re: monitor internet connection with virtual sensors
Posted: Sunday 23 October 2016 18:43
by Derik
mm
Only i stupid question...
What is the best ping high or low..?
Re: monitor internet connection with virtual sensors
Posted: Sunday 23 October 2016 18:55
by emme
low ping means fast response.... so it's good
high ping means slow response... that' bad
Watch out the unit: normally it's ms-->milliseconds