Sensor data sanity check

Use this forum to discuss possible implementation of a new feature before opening a ticket.
A developer shall edit the topic title with "[xxx]" where xxx is the id of the accompanying tracker id.
Duplicate posts about the same id. +1 posts are not allowed.

Moderators: leecollings, remb0

Post Reply
TSkar
Posts: 29
Joined: Wednesday 20 June 2018 16:47
Target OS: Linux
Domoticz version: 2.13055
Location: Stavanger, Norway
Contact:

Sensor data sanity check

Post by TSkar »

Occationally Domoticz get's sensor data that is clearly wrong. This is by no means specific for Domoticz; any smart sensor occationally mess up and report wrong data, signals aren't perfect, and so on.
..but why not an option to automatically discard data that is clearly wrong?

- Temperature drops instantly over/under 30 degrees, when the rest is stable, and there is no fire, nor being in a freezer.
- kWh meters steady on 1-200W, and sudden spike of 100 000 000 Watt - or negative values.
- Lux reading indicating a god descended in the bathroom..

For me 99% of the time it's network issues and/or sensors messing up.
Like everyone else I do occationally experiment, try out new devices, or similar more or less smart things, where I would want to know if something is way outta line. Turning this on for a Motion detector, Fire Alarm, Flood Sensor, or similar, would also obviously be a bad idea, hence the optional toggle per device.

I guess it would also be smart to have a counter for values out of sanity, and/or a message in log - so that it's possible to get notification if it happens too frequent.
TSkar
Posts: 29
Joined: Wednesday 20 June 2018 16:47
Target OS: Linux
Domoticz version: 2.13055
Location: Stavanger, Norway
Contact:

Re: Sensor data sanity check

Post by TSkar »

While trying to figure out logs after a z-wave node crashed, I realized Domoticz already does this to some extent:
"2021-04-08 15:09:44.075 Error: OpenZWave: Invalid counter value received!: (-16106127.000000) Node: 3 (0x03), CommandClass: METER, Instance: 1, Index: 0, Id: 0x34C8012"
What I tried to suggest, was a way to further limit what Domoticz sees as valid/invalid values - like not all nodes can have negative values, detect outliers too far off norm, and so on. I'm not suggesting to remove these, beyond what Domoticz does today; just the option to set the limits for what Domoticz accept as valid values.
Toulon7559
Posts: 843
Joined: Sunday 23 February 2014 17:56
Target OS: Raspberry Pi / ODroid
Domoticz version: mixed
Location: Hengelo(Ov)/NL
Contact:

Re: Sensor data sanity check

Post by Toulon7559 »

Like Alverta writes
(that the material's creator was thinking rationally, applying sanity)
it is the user which functionally determines the sanity of a situation:
Domoticz (or any other software) might be an aide, if the user programs the check according his/her rationale.
Last edited by Toulon7559 on Friday 15 October 2021 10:49, edited 1 time in total.
Set1 = RPI-Zero+RFXCom433+S0PCM+Shield for BMP180/DS18B20/RS485+DDS238-1ZNs
Set2 = RPI-3A++RFLinkGTW+ESP8266s+PWS_WS7000
Common = KAKUs+3*PVLogger+PWS_TFA_Nexus
plus series of 'satellites' for dedicated interfacing, monitoring & control.
User avatar
JackD
Posts: 68
Joined: Monday 02 November 2020 20:53
Target OS: NAS (Synology & others)
Domoticz version:
Contact:

Re: Sensor data sanity check

Post by JackD »

I don't fully agree.....

I have 12 temperature sensors 433mhz and 3 temp sensors running on z-wave. Cheap Ali sensors and expensive brand sensors, they all seem to never give a false reading(knock on wood :lol: ).

But still I understand your desire to have the functionality to automatically get rid of the most extreme spike readings if and when the might occur.
That said, who decides what sanity would be. Temperature you could automatically get rid of below -50 and above 60 but what about if the temperature go's in 5 minutes from 25 to 10 degrees celcius.... could in some circumstances be right but could also be false. "Sanity" is not that easy to program :geek:
mancide
Posts: 29
Joined: Wednesday 17 May 2017 0:18
Target OS: Raspberry Pi / ODroid
Domoticz version: 2024.6
Location: Belgium - Ghent
Contact:

Re: Sensor data sanity check

Post by mancide »

Hi,

I'm have similar problems.
I have some temp sensor that sometimes report 70C
I have some wallplug energy meters that sometimes report negative values of eg 7000kWh in a hour.

Optional limits on a sensor would be a nice thing to have. Of course it is up to the user to set correct limit values.
Raspberry Pi 3B v1.3 - 16GB SDcard - Raspbian Bullseye
Aeotec Z-Stick Gen5 (ZW090) -> Z-Wave JS UI -> mosquitto
RFXCOM RFXtrx433E
Unifi controller -> Domoticz-Unifi-Presence plugin
Nest thermostat
SAJ solar convertor
Smart Gateway Gas/water meter
Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest