New compare graphs

[Supertramp4]

Hi all,
Really like the new compare graphs - thanks, however I believe it is displaying the average of the 'average' dataset?

Is it possible to me to change this to display the average of the 'maximum' for the monthly view?

[lost]

Is it possible to me to change this to display the average of the 'maximum' for the monthly view?

You can already do this, with a virtual sensor that records true max. General issue with datasets is they are sampled on a 5mn basis. Not an issue for data that do not change quickly (like temperature/pressure...) but I myself had to build something better for my global home electric power records:

The power meter (Qubino Smart-Mater) is tuned to send power measurements if they change vs last sent measure >= 2%, or every 120s (2mn, worst case)... so in 5 mn lots of variations are just lost and they can be huge.

=> I have 2 virtual meters that records true max (this one is easy) as well as averaged power (for this one, that's discrete values integration over time slices, summing 'measure*time since device last-update' in the end divided by time slice duration using virtual last update) over a time slice target (that's device scripts, so cannot enforce an exact value) of 10mn.

For instance, my last 24h average power graph (as you can see, low cost electricity hours between 22h00 and 6h00 are quite well optimized today as high price hours are huge theses cold days!) virtual meter:

pAvegage.jpg (17.92 KiB) Viewed 1364 times

Here is the corresponding max virtual graph:

pMax.jpg (19.42 KiB) Viewed 1364 times

And the domoticz power figures of the device attached to the (true, not virtual) hardware meter, what's you get by default:

pTrueMeter.jpg (19.25 KiB) Viewed 1364 times

As you can see, last figure is not the best representation of my true home power drain. Average is IMO the best and true max (the one domoticz record by default was 2.5kW under true max here!) is useful to limit power drain according to electric provider contract to avoid a sudden home shutdown.

Of course, for theses virtual devices, you still benefit from week/month graphs as well as year comparisons since they were active!

[waltervl]

Supertramp4 is referring to the yearly compare graph (most bottom graph), new in 2024.x They are now available not only for (energy) counters but also for every other sensor.
For temperature sensors you are able to switch between Min, Max or Avarage to compare. For other sensors it seems only to compare Avarage.
In general it should be possible to change but only when the data is there of course....

[Supertramp4]

Hi Waltervl,
Yes that's what I'm referring to. I have inputs coming from solar panels via mqtt and for some of the compare graphs it would make more sense for it to use the MAX data.
I have a full year of data so no shortage there.
Thanks

[lost]

For other sensors it seems only to compare Avarage.
In general it should be possible to change but only when the data is there of course....

That's IMO always an average of data that makes their way in the DB, every 5mn. As I said for data that do not change much in 5mn that's not an issue and these min/max/average are computed correctly, no need to build any complex setup here. But that's not the case for measurements that can change a lot over 5mn!

IMO that's not a question of having data, would make a huge DB increase to record all received values. That may be just keeping these 5mn time slice true min/max and computed average then send these exact values in DB.

Note my setup only works because I use a 10mn time slice target (can exceed a bit, that's dependant on 1st true HW report after the 10mn since last virtual sensor update is over), so over the 5mn DB sampling. Going under 5mn would mean data loss as well for the virtual meters.

[lost]

I have inputs coming from solar panels via mqtt and for some of the compare graphs it would make more sense for it to use the MAX data.

IMO, whatever the source is (MQTT or HW directly supported by domoticz core), same issue: If you get almost continuous power data feed, with lots of measurements in a 5 mn time slice, only last measure in this 5mn slice make it's way in domoticz DB and graphs.

Can probably be huge change for solar panels on a scattered clouds day...

See my 2nd and 3rd graphs: Around 23h00 true max power drain was ~10kW and domoticz sampling got only ~7.5kW! That's huge difference.

[waltervl]

For other sensors it seems only to compare Avarage.
In general it should be possible to change but only when the data is there of course....

That's IMO always an average of data that makes their way in the DB, every 5mn. As I said for data that do not change much in 5mn that's not an issue and these min/max/average are computed correctly, no need to build any complex setup here. But that's not the case for measurements that can

These compare graphs we are talking about here are based on the calendar_xxx table data that is only has a daily min/max/avarage entry.
Example the counter compare below. A temperature has a selection box to select Min/Max/Average

Compare_graph.jpg (212.87 KiB) Viewed 1318 times

[lost]

These compare graphs we are talking about here are based on the calendar_xxx table data that is only has a daily min/max/avarage entry.

I know. Issue IMO is you'll get true (or at least very close) values for data like temperature with very smooth changes considering the 5mn sampling without any processing between sample times. But that's not the case for a power meter for instance.

And IMO true values are not stored or otherwise computed (for average, I mean from each received data): My current max power drain in month graph was last 25/12 at 10.4kWh in my virtual meter that store them.

In base device exact same monthly graph, that's 6.2kWh: That's ~1.7 error factor!

So I still believe someone who wants to get true min/max/average values for devices data with huge possible variations in a 5mn time slice must currently script those in a virtual meter. Otherwise they get historical data that can be far from reality.

At the time, here is the device script extract showing what is done to feed my max power virtual meter from real hardware, as well as average/mean (using 2 user variables to store current power max and sum/integration over time deltas):

Code: Select all

-- Smart-Meter Mean/Peak :
if (devicechanged[devSmartMeterPwr]) then
    curPwr = tonumber(devicechanged[devSmartMeterPwr])

    -- Immediately log & return on bad pwr figures...
    if (curPwr > maxPeakPwr) then
        print('WARNING : '..devSmartMeterPwr..' reports '..curPwr..'W !!!')
        return commandArray
    end

    -----------------------
    -- Update Peak power --
    -----------------------

    pkPwr  = tonumber(otherdevices[devPeakPwrVrtSen])
    tLastPk= timeLastUpdate(devPeakPwrVrtSen, false)

    -- DEBUG :
    --print(devSmartMeterPwr..': P='..curPwr..'W ('..type(curPwr)..')')
    --print(devPeakPwrVrtSen..': P='..pkPwr..'W ('..type(pkPwr)..') ; Last : '..tLastPk..'s')

    -- Peak Pwr is stored rounded...
    if (curPwr > (pkPwr + 1)) then
       -- Log all intermediate peaks (with events trigger to allow overload management)
       updateNum(devPeakPwrVrtSen, curPwr, true)
    else
       -- Log last slice peak + "reset" new intermediate peak
       -- to current pwr (without events trigger) if slice time exceeded.
       if (tLastPk > maxLastUpdPeak) then
           print(string.format('%s : New time slice (since %ds) => Got Pmax=%.2fW', devPeakPwrVrtSen,  tLastPk, math.max(pkPwr, curPwr)))
           updateNum(devPeakPwrVrtSen, curPwr, false)
       end
    end

    -----------------------
    -- Update Mean power --
    -----------------------

    tLastMean = timeLastUpdate(devMeanPwrVrtSen, false)

    -- DEBUG :
    --print(devSmartMeterPwr..': P='..curPwr..'W ; Last : '..tLastMean..'s')

    pwrLast   = uservariables[varPwrLast]
    pwrSum    = uservariables[varPwrSum]
    sumDeltaT = timeLastUpdate(varPwrSum, true)

    -- Calc pwr sum + store. Set current mean pwr if time slice exceeded & reset user vars.
    if (tLastMean > maxLastUpdMean) then
       meanPwr = (pwrSum + pwrLast*sumDeltaT) / tLastMean

       --Log update :
       print(string.format('%s : New time slice (after %ds) => Record Pmean=%.2fW', devMeanPwrVrtSen,  tLastMean, meanPwr))

       updateNum(devMeanPwrVrtSen, meanPwr, true)
       commandArray['Variable:'..varPwrSum]  = '0'
       commandArray['Variable:'..varPwrLast] = tostring(curPwr)
    else
       pwrSum = pwrSum + pwrLast*sumDeltaT

       -- DEBUG :
       --print('Update current Psum='..pwrSum..'Ws ; deltaT='..sumDeltaT..'s')

       commandArray['Variable:'..varPwrLast] = tostring(curPwr)
       commandArray['Variable:'..varPwrSum]  = tostring(pwrSum)
    end
end

[waltervl]

I know what you are talking about but is is not what topic starter is asking requesting... I am already glad they are created so you can see the trend.

[lost]

I am already glad they are created so you can see the trend.

Must agree, as I never dig in DB, did not even know there was data from 2016 there for hardware that was present till I started with domoticz. That's really nice addition & virtuals designed for one's needs will also benefit from them.

[Supertramp4]

Hi,
So back to my original question, where in the code would I need to change in order to perhaps enable the selection box that temperature has, but for any sensor type, so that min/max/Ave can be displayed on the compare graph?

[waltervl]

These were the commits to get it implemented:
https://github.com/search?q=repo%3Adomo ... pe=commits

[gizmocuz]

@Supertramp4, to be fair, if you do not know how to search the code, I don't think you are able to make a patch for this.
It took me many hours to figure most of the stuff out.
You need 'searching' skills, angular/JS skills, c++ skills to make a change.

There are multiple tables where all information is stored in.
For most non counters devices we do store the min/max/avg values.
For counter types (like a kWh sensor) you have to do the math yourself.
But yes, the Temperature chart is a useful example.

I can tell you it is not a matter of just adding the dropdown and it magically works!

[gizmocuz]

Besides, for counters (like managed or kWh) there is no min/max/avg, it's just the value (yes... the counter)

[joostvanderlinde]

Popping in on this question of the compare graph.

I have a managed counter that records the hourly energy prices (date + timestamp)
Once a day I store the average price per day into the history of the managed counter (date without timestamp)
The compare graph seems to be following that last daily store.

Screenshot from 2024-11-11 10-03-00.png (285.45 KiB) Viewed 594 times

But I cannot figure out what the value is that is shown in the compare grahh in my case.
It is NOT the sum or count or avg of the values in the history graphs (last week/last month). The number is much lower than the sum of the history graph figures.
Summed AVG daily historic price in Oct: 273463
Compare graph value in Oct: 177728

Note: as I have been fiddling arpund (an debugging my dzVents code) I have updated the managed counter several times for the same period. Not sure if that might be the cause of the unrecognized value. If it is: How can I fix that?