dzVents 2.0 just got integrated into Domoticz (beta v3.8023)

[dannybloe]

Don't worry about that. I see you have debug mode enabled. That does slow things down though.

[jeanclic]

Don't worry about that. I see you have debug mode enabled. That does slow things down though.

Yes, it is enabled on my test VM, but disabled on my Pi (not possible to used anyway if I enable dzVents...

-> I do not use any version of dzVents: no "dzVents" folder in my "lua" folder...
-> "dzVents_settings_example.lua" is back inside "/scripts/dzVents" when I force a beta update via ./updatebeta, and I have errors if I have dzVents enabled on the web interface and no "dzVents_settings.lua" file in "/scripts/dzVents".

PS: did you see my post on the other topic ? http://www.domoticz.com/forum/viewtopic ... 20#p140066

[dannybloe]

That's totally weird as in the repo there is no such file in that folder.

Screen Shot 2017-07-02 at 16.37.09.png (123.3 KiB) Viewed 3154 times

[jeanclic]

That's totally weird as in the repo there is no such file in that folder.

Just tested it again : I moved the incriminated file into a "old" folder, and then redo the ./updatebeta.

this time it did not came back, and my dzVents script is still working... (on my test VM). I do not understand what this file was doing there at the first place, because I will never have thought about creating it ! -> now I will try to remove it on my RPI and see what happen *crossing fingers*

[Derik]

dear...
What is status of Dzvents???
When i enable the option my CPU is running +10%...
Is this normal?
Or is there a thing that i do wrong?

[SweetPants]

Fixed in latest Beta

[dannybloe]

We are preparing some more updates that will bring down cpu usage a lot more. Stay tuned...

[Derik]

ok... hope i learn some scripting with Dzvents...
Thanks for the great work

[heggink]

Just upgraded as well to latest version. putting any dzVent script in scripts makes CPU go up/down violently (from few %% to 100% and back), removing the script calms it down again. Updated OS to the latest version as well so fully updated jessie.

[dannybloe]

Yes, we know. We are currently optimizing this. Gathering the data being handed over to the scripts takes quite some time right now. We are currently testing some optimization schemes to make this a fast as possible without having to rewrite parts of Domoticz significantly. But it will take cpu time no matter what. But then again.. what good is a cpu if it is never tickled to do what it was designed for? But you will see the spikes mostly on slower machines (like Pi1, P2) with many devices that are updated very frequently.

[heggink]

Agree as long as it is workable. On my pi3, the web frontend became quite unresponsive.

[heggink]

Strange tho that direct integration (v2) draws so much more than this was a lua addon (v1). Never saw that kind of load previously.

[dannybloe]

It's not strange. Previously with 1.x, dzVents relied on having a pre-fetched json table ready to be loaded every time dzVents was invoked. That json data had to be fetched on a regular interval (also done by a kind of internal dzVents script that runs every so many minutes as defined in the dzvents_settings.lua file). The problem with that was that this prefetched data was alway old unless you had it updated every minute (even then). But that is very costly cpuwise. This old solution had other problems as well as this data had to be streamed to the filesystem every time and then read by dzVents again into memory, parsed and processed. So now it is integrated, Domoticz prepares this data using live data. So now, all devices you can access in your scripts have live data which is what you want of course.
And also, 2.0 has changed in how you access these devices in such a way that only queried devices(devices you really use in your script) are processed instead of all devices at dzVents startup. All this made sure that your script can now run in way less milliseconds than before. Alas, the preparation of the data that is fed to dzVents is something that we are now optimizing as that takes most of the time.

[simonrg]

And also, 2.0 has changed in how you access these devices in such a way that only queried devices(devices you really use in your script) are processed instead of all devices at dzVents startup. All this made sure that your script can now run in way less milliseconds than before. Alas, the preparation of the data that is fed to dzVents is something that we are now optimizing as that takes most of the time.

Would it be possible to take a different approach of turning otherdevices tables into functions so that only the required information is retrieved in realtime when needed in the Lua script, perhaps as a look up from the database.

The original idea of filling otherdevices tables was I presume to reduce load by only doing it occasionally, but sending all realtime data will result in a lot of unnecessary information being collected multiple times, hence reducing the information collected will help. Isn't the logical extension of this the function in Lua to retrieve the information when needed?

Information in many of my scripts will be difficult, as the devices that are queried are calculated based in the script, so very many values would need to be prefetched to account for all circumstances.

[simonrg]

Yes, we know. We are currently optimizing this.

Has this optimisation now been done?

I have installed 3.8063 on my Domoticz systems and had to disable DzVents on my slave(Raspberry Pi B version 1, only 4 devices and 2 Lua scripts active), however while the normal CPU load is 1.5%, when I attempt to access the web frontend with DzVents active CPU load goes to 98%. Attempting to disable DzVents wasn't initially possible as even when the settings page displayed there was no option for DzVents, despite multiple forced refreshed (Ctrl-R Firefox Win7), it was only when I disabled all scripts that CPU usage dropped and refreshing brought up the DzVents options. So I have now disabled DzVents and restarted enabled scripts, the web frontend is now responsive as before.

Any suggestions?

[dannybloe]

yeah, we are preparing a pull request that should solve this. On some systems we see this happening.

[simonrg]

yeah, we are preparing a pull request that should solve this. On some systems we see this happening.

Great, thanks.

[heggink]

Interested to hear how this is progressing. Any feel around an ETA?

[dannybloe]

The pull request was ready yesterday

[heggink]

So waiting for Giz to accept and build into new beta