Upgrade to 2020.2 with Zwave WITHOUT fatal error.

[JuanUil]

I have a AEON Labs ZW090 Z-Stick Gen5 EU as well.
So I can delete ozwcache*.xml without any problem an then waken up my battery powered nodes and that should do the trick?

[JuanUil]

Just to be sure:
My plan is:
- delete ozwcache*.xml
- reboot my pi
- refresh node info and wake up my battery powerd nodes.

I will keep a copy of the old ozwcache file.

is this correct?

Jan

[Domoticzuser3109]

Correct, be sure to press within 5 seconds as the node refresh will likely timeout. Other option is to just use the include button in domoticz and on your device. It works the same as refresh as far as I can see but gives more time. The nodes are already known so the won’t be added again.

[lost]

My plan is:
- delete ozwcache*.xml
- reboot my pi
- refresh node info and wake up my battery powerd nodes.

I would stop domoticz service before deleting ozw cache file. Not sure if this is really needed, but that's always good care not to modify/suppress files that may be used by a running process anytime.

[EddyG]

You should stop domoticz before delete the cache file.

[Domoticzuser3109]

True! If you don’t stop domoticz it wil not change anything till restarting domoticz, but when restarting domoticz it will first write a new cachefile on closure.

[FearNaBoinne]

I am also having issues with Z-Wave devices not coming back or not being identified, and all of those are secure devices as well...

[Ben0it]

Thanks for the hint, my OpenZWave OZW-driver stack was randomly crashing with a fatal error :

Code: Select all

[size=85][...]
2020-01-07 17:53:31.306  Error: Domoticz(pid:7084, tid:7179('OZW-driver')) received fatal signal 11 (Segmentation fault)
2020-01-07 17:53:31.307  Error: siginfo address=0x24, address=0x55797bb160fc
2020-01-07 17:53:31.316  Error: Failed to start gdb, will use backtrace() for printing stack frame

2020-01-07 17:53:31.321  Error: #0  /home/domoticz/domoticz/domoticz :  + 0x3ecd53 [0x55797b67dd53]
2020-01-07 17:53:31.321  Error: #1  /home/domoticz/domoticz/domoticz : signal_handler(int, siginfo_t*, void*) + 0x205 [0x55797b67e7b5]
2020-01-07 17:53:31.321  Error: #2  /lib/x86_64-linux-gnu/libpthread.so.0 :  + 0x12730 [0x7f6d1d62f730]
2020-01-07 17:53:31.321  Error: #3  /home/domoticz/domoticz/domoticz : OpenZWave::Node::GetValue(OpenZWave::ValueID const&) + 0x4c [0x55797bb160fc]
2020-01-07 17:53:31.321  Error: #4  /home/domoticz/domoticz/domoticz : OpenZWave::Manager::IsValueValid(OpenZWave::ValueID const&) + 0x3a [0x55797bacc6da]
2020-01-07 17:53:31.321  Error: #5  /home/domoticz/domoticz/domoticz : COpenZWave::UpdateNodeEvent(OpenZWave::ValueID const&, int) + 0x2b [0x55797b8d92ab]
2020-01-07 17:53:31.321  Error: #6  /home/domoticz/domoticz/domoticz : COpenZWave::OnZWaveNotification(OpenZWave::Notification const*) + 0xfd3 [0x55797b8f1343]
2020-01-07 17:53:31.321  Error: #7  /home/domoticz/domoticz/domoticz : OnNotification(OpenZWave::Notification const*, void*) + 0x22 [0x55797b8f18c2]
2020-01-07 17:53:31.321  Error: #8  /home/domoticz/domoticz/domoticz : OpenZWave::Manager::NotifyWatchers(OpenZWave::Notification*) + 0x85 [0x55797bac9085]
2020-01-07 17:53:31.321  Error: #9  /home/domoticz/domoticz/domoticz : OpenZWave::Driver::NotifyWatchers() + 0x7a [0x55797bafe6fa]
2020-01-07 17:53:31.321  Error: #10 /home/domoticz/domoticz/domoticz : OpenZWave::Driver::DriverThreadProc(OpenZWave::Internal::Platform::Event*) + 0x2f8 [0x55797bb0b438]
2020-01-07 17:53:31.321  Error: #11 /home/domoticz/domoticz/domoticz : OpenZWave::Internal::Platform::ThreadImpl::Run() + 0x13 [0x55797bb73db3]
2020-01-07 17:53:31.322  Error: #12 /home/domoticz/domoticz/domoticz : OpenZWave::Internal::Platform::ThreadImpl::ThreadProc(void*) + 0x9 [0x55797bb73dc9]
2020-01-07 17:53:31.322  Error: #13 /lib/x86_64-linux-gnu/libpthread.so.0 :  + 0x7fa3 [0x7f6d1d624fa3]
2020-01-07 17:53:31.322  Error: #14 /lib/x86_64-linux-gnu/libc.so.6 : clone + 0x3f [0x7f6d1d3a74cf]
[...][/size]

Since I allowed new nodes (allow new devices for 5 mins in Settings) and refreshed each OZW node manually, everything ticks like a Swiss clock and is very reactive to events.

Versions so far :

Code: Select all

 - OpenZWave : 1.6-1115-g6289ebb5
 - Domoticz : 2020.2 built 12017

OpenZWave devices show a highlighted ID in their identifier :

[akamming]

I have been reading around here for a while due to the fact that I also had major problems with my zwave network after upgrading to 2020.2

So far I found out:
- Enabling the allow new hardware option prevents domoticz from crashing after the upgrade, this is a great find!
- Some nodes where still being interviewed each time I restarted domoticz.
- After repairing all the nodes using the include button and wake them up again (the old node activates again, no new node gets added) I took a snapshot of the ozwcache file.
- Then I shutdown Domoticz and made another snapshot from the cache file
- It turns out that all my troubled nodes keep being deleted from the cachefile when i shutdown domoticz (sudo service domoticz stop)!
- I opened up the cachefile which was taken with all the nodes working and searched two identical nodes. One was being deleted and needed manual wakening each time, the other was working fine.
- It turned out that the working node was added as a non secure device, while the exact same node that was forgotten each time was added as a secure device.
- So I looked in it some more, and all my troubled nodes where added as secure and where reinterviewed each time again. After which they worked perfectly until the next restart of domoticz.

I altered the cachefile which had all the nodes in it and removed all instances of:

Code: Select all

secured="true" 

If i shutdown domoticz I see the cachefile being broken again (size is halving). When i copy the altered cachefile (the one with secured="true" removed) over the broken cachefile and start domoticz everything is running fine straight away.

Hope one of the developers can replicate this issue and sort this out. I think these two problems (new harware must allow and devices must be unsercure connected) are the two main issues with Zwave right now.

just curious: do you have polling enabled on some devices? If so: Could you try disable polling on all your nodes before you reboot and see what happens? There is a bug in openzwave which truncates the cachefile on exit of the application if polling is enabled on some nodes… i found a solution (https://github.com/OpenZWave/open-zwave/pull/2227) and am looking for feedback of other users...

Since I allowed new nodes (allow new devices for 5 mins in Settings) and refreshed each OZW node manually, everything ticks like a Swiss clock and is very reactive to events.

as a hobby programmer i am investigating this one. I already found a statistical relation between a bad cachefile (in my case caused by the bug in openzwave i mention above) and the segfault. all users who found a way around the segault rebuilt their cache file (e.g. fresh install, deleting ozwcache, or in your case: doing a refresh on every node. This also leads renews the info in the ozwcache). So i would be very interesting to compare the contents of your cache file before and after the refresh of the devices, so hopefully we can find a difference which causes the crash.

So do you still have you old cachefile (before the crash) available? If so, could you make a comparison with your current one?

[Ben0it]

I compared the cache files, here are the differences I can see :

• in the new cache, I have new entries like this in the CommandClass/State section :

  Code: Select all

  <CCVersion>1</CCVersion>

• new cache size is more than twice the size of the previous cache file

[akamming]

I compared the cache files, here are the differences I can see :

• in the new cache, I have new entries like this in the CommandClass/State section :

  Code: Select all

  <CCVersion>1</CCVersion>

• new cache size is more than twice the size of the previous cache file

Can you run some tests by omitting some of this data in tje ozwcache file and see if you can this way trigger the segmentation fault

[Ben0it]

I stopped Domoticz, removed every single instance of CCVersion from the cache file, the restarted Domoticz. What I could see, is that the entries with CCVersion came back, they are recreated automatically. No crash to blame so far.

Another information : when I upgrade, I checkout the latest version in a new folder, recompile everything, copy my previous domoticz.db and lua scripts to the new directory and start Domoticz again : so the ozwcache is never reused from an older version in my case, it is recreated automatically.

[akamming]

I stopped Domoticz, removed every single instance of CCVersion from the cache file, the restarted Domoticz. What I could see, is that the entries with CCVersion came back, they are recreated automatically. No crash to blame so far.

Another information : when I upgrade, I checkout the latest version in a new folder, recompile everything, copy my previous domoticz.db and lua scripts to the new directory and start Domoticz again : so the ozwcache is never reused from an older version in my case, it is recreated automatically.

Aha clear. Was worth a shot. But i understand you jave a different upgradepath than most users....

[JuanUil]

Hi Guys,

Today deleted my ozwcache* file.
Restarted domoticz and woke up all battery powered nodes.
After restart all nodes are read from the file so problem is solved for me

[akamming]

Beta 12076 (released today) contains the fix for openzwave corrupting the ozwcache. Fixes issues for installations which use polling

[Domoticzuser3109]

Yes! I tried beta build 12077. Did do this:

- update rpi with sudo apt-get update and sudo-apt upgrade
- update Domoticz to latest beta
- started Domoticz to see zwave in trouble with lost nodes again
- sudo service stop domoticz
- copied the backup of my ozw cachefile over the broken cachefile
- sudo service start domoticz
- checked zwave had all the nodes again
- stopt domoticz again, normally zwave cachefile would break at this point
- started domoticz and checked the zwave nodes. No nodes lost this time!
- verified it with sudo reboot now
- again no nodes lost!

I indeed have polling enabled for 3 nodes (with fixed 230v power) and this seems to have fixed the caching issue. Hands down for solving this bug!

[roblom]

So far so good at my side also. Thanks!

[Domoticzuser3109]

Well, it was running perfectly for months. Then I added some new curtain motors and everything was running fine till the next reboot of the RPi. After the reboot Zwave lost some nodes again. Restoring the cachfile from the backup (see how-to above) fixed it. As I saw that there where config update for Zwave in the new Beta's from Domoticz i decided to upgrade to Beta Build 12491

Once more, I lost all the ZWave nodes during this upgrade. The non sleeping restored after a while, but the sleeping nodes where not (not even after sending their data correctly). Restoring the cahcefile with the cachefile from the backup did not solved the problem. After some research I found out that the ozwcache_*******.xml file is now rewritten on closure (as before) but also while starting the ZWave driver!

I have found no way to restore the sleeping nodes other then to go to every physical device and start a include command on the device. It is not needed to set Domoticz in 'node including mode'. This is however a painfully long process if you have as many nodes as I do Worst thing is that on one of the last nodes Domoticz froze and needed to be rebooted....losing all the sleeping nodes again.

Why does ZWave overwrite the cachefile on startup? I dont want my cachefile ever being deleted beside the one time I command it to be deleted. Can anyone help to solve this unwanted deleting?

[Domoticzuser3109]

Updated to latest Beta 12639 but still the same problem with the xml file which is being overwritten.

I am not alone with this problem, these seem all related to Zwave losing its cachfile:
viewtopic.php?f=24&t=33892
viewtopic.php?f=24&t=32667
viewtopic.php?f=24&t=34494
viewtopic.php?f=24&t=34458

Anyone who has a idea to fix this? When including all nodes by resending include (refresh node) command work fine till next reboot or update.

[EddyG]

You should not lose your cache file if you wait long enough for ALL devices to include properly.
See first post.
As soon as ALL devices are created the cache file will not be lost anymore, and at the start of Domoticz it will be used. Be patient