Upgrade to 2020.2 with Zwave WITHOUT fatal error.

[EddyG]

I managed to get Domoticz upgraded from 4.10717 to 2020.2 without the fatal error in Domoticz, which happen when people have Zwave devices.
First before upgrading enable new devices Setup -> Settings -> enable "Accept new hardware devices" Don't forget to click "Apply settings"
Then upgrade! That's was it all, at least for me. Hopefully it works for others too.
Reason:
With OZW 1.6 new devices are created. In my case a lot of devices with the name "Previous Event Cleared"
Device as Fibaro FGMS001-ZW5 also create a few other devices.
Initially every battery powered devices has to wakened or included again, those devices with all zero's in the Zwave overview.
You have to check if every hardware device has an accompanying device with 2 zero's at the end of the ID.
If not, select the hardware device in the Zwave overview and click on "Refresh Node Info" Be patient.
Wait until the 2 zero device is present en proceed to the next until all hardware devices have a device with 2 zero's at the end.
Some devices already have such a 2 zero device from OZW 1.4, mainly the battery powered.
Leave the "Accept new hardware devices" enabled because some devices come after sometime with an extra device.
In my case I have an "Shenzhen Neo Electronics Co Ltd Siren Alarm+" Siren and the extra device came after more then 12 hours.
THATS ALL. At least for me this worked.

The fact that I did not discover earlier is that I am surrounded by a lot, I mean a lot, of neighboring device belonging to someone else.
I do not want those so my setting are always disable "Accept new hardware devices"
I once had it enabled and it went fine for a while, but I did not know how long to wait for the last device, see my siren.
So I disabled it so soon and got into the error again.

I believe that the fatal error is coming from the fact that a value has to updated on a device that does not exist or that the device could not be created. This is a challenge for the developers to find out and correct.

[waaren]

I managed to get Domoticz upgraded from 4.10717 to 2020.2 without the fatal error in Domoticz, which happen when people have Zwave devices.
First before upgrading enable new devices Setup -> Settings -> enable "Accept new hardware devices" Don't forget to click "Apply settings"
Then upgrade!

Many thanks for you thorough analysis and tip on how to upgrade without facing this nasty bug
I am sure it will help many fellow forum members

[EddyG]

Tnx, I hope it helps others too to get the Zwave 1.6 problem solved.
I hope the developers can solve it for the future.

[Bikey]

Mmmm, I have to say that my experience is much smoother, luckily.

I took the plunge too, yesterday and only copied my domoticz.db from 4.1xxx RPI2b to a new fresh installed installation on another RPI2b. That worked fine, without the hardware. So the 2020.2 version works with the older database.
I'm not sure if it makes a difference but actually I think what I should have done is an "import" from the web interface (although I also wonder If the regular upgrade scripts ever really do a database migration or just copy the new files).

Anyway, I then I moved my Z-Wave.Me USB stick to the new system and enabled the hardware. My Z-Wave device came online pretty quicly, the only (tedious) thing to do was that I had to activate the battery powered PIR's by pressing the activation button 3 times as if they had to be re-included (but without setting Domoticz in include mode). After that it worked pretty well. I have also Fibaro FGMS001 and FGMS001-ZW5+ PIR's and NeoCoolcam PIR's.

The Neo Coolcam doorsensors, FIBARO System FGSD002 Smoke Sensor+ and even an Shenzhen Neo Electronics Co Ltd Siren Alarm+ that came up all by itself, I'm not sure if that was immediately, but the are here no after a night.

I have no new devices and the temperature devices from the Fibaro PIR's still have an Id without 2 zero's at the en that work great.

So just writing this to let people know hat perhaps starting fresh is also a strategy that my work for some (probably not for all)

[sammyke007]

My first Z-Wave stick+ devices arrive tomorrow, but I'm already at 2020.1. Great find! I hope this will help the dev with pinpointing the issue. Did you already add this at Github?

[djalexnl]

Does it also resolve the reboot bug?

Mine runs stable for weekse... Until i have to reboot.. Then have to restart the domoticz service Until it suddenly runs for more few minutes.

Also have add New devices disable... Due to neighbours..etc.

Once it runs it runs Perfect...

I know its zwave related..like you mentioned


Greetings Alex

[EddyG]

Does it also resolve the reboot bug?

Yes ,it does. Before the reboot, enable add devices and wait or enforce until all new devices are created.

[EddyG]

So the 2020.2 version works with the older database.
I'm not sure if it makes a difference but actually I think what I should have done is an "import" from the web interface (although I also wonder If the regular upgrade scripts ever really do a database migration or just copy the new files).

If you execute the 'new' 2020.2 domoticz executable the 'old' database is automatically adapted to the new version.
You cannot go back with a database only forward.
Strange that your Fibaro PIR does not create a seismic device and other devices.

[Basssment]

@EddyG
Would this also work when upgrading from 2020.1 to 2020.2?
OR do i have to get back to my old image first?

[EddyG]

Yes, it should work upgrading from 2020.1 to 2020.2.
Main thing is that before the upgrade you should allow new devices
And probably you should delete the ozwcache*.xml file (keep a copy) before the upgrade.
That file might be corrupted.
That file is in /home/pi/domoticz/Config for a normal install.

[EddyG]

So just writing this to let people know hat perhaps starting fresh is also a strategy that my work for some (probably not for all)

I also did start fresh, because I (and everybody else) had to upgrade there OS system from stretch (or jessie/wheezy) to buster.
After the switch from OZW 1.4 to OZW 1.6 things crashes. I came to that conclusion when I tried the first beta with OZW 1.6.
I left it there and tried the next upgrade with 2020.1 same result with crashes.
Now I found out that disabling the "Accept new Hardware devices" was the culprit for me.
Still strange that none of your Hardware devices have devices with 2 zero's at the if of the ID
I am talking of the 5th column in the device overview (between Hardware and Unit) and looks like 00011000

[EddyG]

There is 1 thing extra in the Domoticz logging about OZW that I do not fully understand.

Code: Select all

2020-05-07 08:47:24.776  OpenZWave: Value_Added: Node: 9 (0x09), CommandClass: SENSOR BINARY, Label: Sensor, Instance: 1, Index: 0
2020-05-07 08:47:24.777  OpenZWave: Value_Added: Node: 9 (0x09), CommandClass: ALARM, Label: Access Control, Instance: 1, Index: 6
2020-05-07 08:47:24.778  OpenZWave: Value_Added: Node: 9 (0x09), CommandClass: ALARM, Label: Previous Event Cleared, Instance: 1, Index: 256
2020-05-07 08:47:24.778  Status: SendSwitchIfNotExists: Device '9.instance.1.index.256.commandClasses.113' (Previous Event Cleared) with DeviceID '00000900' matches '9.instance.1.index.0.commandClasses.48' (Sensor). Domoticz will use the Dimmer (and hide the Switch).
2020-05-07 08:47:24.779  OpenZWave: Value_Added: Node: 9 (0x09), CommandClass: BATTERY, Label: Battery Level, Instance: 1, Index: 0

All my battery operated devices has already a so called a device with 2 zero's at the end of the ID.
This is one of them.
What I understand from this is that 2 commandclasses ( 113 and 48 ) fight for the same device.
Domoticz will use the Dimmer (and hide the Switch). does not tell who won the battle and what it implements.
Does anybody know

[Bikey]

So just writing this to let people know hat perhaps starting fresh is also a strategy that my work for some (probably not for all)

I also did start fresh, because I (and everybody else) had to upgrade there OS system from stretch (or jessie/wheezy) to buster.
After the switch from OZW 1.4 to OZW 1.6 things crashes. I came to that conclusion when I tried the first beta with OZW 1.6.
I left it there and tried the next upgrade with 2020.1 same result with crashes.
Now I found out that disabling the "Accept new Hardware devices" was the culprit for me.
Still strange that none of your Hardware devices have devices with 2 zero's at the if of the ID
I am talking of the 5th column in the device overview (between Hardware and Unit) and looks like 00011000

Update: I set the "Accept new Hardware devices" to on and did a "Heal Zwave network" yesterday.

That didn't go well, after a while Domoticz became unreachable and in the (Docker container) log I could see that the domoticz webserver was repeatedly restarting.

After a restart of Domoticz however everything works again. However I now do have new devices, like the "seismic intensity".
And also an number of "Previous Event Cleared" devices.

As said before the device ID's of the thermometers (that work fine) still have no 00's at the end: E.g. for a FIBARO System FGMS001-ZW5 Motion Sensor, the IR switch-device is "00011E00" and the temperature device is"1E01". This is only for the temperature devices, all others have the 00 at the end.

[EddyG]

What I ment is that from 1 Hardware device (like a PIR device) several devices are created in Domoticz.
One of those device has an ending with 2 zero's on the ID. Other devices differ in all kind of ways.
Mostly the new created "00" ending devices got the name "Previous Event Cleared"
But mostly on battery device from the previous OZW 1.4 they already have a "00" device.
That where this is about. It is not an error, everything is working fine.
But I hope that someone can explain what this is about.

Code: Select all

2020-05-07 08:47:24.778  Status: SendSwitchIfNotExists: Device '9.instance.1.index.256.commandClasses.113' (Previous Event Cleared) with DeviceID '00000900' matches '9.instance.1.index.0.commandClasses.48' (Sensor). Domoticz will use the Dimmer (and hide the Switch).

[djalexnl]

Rebooted.. Today after accept New hardware..Switches on...Well nothing changed... Still hangs after one Minute...
After many attempts it start running again... So for me its NOT a fix...

Also i cannot find any New devices... Anyone Has a screenshot what is meant??

[EddyG]

You have to provide a little bit more info over what you are running if you want to be helped.
Domoticz version, OS version, what hardware device, logging, crash report.

[Domoticzuser3109]

I have been reading around here for a while due to the fact that I also had major problems with my zwave network after upgrading to 2020.2

So far I found out:
- Enabling the allow new hardware option prevents domoticz from crashing after the upgrade, this is a great find!
- Some nodes where still being interviewed each time I restarted domoticz.
- After repairing all the nodes using the include button and wake them up again (the old node activates again, no new node gets added) I took a snapshot of the ozwcache file.
- Then I shutdown Domoticz and made another snapshot from the cache file
- It turns out that all my troubled nodes keep being deleted from the cachefile when i shutdown domoticz (sudo service domoticz stop)!
- I opened up the cachefile which was taken with all the nodes working and searched two identical nodes. One was being deleted and needed manual wakening each time, the other was working fine.
- It turned out that the working node was added as a non secure device, while the exact same node that was forgotten each time was added as a secure device.
- So I looked in it some more, and all my troubled nodes where added as secure and where reinterviewed each time again. After which they worked perfectly until the next restart of domoticz.

I altered the cachefile which had all the nodes in it and removed all instances of:

Code: Select all

secured="true" 

If i shutdown domoticz I see the cachefile being broken again (size is halving). When i copy the altered cachefile (the one with secured="true" removed) over the broken cachefile and start domoticz everything is running fine straight away.

Hope one of the developers can replicate this issue and sort this out. I think these two problems (new harware must allow and devices must be unsercure connected) are the two main issues with Zwave right now.

[EddyG]

Great find.
Hope the developers will follow up on this.
I looked in my ozwcache*.xml file and have only 1 device (Fibaro FGMS-001) which has secured="true" the rest has nothing.
Still this device is handled normally.

[JuanUil]

What a mess.
After each restart I have to wake up my battery powered devices
My ozwcache has more the 100 listings of each device.
when is domoticz becoming reliable for a long time.
much respect for the developers but the last months I had nothing but troubles.

and to be honest I don'tknow what to do now.
I am not a programmer.
Can I just delete the ozwcachex.xml
Do I need to reinclude all my nodes again

really disapointed

[EddyG]

What Zwave controller do you have. I have a "AEON Labs ZW090 Z-Stick Gen5 EU" which has the configuration onboard.
Normally you can delete the ozwcache*.xml file, but just make a copy and save that.
You should waken / include battery powered devices device by device and make sure that it is properly included, then move to the next device.
Be patient.