Page 1 of 1

Duplicate devices when device idx = group idx on devices tab

Posted: Thursday 12 March 2020 9:50
by flemm
Hello,

I couldn't find any related topics, so I assumed it was best to create a new one.
Last night I noticed that my p1 meter has duplicate devices in the database. I'll attach an image that shows what I mean.

When I set the Power device (IDX 1) to unused, both instances change to unused. When I add an unused device, they both get added. So, it's the same device somehow. There's only a single instance of the p1 hardware in my hardware table.

I assume this can be fixed by deleting the p1 hardware and adding it again, but I guess that means that i'll lose all the history :cry:.
Is there any way to fix this without losing the collected data up to now?

Thanks in advance!

Re: P1 Meter - duplicate devices

Posted: Thursday 12 March 2020 10:35
by waaren
flemm wrote: Thursday 12 March 2020 9:50 Last night I noticed that my p1 meter has duplicate devices in the database. I'll attach an image that shows what I mean.
It is unlikely that your screen reflects what is in the database because the Idx is the primary key of the devicestatus table and must be unique (or null) by sqlite design.
If you already cleared cache and appcache (different things) it's time to look in your database (table devicestatus) with sqlite3 or similar tools.

Re: P1 Meter - duplicate devices

Posted: Thursday 12 March 2020 10:38
by manjh
It might be worth to do a backup and then restore. Just guessing, but it might reorganize the DB if anything is wrong in there. And it's not a big deal to try.

Re: P1 Meter - duplicate devices

Posted: Thursday 12 March 2020 10:56
by flemm
waaren wrote: Thursday 12 March 2020 10:35 It is unlikely that your screen reflects what is in the database because the Idx is the primary key of the devicestatus table and must be unique (or null) by sqlite design.
If you already cleared cache and appcache (different things) it's time to look in your database (table devicestatus) with sqlite3 or similar tools.
Thanks for your explanation. I already tried clearing my browser cache, that didn't make any difference. Feeling kinda dumb, but could you tell me what the app cache is, and how I should clear it?
manjh wrote: Thursday 12 March 2020 10:38 It might be worth to do a backup and then restore. Just guessing, but it might reorganize the DB if anything is wrong in there. And it's not a big deal to try.
Thanks for your suggestion. I've just made and restored a backup of my database. When the dashboard re-appeared it showed no devices at all, so that was pretty nerve-racking :D Restarted domoticz, and thank god my devices were back. No difference in the double entries unfortunately..

Duplicate devices when device idx = group idx on devices tab

Posted: Thursday 12 March 2020 11:03
by waaren
flemm wrote: Thursday 12 March 2020 10:56 Thanks for your explanation. I already tried clearing my browser cache, that didn't make any difference. Feeling kinda dumb, but could you tell me what the app cache is, and how I should clear it?
I posted below already a couple of times on the forum....
A lot of the problems users experience after a domoticz update are gone when the browsercache and appcache (different things !) are cleared. There are also quite a number of posts on this forum related to these kind of problems.

To summarize and sorted from little effort to a bit more effort take these steps and check after each step if it address the issues you encounter.

- Try another browser. Edge, Internet Explorer, Firefox, Chrome, Brave, Opera.. They all are up to the task

- clear browser cache and appcache
Chrome: chrome://appcache-internals/#
Firefox: https://support.mozilla.org/en-US/kb/storage

- in www/js look for domoticz.js.gz, if its there remove it, (KEEP domoticz.js !! )
- use incognito mode using
Chrome [control] [shift] n
Firefox: [control] [shift] p

- restart domoticz
- rename the location of the original installation and install the new version to an empty target directory. Next copy database and scripts from the old location and fire it up.


Hope this will also solve your problem.

Duplicate devices when device idx = group idx on devices tab

Posted: Thursday 12 March 2020 11:10
by flemm
waaren wrote: Thursday 12 March 2020 11:03 I posted below already a couple of times on the forum....
Sorry, In my excitement and eagerness to learn more I asked, while I should have searched for it. Thanks a lot, i'll give it a try!

Duplicate devices when device idx = group idx on devices tab

Posted: Thursday 12 March 2020 11:38
by flemm
I've found the issue, sort of. It didn't have anything to do with caching or something like that.

The devices table somehow mixes up devices and groups. I had a few groups of switches. When the table with devices loaded, the devices and groups were shown correctly. But after a few seconds, the groups change into the P1 devices. After I deleted the groups, the duplicates were gone.

So, the group IDX's conflict with the devices IDX's. Are these supposed to be 'different' indexes and therefore to be able to exist alongside each other? Or is the error in the fact that a new group gets an IDX that already exists? When I create a new group, it automatically gets IDX 1, which is already in use, and I don't see the option to change that.

After IDX 1-5 I had a gap until 22. So I fixed it by creating groups until they reached IDX 6. Not the answer to the issue, but it works.

Duplicate devices when device idx = group idx on devices tab

Posted: Thursday 12 March 2020 14:31
by waaren
flemm wrote: Thursday 12 March 2020 11:38
The devices table somehow mixes up devices and groups. I had a few groups of switches. When the table with devices loaded, the devices and groups were shown correctly. But after a few seconds, the groups change into the P1 devices. After I deleted the groups, the duplicates were gone.

So, the group IDX's conflict with the devices IDX's. Are these supposed to be 'different' indexes and therefore to be able to exist alongside each other? Or is the error in the fact that a new group gets an IDX that already exists? When I create a new group, it automatically gets IDX 1, which is already in use, and I don't see the option to change that.

After IDX 1-5 I had a gap until 22. So I fixed it by creating groups until they reached IDX 6. Not the answer to the issue, but it works.
I am on the almost same version as you (V4.11781) but don't see this behavior. When I create a new group it gets the next available group-idx.
How do you create a new group ?

Groups are shown in the devices tab and can have the same IDX number but with they are shown with a different icon, 'Domoticz' as hardware and without id and without unit.
dups.png
dups.png (34.47 KiB) Viewed 2314 times
There cannot be a conflict between group IDX and device IDX as they are not mixed anywhere in the code and are in different tables in the database.
Same is true for Hardware IDX, Variable IDX , Camera IDX, etc..

If there is a conflict with these IDX's in your system there must be something seriously wrong in the database.

Duplicate devices when device idx = group idx on devices tab

Posted: Thursday 12 March 2020 17:45
by flemm
Thanks a lot for your explanation. I suspected groups have 'different' IDX's and can therefore have the same number as another device. I create a new group by going to 'Scenes' and clicking 'Add scene' and selecting 'Group'. I'm unaware of any other way to do that..

I've tested a little further; deleted all the groups and started fresh. A new group gets IDX 1. The problem seems to occur only with my P1 electricity meter, which has IDX 1, 2, 3 and 4. My gas meter is next (IDX 5), and doesn't show the issue. Also, the higher used device IDX's (22 and higher) work fine.

Since screenshots might not be clear enough, here's a screen recording of what happens: https://www.dropbox.com/s/iil32kvvtzxyj ... s.mov?dl=0. Hope that makes it more clear. As you can see, after a few seconds, the groups with 'conflicting' numbers, are replaced by duplicates of the P1 electricity devices. I hope you understand my confusion earlier today.. :?

Since the other IDX's than the ones from my P1 electricity work fine, I hope my database is ok.. :shock:

Thanks again for your thoughts!

Duplicate devices when device idx = group idx on devices tab

Posted: Friday 13 March 2020 1:01
by waaren
flemm wrote: Thursday 12 March 2020 17:45 Since screenshots might not be clear enough, here's a screen recording of what happens: https://www.dropbox.com/s/iil32kvvtzxyj ... s.mov?dl=0. Hope that makes it more clear. As you can see, after a few seconds, the groups with 'conflicting' numbers, are replaced by duplicates of the P1 electricity devices. I hope you understand my confusion earlier today.. :?
Your recording does indeed give me another view on the issue. The 'magical' change of the groups to the device happens when the device is updated. You see that in the P1 devices because they are updated very frequent but I would not be surprised if it also happens on any other group/device combination at the moment the device is updated.
I posted this as issue 3978 on github

Duplicate devices when device idx = group idx on devices tab

Posted: Friday 13 March 2020 8:54
by flemm
Thanks for your reply. I didn't realise it happened when the devices were updated. And indeed, my P1 meter is updated every 5 seconds. I've checked, and you're right! With a group with IDX 22, same number as one of my Hue bridge devices, changes to 'Hue Bridge' when the Hue is updated. So it isn't related to my P1 after all.

Re: Duplicate devices when device idx = group idx on devices tab

Posted: Friday 13 March 2020 22:37
by waaren
flemm wrote: Friday 13 March 2020 8:54 I didn't realise it happened when the devices were updated. And indeed, my P1 meter is updated every 5 seconds. I've checked, and you're right! With a group with IDX 22, same number as one of my Hue bridge devices, changes to 'Hue Bridge' when the Hue is updated. So it isn't related to my P1 after all.
Issue has been picked up and a PR is created. Fix will be in one of the next Beta's

Re: Duplicate devices when device idx = group idx on devices tab

Posted: Saturday 14 March 2020 9:56
by waaren
waaren wrote: Friday 13 March 2020 22:37 Issue has been picked up and a PR is created. Fix will be in one of the next Beta's
Fixed in V4.11784

note: after installing this version an extra clear of the browser cache / history is needed.

Re: Duplicate devices when device idx = group idx on devices tab

Posted: Sunday 15 March 2020 11:32
by flemm
That's awesome. I've installed the update, cleared everything and it works like a charm. Thanks so much!