Migrations from Enphase plugin to default Domoticz Enphase hardware

[Xavier82]

hi all,

Question:
I would like to migrate from the Enphase plugin which I use from https://github.com/H4nsie/EnphaseEnvoy to the Domoticz Default Enphase hardware.

So the created devices for each solarpanel is not an issue, they are of the same type.
But the counter of total is of a different type between both hardware.

Is there any change how to migrate without loosing any data.
I'm aware of the "replace" button in each device (under Edit device) but since these are 2 different types I can't use this.

This is with the Enphase Plugin from H4nsie Enphase Envoy Plugin:


This is with the Default Domoticz Enphase hardware:

Enphase Default hardware Domoticz.png (11.11 KiB) Viewed 1734 times

[Xavier82]

So I understand there is no easy fix because the external plugin uses subtype "P1" and the internal hardware uses "General kWh", so the "Replace" function can't be used.
1 possible way to do this is to change the data in de database with "SQLite Expert Personal Edition".

I have downloaded the SQLite software and downloaded the database from Domoticz (Under settings\backup) and imported the database file into SQLite.
In the databse there is a table named "DeviceStatus".
Does anyone knows if this is the correct table to change the data?

I think I need to adjust the Type and SubType for the Envoy device which has been created as P1 and replace it by "General kWh".
I don't want to lose the device data, thats the reason why I want to change the SubType of the external plugin to "General kWh" so that I do can replace the device with the "Replace" function to the internal default Envoy hardware.

Could anyone point out to me if this is the correct thing to do?

[waltervl]

In the database there is a Meter_Calendar and the Multimeter_Calendar table. In these tables the long term history is stored based on IDX (DeviceRowID). I am not sure if all P1 data is in the multimeter table and if the new "General kWh" device is stored in the meter table....

But these tables you have to modify SQLite Expert Personal Edition or https://sqlitebrowser.org/ (my preference).

I hope the hints can help you merge the data.

[Xavier82]

Thanks Walter for your reaction.

I hoped it would be as simple as changing type and subtype in the "DeviceStatus" table?
I can see that P1 has as type the value "250" with SubType value "1" and SwitchType is "0" and that General, kWh has as type the value "243" with SubType as value "29" and SwitchType is "4".

In the table "Meter_Calendar" I find (based on IDX 926) the meter data of the native "Enphase kWh Production" device.
In the table "MultiMeter_Calendar" I find (based on IDX 433) the meter data of the Python plugin "Enphase kWh Total device" as P1 device.

As far as I can figure it out this is the correct way. P1 data is registered in "MultiMeter_Calendar".
In MultiMeter_Calendar table there are 6 values (2 values are used--> Value1 and Value3) and 4 counters (1 counter is used --> Counter1) which are stored on day-level
n MultiMeter table there are 6 values (2 values are used--> Value1 and Value3) which are saved for 24hours with an interval of 5 minutes.

In Meter_Calendar table there is only 1 value and 1 counter. Which are stored on day-level.
In Meter Table there is value and usage (Power Generated) and these are saved for 24hours with an interval of 5 minuten.

My SolarPanels are installed since february 2023, this is also the beginning of using the Python Plugin for Enphase (based on P1 device).
So I would like to migrate the data from the table "MultiMeter_Calendar" to "Meter_Calendar" and then from hardware from old Python Plugin (With IDX433 to Native Domoticz hardware with IDX926.
I think I also would need to change the device type in the DeviceStatus table so I can use the Replace button?

Do I need to merge the data in all 4 tables? (from Multimeter to Meter and from MultiMeter_Calendar tot Meter_Calendar)
And how does this reflect on my P1 data from my power/energymeter for Usage and Return?

[waltervl]

What I would do is keep using the General kWh meter and get rid of the P1 meter.

You could already disable the python plugin (not delete!) in menu setup - hardware, so the P1 meter will not be updated anymore in the database.

Then recalculate the history P1 values of idx 433 in multimeter_calender and import those in the meter_calender table as history values of the new General Kwh device (idx926). Also check for overlapping dates as you are running them now both.

Dont change anything in the devicestatus table. After you have moved the history data from P1 to General kWh delete the python plugin in menu setup - hardware and all releated devices will be removed from the database.

PS. Does the python plugin not have a General kWh meter already? Because then you could replace the 2 Gneral kWh meters and forget about the P1 meter.

[Xavier82]

What I would do is keep using the General kWh meter and get rid of the P1 meter.

Yup thats what I want to do

You could already disable the python plugin (not delete!) in menu setup - hardware, so the P1 meter will not be updated anymore in the database.

Yup, Also done in test environment.

Then recalculate the history P1 values of idx 433 in multimeter_calender and import those in the meter_calender table as history values of the new General Kwh device (idx926). Also check for overlapping dates as you are running them now both.

What do you mean by "recalculate the history of P1? I know "imported" the history by copy from multimeter_calender and paste in meter_calender.
Is this the right way to do? I use SQLite Expert Personal but can't find something like "import".
Since the external Python Plugin is still active on my production envirnment (Rpi3B) I want to migrate to new production environment (Rpi4b 8gb).
Simultanious I want te migrate from external Python Plugin Enphase to Native Enphase hardware.
So there is no "same" data since I can remove/delete data from the Native Enphase then first import from old external Python plugin and the enable the new native Enphase hardware..

Dont change anything in the devicestatus table. After you have moved the history data from P1 to General kWh delete the python plugin in menu setup - hardware and all releated devices will be removed from the database.

Ok clear on that, I will leave the devicestatus table untouched.

PS. Does the python plugin not have a General kWh meter already? Because then you could replace the 2 Gneral kWh meters and forget about the P1 meter.

No, the external Python Plugin only has 1 P1 meter.....

[waltervl]

In the meter calenders table there is one column for the total counter value and on for the delta between the to rows.

The multimeter_calendar table I don't know but you have to rearrange or recalculate calculate the values to the ones used in the meter_ calendar table. There will be a row with values per day but perhaps for this conversion you could use one row per month.

There probably is a sqlite expert personal manual to be found how to add rows with values to a table. Perhaps with an excel import. Also sql will do it but then you have to know the statements. But if you only do month it will be only a few lines so can be done manually too

[Xavier82]

Thanks for the information.
Last weeks during holiday I played a bit with the different values in the different tables.
I think I now get it how it works and what I need to do to migratie the data between the 2 hardware applications.