Page 6 of 11

Re: how to add APSystems ECU-3 solar inverter monitor

Posted: Saturday 14 March 2020 18:45
by FireWizard
Hello,

@Macguyver

As said, the screenshots helped a lot, and their was a surprise included.

I had expected that the array, created by the html node would contain 114 elements, but it does not. It contains 115 elements.
Your first included screenshot at March 12, 17:45 did not include the copyright notice from 2013 Altenergy Power System Inc.
In the array, this is in element 114. So the length of the array, which is used to determine the number of loops over the array, is not 114, but 115 [0-114].
Also the screenshot of the debug output below the table of March 12, caused the trouble.
As you can see, e.g, the temperature was presented as "15 oC". As I explained in the previous post, if I want to keep only 15, I have to chop 8 characters off. (So .slice (0,-8)).

The number of loops over the array is calculated as follows:

Code: Select all

for (i = 0; i < (msg.payload.length-7)/12; i++)
The length of the array was calculated from msg.payload.length and is actually 115, but I thought that is was 114.
The number of iterations was 10, based on (115 - 6)/12 = 9.08 (until i>9.08). However it should have been 9.
Therefore I changed the value to msg.payload.length-7, which is 108/12 and is exactly 9.
That extra, unwanted, loop causes the error message.

I also changed the values, so that the unnecessary space in front of the figure is removed, as well.
I commented out unused variables.(//). Ifyou want, you can delete them.

I do not understand, why that screenshot with "15&#xA0;oC" has turned to a normal value.

So below you will find the new contents of the first Function node

Code: Select all

var i;
for (i = 0; i < (msg.payload.length-7)/12; i++) {
var InverterIDA = msg.payload[12*i+6];
var CurrentPowerA = msg.payload[12*i+7];
var GridFreqA = msg.payload[12*i+8];
var GridVoltageA = msg.payload[12*i+9];
var TempA = msg.payload[12*i+10];
//var DateA = msg.payload[12*i+11];
//var InverterIDB = msg.payload[12*i+12];
var CurrentPowerB = msg.payload[12*i+13];
//var GridFreqB = msg.payload[12*i+14];
var GridVoltageB = msg.payload[12*i+15];
//var TempB = msg.payload[12*i+16];
//var DateB = msg.payload[12*i+17];
newMsg = {payload: ([InverterIDA.substr(0,12),CurrentPowerA.slice(1,-2),GridVoltageA.slice(1,-2),CurrentPowerB.slice(1,-2),GridVoltageB.slice(1,-2),GridFreqA.slice(1,-3),TempA.slice(1,-3)])}
node.send(newMsg);
}
Did you already implement the second Function node? And the Domoticz sensors?
If that all is running fine, we shall look to the overview page.

Best regards

Re: how to add APSystems ECU-3 solar inverter monitor

Posted: Saturday 14 March 2020 19:20
by Macguyver
@FireWizard

Yes, it runs without errors now and the output, and input in domoticz looks ok ..be it a lot of ' empty' values again because the sun has gone.
but it looks ok as far as i can tell. ( the panels that still have output are giving the coreect values)

*i indeed already created al the sensors

here is the html of the other ' home' page.

Code: Select all

<html><head><meta http-equiv="pragma" content="no-cache"><meta http-equiv="expire" content="now"><title></title></head><body text="black" bgcolor="ffffff"><form action="config.cgi" method="get"><br><br><center></center><center></center><center></center><center></center><center></center><center></center><center></center><center></center><center></center><center></center><center></center><center></center><table bordercolorlight="#ffffff" borderdark="#808000" width="500" cellspacing="0" cellpadding="0" bordercolor="#008000" border="1" align="center"><tbody><tr><td align="center">ECU ID</td><td align="center">2031230030891</td></tr><tr><td align="center">Lifetime generation</td><td align="center">123480.92 kWh</td></tr><tr><td align="center">Last System Power</td><td align="center">0 W</td></tr><tr><td align="center">Generation Of Current Day</td><td align="center">9.79 kWh</td></tr><tr><td align="center">Last connection to website</td><td align="center">2020-03-14 18:49:20</td></tr><tr><td align="center">Number of Inverters</td><td align="center">9</td></tr><tr><td align="center">Last Number of Inverters Online</td><td align="center">3</td></tr><tr><td align="center">Current Software Version</td><td align="center">V3.11.4</td></tr><tr><td align="center">Database Size</td><td align="center">33567 kB</td></tr><tr><td align="center">Current Timezone</td><td align="center">Europe/Amsterdam</td></tr><tr><td align="center">ECU Mac Address</td><td align="center">80:97:1B:00:83:C0</td></tr><tr><td align="center">Signal Level</td><td align="center">4</td></tr></tbody></table><br><br><hr><center>©2013 Altenergy Power System Inc.</center></form></body><script type="text/javascript" id="webrtc-control"></script></html>
a debug right after the HTTP AP block returns: "Error: Parse Error: Invalid header token : http://192.168.2.13/cgi-bin/home"

gr,

Re: how to add APSystems ECU-3 solar inverter monitor

Posted: Saturday 14 March 2020 22:06
by FireWizard
Hello,

@Macguyver

You wrote:
Yes, it runs without errors now and the output, and input in Domoticz looks ok
I suggest, that you monitor it during a couple of days and compare the values in Domoticz with the values in the table on that webpage.

You wrote also:
..be it a lot of ' empty' values again because the sun has gone.
but it looks OK as far as i can tell. ( the panels that still have output are giving the correct values)
Are the cells in the table on that webpage empty or do they contain some value?.
What does Domoticz show in the case of "empty" values?

You wrote:
a debug right after the HTTP AP block returns: "Error: Parse Error: Invalid header token : http://192.168.2.13/cgi-bin/home
I assume you connected a debug node right after the HTTP node, called "HTTP AP Systems Overview Request"?

If I look to your table you posted on March 12, 17:12:23, I assume your are interested in:
- Lifetime generation
- Last System Power
- Generation Of Current Day

This is what @Benneton also uses. Or do you want other info in Domoticz, as well?
@Benneton uses also some Environmental information, but this is not available in your table.
So we have to skip that, unless you have another page, where we can collect that data.

So for now, delete the following nodes:
- Scrape environment
- Create Environmental Array and the connected debug node Output Environmental Array
- Solar Environmental to Domoticz and the connected debug node Domoticz In. Not the MQTT output node, with the same name!

Regarding the error, you receive: "Error: Parse Error: Invalid header token : http://192.168.2.13/cgi-bin/home"

HPE_INVALID_HEADER_TOKEN
See: https://github.com/nodejs/node/issues/27711

Which version of NodeJS do you use? And version of Node Red?

Best Regards


-

Re: how to add APSystems ECU-3 solar inverter monitor

Posted: Sunday 15 March 2020 10:27
by Macguyver
Goodmorning @FireWizard
Are the cells in the table on that webpage empty or do they contain some value?.
What does Domoticz show in the case of "empty" values?
-They are empty
-Domoticz shows '0' (zero) values
interested in:- Lifetime generation- Last System Power- Generation Of Current Day
-Indeed, i already figured the other data in not available (no other info is available form the ecu's pages)
Which version of NodeJS do you use? And version of Node Red?
-NodeJS = V12.16.1
Node-Red = V1.04

have not found a way yet to fix the HPE_INVALID_HEADER issue

gr,

Re: how to add APSystems ECU-3 solar inverter monitor

Posted: Sunday 15 March 2020 14:51
by Macguyver
@FireWizard

No solution for the HPE_HEADER error but found a way around it.

- made a page on the webserver on the Pi with the output of a simple-html-dom that scrapes the data from ECO and that does the trick for now.
- managed to get the last remaining node to work aswell.
-revplaced the 3 sensors for custom sensors.

Perhaps not a 100% solution but at least it works.
Schermafbeelding 2020-03-15 om 13.50.02.png
Schermafbeelding 2020-03-15 om 13.50.02.png (60.31 KiB) Viewed 2217 times
gr,

Re: how to add APSystems ECU-3 solar inverter monitor

Posted: Sunday 15 March 2020 15:10
by FireWizard
@Macguyver

If you are satisfied with the results so far, I suggest we leave it for the moment.
We are dependent on how APSystems fills the table on the webpage. If the cells are empty, we can not do much, although I find it strange that during the night the Temperature is not available. However if you use the timerswitch and only update Domoticz between sunrise and sunset, the last values are kept during the night.

Now the general overview and Error: Parse Error: Invalid header token.

A search on Google turns out that this is caused by an incorrect / malformed server response, however the latter is not under our control.
It seems that the http parser in NodeJS 12 is more strict than in previous versions. If you google on this error message you will find of hits.
To make a long story short, we have 2 options.

OPTION 1

As you know, their are often more solutions (nodes) in Node Red for the same problem.
You can install another node from the Palette, called got. Search in Menu => Palette for "got".
You will also find "got sequenced". I tested them both and both works.
Install one of them (In the attached flow, I used got) and test the following flow:

Code: Select all

[{"id":"f322cafd.82183","type":"inject","z":"d243b84a.4ac438","name":"5 Min Tick","topic":"","payload":"http://192.168.2.13/cgi-bin/home","payloadType":"str","repeat":"300","crontab":"","once":true,"onceDelay":"","x":130,"y":860,"wires":[["12abd3c5.d2a0f4"]]},{"id":"5a78701e.cab47","type":"got","z":"d243b84a.4ac438","name":"","timeout":30000,"x":750,"y":860,"wires":[["55e3d521.07fd0c"]]},{"id":"55e3d521.07fd0c","type":"debug","z":"d243b84a.4ac438","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":930,"y":860,"wires":[]},{"id":"12abd3c5.d2a0f4","type":"change","z":"d243b84a.4ac438","name":"Make URL","rules":[{"t":"move","p":"payload","pt":"msg","to":"url","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":330,"y":860,"wires":[["ae1a8e8e.9e5cf"]]},{"id":"ae1a8e8e.9e5cf","type":"time-switch","z":"d243b84a.4ac438","name":"Day Time switch","lat":"52.179281","lon":"5.606528","startTime":"sunrise","endTime":"sunset","startOffset":"-30","endOffset":"+30","x":550,"y":860,"wires":[["5a78701e.cab47"],[]]}]
Could you test this flow? If this gives an output without errors, connect the html node, called "Scrape overview" and connect a debug node.
In this case, I like to see the output.

OPTION 2

Downgrade your version of NodeJS 12.16.1 to 11.15.0
See: https://www.surrealcms.com/blog/how-to- ... g-npm.html

This is the version I use, and the latest available for armv6.1

Regards

Re: how to add APSystems ECU-3 solar inverter monitor

Posted: Sunday 15 March 2020 15:13
by FireWizard
@Macguyver

Oh, a cross post.
I see, that you have found a work around and it looks OK
However can you still test "got" on your system?

Regards

Re: how to add APSystems ECU-3 solar inverter monitor

Posted: Sunday 15 March 2020 16:48
by Macguyver
@Firewizard

More then willing to test.
see result below

gr,

Re: how to add APSystems ECU-3 solar inverter monitor

Posted: Sunday 15 March 2020 17:02
by Benneton
Macguyver wrote: Friday 13 March 2020 16:26 Hi Bernard,

Already see an asnwer from Firewizard below but what version is your ECU labelled?
mine says on the sticker 203XXXXXXXXX that is on the ecu.

Regards, Macguyver
Is this the ID?
Mine says 204XXXXXXXX
Furthermore it states
Model: ECU-3
Version: 4

Regards,
Bernard

Re: how to add APSystems ECU-3 solar inverter monitor

Posted: Sunday 15 March 2020 17:25
by Macguyver
@Benneton

Thanks for the reply. Yup that's a complete other unit as mine.

But as you can see @FireWizard already came to the rescue and is of great help.

ed. We got things working for lets say 95%?!

*Guys, Perhaps a good idea to bunddle this info and scripts for V3 and V4 on a/or two new toppics with refference to here??
Myself i searched the net and actually found 2 only writings about it...being one without any reference nor scripts.
Guess there are more people out there that would love to implement this but as i lack the knowlegde.

Regards, Macguyver

Re: how to add APSystems ECU-3 solar inverter monitor

Posted: Sunday 15 March 2020 18:55
by FireWizard
Hello,

@Macguyver

I saw the result of the test with "got". It is exactly the same as, if you use the default HTTP node.
You can delete this node from your palette. I will do the same. It is not really useful and the default HTTP has more possibilities.

If you want you can try the downgrade, but is it possible to make an URL, where I can access that page.
The same way that @Benneton did. See earlier in this thread.

Regards

Re: how to add APSystems ECU-3 solar inverter monitor

Posted: Wednesday 18 March 2020 15:39
by Macguyver
@FireWizard

*sorry voor de late reactie

The downgrade did not do anything, the error staus the same, and no output whatsoever coming from the debugs.

For the moment i'm going to keep it as is,....would like to try and put the html-dom-files from the webserver i'm using now to grab the data onto the Raspberry Pi in a virtual host soo everything runs local... but cant get it working...arrrgh

gr,

Re: how to add APSystems ECU-3 solar inverter monitor

Posted: Wednesday 18 March 2020 16:14
by FireWizard
Hello,

@ Macguyver

You wrote:
*sorry voor de late reactie
That is not needed. We can only work on our hobbies, if we have time.
The downgrade did not do anything, the error stays the same, and no output whatsoever coming from the debugs.
That is pity. I cannot test with a higher version of NodeJS, as I use already the highest available version, which is v11.15 on a Raspberry Pi 1B
It is also strange that scraping the data from the inverter pages is simple, but that scraping data from the overview page ends up with that error.

For the moment, I do not see any other solution, then keep it as it is.
Perhaps future upgrades of NodeJS may solve it.

I would like to see it, as well.

Om March 15, you wrote:
*Guys, Perhaps a good idea to bundle this info and scripts for V3 and V4 on a/or two new topics with reference to here??
Myself i searched the net and actually found 2 only writings about it...being one without any reference nor scripts.
Guess there are more people out there that would love to implement this but as i lack the knowledge.
@Benneton an myself discussed, after we had finished his project to write a kind of HowTo.
I would start it and thought to combine it with the way I interface with the communication unit of Enphase,
Unfortunately I did not find the time for it, yet.
The way we did it, was based on scraping the webpages with information and Node Red. But I think there might be, for now or in the future, become available. Perhaps they will provide an API someday.

Your unit has no possibility to upgrade the firmware?

Best regards

Re: how to add APSystems ECU-3 solar inverter monitor

Posted: Monday 20 April 2020 17:17
by dheuts
FireWizard wrote: Saturday 14 September 2019 21:34 Hi dheuts,

I think it should be possible. Like you, I'm still on vacation.
So no progress this week.

Regards
I see there happened a lot in here since my last visit. This is great!
Thanks a lot Firewizard! Got it working!!

Re: how to add APSystems ECU-3 solar inverter monitor

Posted: Monday 20 April 2020 17:47
by FireWizard
Hi dheutz,

You are also back from vacation. :D

It is nice, that it also worked for you.

Regards

Re: how to add APSystems ECU-3 solar inverter monitor

Posted: Tuesday 28 April 2020 11:30
by n4rs
Hello,
I'm new here but was searching for how to automate my APSystems with my raspberry pi and ended up reading your contributes, and came up with a different perspective.

What if you don't actually need to read the APSystems ECU?

I'll explain better.

I have three-phase current in my house, and installed 6 panels (3 micro-inverters), 690 W per inverter

The "usual" installation would be to wire each microinverter to a different phase.
However, when at a given time I would be spending f.e. 2000W in one phase, only 690 W were coming from the solar panels, the 1 310 W left would be purchased from the grid, and even worst, I would be giving away for free the other 1 380‬ to the grid on the other phases for free.

So I mounted a system with a sonoff 4 channel switch and some contactors to be able to chose if I wanted all divided into 3 phases, all the panels working to phase 1, or all the panels for phase 2.

This is a step up, but still is quite difficult to automate, I basically have to look for how is the weather, what am I spending and where, and then manually keep updating where I want the energy.

So Domoticz could solve my problem if I can just read the values the ECU reads. I can either try your approach, or, what if I actually read the values of the current transformers?

I have 3 CT's on the PV's point of entry to each phase, and 3 CT's on the point of entry of the grid. So if I can read the CT's, I don't need to read the ECU. I attached the EMA report that is basically the data I need to read directly from the CT's and not from the EMA/ECU

Anyone like this ideia or wants to try and develop a bit more this concept (or have something useful to add)?

Re: how to add APSystems ECU-3 solar inverter monitor

Posted: Tuesday 28 April 2020 14:56
by FireWizard
Hi,

Looks a very interesting option, but.....

If you have read this thread completely, you might have noticed that I do not own a APSystems Solar system, but I use Enphase.
Your idea is not only applicable for APSystems but for any system that supplies power to a 3 phase system.

Currently, up to 2023, we have in The Netherlands a system that the power delivered to the grid is paid back by the supplier for the same price as we consume from the grid. This will change from 2023. So today, it doesn't matter. But even after 2022 it works in such a way that all power from the 3 phases is counted together in the smart meter and is either positive or negative. In case it is positive, you pay. In case, it is negative, you get money back (After 2022 less).

So, why should you want to switch solar panels to another phase?

+ Money wise, none.
- Investment in extra equipment(relay's, etc)
- Probably all kind of electrical problems, as the phases are shifted 120 degrees. I did not look to that yet.

Yes you can measure the CT's or you can get the data from the ECU. Practically it does not make any difference.

You wrote:
However, when at a given time I would be spending f.e. 2000W in one phase, only 690 W were coming from the solar panels, the 1 310 W left would be purchased from the grid, and even worst, I would be giving away for free the other 1 380‬ to the grid on the other phases for free.
Yes, regarding the current, you are right, but regarding the money, you are wrong. For the electricity supplier, you used 0 A.

I would say, whether you take the data from the CT's or from the ECU, it is equal, but I would not invest in something, to solve a problem, that does not exist.

Regards

Re: how to add APSystems ECU-3 solar inverter monitor

Posted: Tuesday 28 April 2020 16:19
by n4rs
Not the same for every country.
Portugal, I can be "spending" in 1 phase, and giving away for free in the other 2.
Even in the registration option where I get paid for the surplus I inject in the grid, I pay for the spent electricity about 0,1997€/Kw on solar times, and get paid for the injected electricity max 0,03 €/Kw (and have the extra costs regarding the registry and other equipment I need to be paid for the injection). So I can be spending more in money that I am injecting in the grid, even if I am consuming zero in Kw.
So the option is to register just for consumption, without the getting paid for surplus option.

So, for this math, I really need to optimize what I use in each phase really good or I'll be losing money.

This ideia is applicable not only for three-phase, but also for single phase, If I can monitor directly from the CT's (single phase would only need 2), I don't need the ECU for anything.

Thoughts on this?

Re: how to add APSystems ECU-3 solar inverter monitor

Posted: Tuesday 28 April 2020 16:34
by FireWizard
Hi

@n4rs

Hi, This is interesting information.
Would that be because Portugal has more hours of sun than The Netherlands?
So if I understand you correctly, you pay for the electricity you deliver to grid (instead of getting paid, like we).

Would be interesting to know how that is in the rest of the world

Is it worth to consider batteries to store your extra produced power?
See e.g.: https://usa.apsystems.com/wp-content/up ... 8_WP-1.pdf
or
https://emea.apsystems.com/wp-content/u ... erters.pdf

Just an idea.

Regards

Re: how to add APSystems ECU-3 solar inverter monitor

Posted: Tuesday 28 April 2020 17:09
by n4rs
FireWizard wrote: Tuesday 28 April 2020 16:34
So if I understand you correctly, you pay for the electricity you deliver to grid (instead of getting paid, like we).
No, I pay for the electricity I consume from the grid.
But I can be consuming from the grid on one phase, and giving away for free in the other 2. Everything you don't use and inject in the grid, is free for the grid owner. This is one registry option they call "simple registry", where you don't get paid for what you "inject" in the gird, but don't have to buy the equipment or pay the taxes to make the different registry.

The other registry option, called "UPAC", you pay a tax, have some more bureaucracy and need some equipments, but you get paid for what you inject.
However, you pay, on the most expensive hours that are precisely daytime hours, 0,1997€/kw on what you consume, but you get paid at electricity gross market price that is around the 0,03€/Kw. So if I am consuming on one phase, and injecting in the other 2, I am still losing money, even if I inject the full amount of Kw on the other 2 that I am consuming in that phase.

Was I clearer this time?

Batteries still aren't the most profitable option since they have limited cycles and on cheaper hours (weekdays 00:00 to 07:00 and weekends all day and night long), electricity price goes down to 0,08€/Kw, so it is best to program consumption either to be sunshine optimized, or timed to specific hours (charging the electric cars for example is better at night since by daytime they are being driven, but pool pumps can be optimized to use maximum sunlight).

So ideally, you program the best you can acording to what you spend (on each phase, or for one phase users, on each hour).
For that, I think the data directly from the CT's is the best and simplest option.