Domoticz

Open source Home Automation System
https://forum.domoticz.com/

GoodWe Solar: get data from the new semsportal (lua script)

https://forum.domoticz.com/viewtopic.php?t=27670

Page 1 of 6

GoodWe Solar: get data from the new semsportal (lua script)

Posted: Sunday 21 April 2019 13:46
by CrazyFrog
I have written a lua script to extract the data from the new "Goodwe semsportal" and pass it on to Domoticz.

I hope this helps you.
Questions or tips to improve the script, I'd love to hear them

Code: Select all

--- Created by Raymond Wiertz
--- DateTime: 2019-04-21

-- Note: The GoodWe data is refreshed every 60 seconds.

-- Note: Enter your own data at [LOGINNAME] and [PASSWORD]

-- Tracking your token
-- To find out your token, you must execute the CURL command below from the command line.
--- curl -d 'account=[LOGINNAME]&pwd=[PASSWORD]&code=' https://www.semsportal.com/Home/Login;

-- Place here your token
token = "[YOUR TOKEN]"

-- The curl command below retrieves the get session ID named "ASP.NET_SessionId", the session ID is used to request the final data
command = "curl --silent --output /dev/null --cookie-jar - -d 'account=[LOGINNAME]&pwd=[PASSWORD]&code=' https://www.semsportal.com/Home/Login"

commandArray = {}
i=1

function UpdateDevice(device, data)
    idx = otherdevices_idx[device]
    if (idx == nil) then
        print('** Unknown device'..device)
    else
        commandArray[i] = {['UpdateDevice'] = idx..'|0|'..data}
        i = i+1
    end
end

function stripchars(str, chrs)
    local s = str:gsub("["..chrs:gsub("%W","%%%1").."]", '')
    return s
end

function split(s, delimiter)
    result = {};
    for match in (s..delimiter):gmatch("(.-)"..delimiter) do
        table.insert(result, match);
    end
    return result;
end

function getValue(s)
    firstPos, lastPos = string.find(s, ":")
    value = string.sub(s, lastPos+1, string.len(s))
    return  value
end

function timedifference(s)
    year    = string.sub(s, 1, 4)
    month   = string.sub(s, 6, 7)
    day     = string.sub(s, 9, 10)
    hour    = string.sub(s, 12, 13)
    minutes = string.sub(s, 15, 16)
    seconds = string.sub(s, 18, 19)

    t1 = os.time()
    t2 = os.time{year=year, month=month, day=day, hour=hour, min=minutes,sec=seconds}
    difference = os.difftime (t1, t2)
    return difference
end

-- Sensor declaration
sensorSolarPanels = "solar panels"

difference = timedifference(otherdevices_lastupdate[sensorSolarPanels])

if (difference > 60) then
    local f = assert(io.popen(command, 'r'))
    s = assert(f:read('*a'))
    f:close()

    -- extract the ASP.NET_SessionId from the curl response
    sessionId = string.sub(s, -26)
    sessionId = string.gsub(sessionId,"[\r\n]", "" )
    sessionId = string.gsub(sessionId, "%s+", "")

    -- update curl command with the session id.
    getData = string.format("%s%s%s%s" , "curl -v --cookie 'ASP.NET_SessionId=", sessionId, "' https://www.semsportal.com/PowerStation/PowerStatusSnMin/",token )

    local f = assert(io.popen(getData, 'r'))
    s = assert(f:read('*a'))
    f:close()

    -- extract the JSON-data from the response
    dummy, startPos = string.find(s, "var pw_info = ")
    dummy, endPos = string.find(s, "}};")
    s = string.sub(s, startPos, endPos-1)

    -- remove excess brackets
    valueString = stripchars(s, "[]{}\"")
    -- and split the records
    splitString = split(valueString, ",")

    -- for debugging only
    -- shows all values and the corresponding record id
    --- for key, value in pairs(splitString) do
    ---   print(key..'='..value)
    --- end

    output_power      = getValue(splitString[182])
    last_refresh_time = getValue(splitString[188])
    eDay              = getValue(splitString[199])
    -- daily_generation  = getValue(splitString[357])
    -- work_mode         = getValue(splitString[189])
    -- warning           = getValue(splitString[194])

    print("last_refresh_time :" .. last_refresh_time)
    print("output_power :" .. output_power)
    print("eDay :" .. eDay)
    --- print("daily_generation :" .. daily_generation )
    --- print("work_mode :" .. work_mode)
    --- print("warning :" .. warning)

    current = tonumber(stripchars(output_power, "kwhKWH"))
    dayTotal = tonumber(eDay)*1000

    UpdateDevice(sensorSolarPanels, current..";"..dayTotal)
end

return commandArray

Have fun.

Re: GoodWe Solar: get data from the new semsportal (lua script)

Posted: Monday 22 April 2019 8:26
by biggav
Hi CrazyFrog, whats the format of the token?

I have put in everything that was returned see below and get an error in the log further below
token = "{ "code" : 0, "msg" : "","data":{"redirect":"/PowerStation/PowerStatusSnMin/21ddbd72-acf0-42ba-8451-9c960d5c5b8d"} }"

2019-04-22 14:20:18.157 Error: EventSystem: in Goodwe: [string "print('Solar Luna Fired');
..."]:15: unexpected symbol near '/'

I have also tried
token = "21ddbd72-acf0-42ba-8451-9c960d5c5b8d'

then I get the following error
2019-04-22 14:23:00.125 Error: EventSystem: in Goodwe: [string "print('Solar Luna Fired');
..."]:53: bad argument #1 to 'sub' (string expected, got nil)

Was there any other setup required? I have never had the inverter running on my system as it is only a few weeks old.

Re: GoodWe Solar: get data from the new semsportal (lua script)

Posted: Monday 22 April 2019 10:31
by CrazyFrog
Your token is:21ddbd72-acf0-42ba-8451-9c960d5c5b8d

I have this script running on my Rasberry Pi.
on what for device are you running this scrip.

Re: GoodWe Solar: get data from the new semsportal (lua script)

Posted: Monday 22 April 2019 10:49
by Tiger362
Hi CrazyFrog,

I'm new at this and would like to know how to implement this script on my RPi...

Thanks in advance!

Re: GoodWe Solar: get data from the new semsportal (lua script)

Posted: Monday 22 April 2019 11:22
by sincze
CrazyFrog wrote: ↑Sunday 21 April 2019 13:46 I have written a lua script to extract the data from the new "Goodwe semsportal" and pass it on to Domoticz.

I hope this helps you.
Questions or tips to improve the script, I'd love to hear them

Have fun.

Tnx for extending my LUA knowledge. :) Great help.

Re: GoodWe Solar: get data from the new semsportal (lua script)

Posted: Monday 22 April 2019 11:33
by biggav
CrazyFrog wrote: ↑Monday 22 April 2019 10:31 Your token is:21ddbd72-acf0-42ba-8451-9c960d5c5b8d

I have this script running on my Rasberry Pi.
on what for device are you running this scrip.
I'm on a Pi as well, i have tried running the script under events and also by adding it in the luna folder with the same out come.

does this script create the devices and hardware? or is there something i need to do there?

Re: GoodWe Solar: get data from the new semsportal (lua script)

Posted: Monday 22 April 2019 20:52
by CrazyFrog
you have create your own sensor.
for hardware you have to create a virtual sensor from the type "Dummy (Does nothing, use for virtual switches only)".

then click on "Create Virtual Sensors", name the sensor "solar panels" and choose "sensor type" - "Electric (instant + counter)"

Re: GoodWe Solar: get data from the new semsportal (lua script)

Posted: Tuesday 23 April 2019 9:16
by CrazyFrog
Tiger362 wrote: ↑Monday 22 April 2019 10:49 Hi CrazyFrog,

I'm new at this and would like to know how to implement this script on my RPi...

Thanks in advance!

Hello Biggav,

At first you have to create a dummy sensor, you can do this in the hardware section
Create a sensor from the type "Dummy (Does nothing, use for virtual switches only)".
then click on "Create Virtual Sensors", give it the name "solar panels" and choose "sensor type" - "Electric (instant + counter)"


The next step is place the lua script in the correct folder.
go to the folder domoticz/scripts/lua and copy the script here,
gives the script the name "script_device_solar.lua"

Note that the script must always start with "script_device_" and ends with ".lua"

Re: GoodWe Solar: get data from the new semsportal (lua script)

Posted: Wednesday 24 April 2019 9:13
by Tiger362
Hi CrazyFrog,
Thnx for the info! :)
Do I have to reboot the system now or will it start automatically?

Grtz

Re: GoodWe Solar: get data from the new semsportal (lua script)

Posted: Wednesday 24 April 2019 9:26
by Tiger362
Hi CrazyFrog,
I know now that the script will run automatically ;)
But I got an error:
Error: EventSystem: in /home/pi/domoticz/scripts/lua/script_device_solar.lua: /home/pi/domoticz/scripts/lua/script_device_solar.lua:89: attempt to perform arithmetic on global 'endPos' (a nil value)

Can you tell me what to do?

Re: GoodWe Solar: get data from the new semsportal (lua script)

Posted: Wednesday 24 April 2019 16:56
by dospider
Hello Crazy Frog,
First of all thanks for diving in the problem and a script. I also have a problem with the script.
The logmessage is:

2019-04-24 16:45:52.603 Error: EventSystem: in Goodwe portal: [string "--- Created by Raymond Wiertz
..."]:51: bad argument #1 to 'sub' (string expected, got nil)

Question about token. Does has to be within ' ....'
The username and password not between '...' and the username is it the plantname or emailadres.

It doesn't what i do. Always the same error..

Hope you can find it.

Re: GoodWe Solar: get data from the new semsportal (lua script)

Posted: Thursday 25 April 2019 8:45
by ricvee
Hi CrazyFrog,

Works like a charm!
Thanks a lot! :D

Re: GoodWe Solar: get data from the new semsportal (lua script)

Posted: Thursday 25 April 2019 13:22
by ricvee
You only have to set the time-difference to a higher number than 60 seconds!
After a couple of hours I got locked out of the semsportal.
So I set the timedifference now to 300 seconds...

Re: GoodWe Solar: get data from the new semsportal (lua script)

Posted: Friday 26 April 2019 14:06
by sandeman684
Thanks for this script. It needed some modifications for me to work though. Apparently I got back a very different list of values:
Spoiler: show
2019-04-26 14:03:38.806 Status: LUA: 458=display:Temperature(℃)
2019-04-26 14:03:38.806 Status: LUA: 459=unit:℃
2019-04-26 14:03:38.806 Status: LUA: 460=target_key:Tempperature
2019-04-26 14:03:38.806 Status: LUA: 461=text_cn:温度
2019-04-26 14:03:38.806 Status: LUA: 462=target_sn_six:null
2019-04-26 14:03:38.806 Status: LUA: 463=target_sn_seven:null
2019-04-26 14:03:38.806 Status: LUA: 464=target_type:null
2019-04-26 14:03:38.806 Status: LUA: 465=storage_name:null
2019-04-26 14:03:38.806 Status: LUA: 466=target_index:21
2019-04-26 14:03:38.807 Status: LUA: 467=target_name:Daily Generation
2019-04-26 14:03:38.807 Status: LUA: 468=display:Daily Generation(kWh)
2019-04-26 14:03:38.807 Status: LUA: 469=unit:kWh
2019-04-26 14:03:38.807 Status: LUA: 470=target_key:EDay
2019-04-26 14:03:38.807 Status: LUA: 471=text_cn:日发电量
2019-04-26 14:03:38.807 Status: LUA: 472=target_sn_six:null
2019-04-26 14:03:38.807 Status: LUA: 473=target_sn_seven:null
2019-04-26 14:03:38.807 Status: LUA: 474=target_type:null
2019-04-26 14:03:38.807 Status: LUA: 475=storage_name:null
2019-04-26 14:03:38.807 Status: LUA: 476=target_index:22
2019-04-26 14:03:38.807 Status: LUA: 477=target_name:Total Generation
2019-04-26 14:03:38.807 Status: LUA: 478=display:Total Generation(kWh)
2019-04-26 14:03:38.807 Status: LUA: 479=unit:kWh
2019-04-26 14:03:38.807 Status: LUA: 480=target_key:ETotal
2019-04-26 14:03:38.807 Status: LUA: 481=text_cn:总发电量
2019-04-26 14:03:38.807 Status: LUA: 482=target_sn_six:null
2019-04-26 14:03:38.807 Status: LUA: 483=target_sn_seven:null
2019-04-26 14:03:38.807 Status: LUA: 484=target_type:null
2019-04-26 14:03:38.807 Status: LUA: 485=storage_name:null
2019-04-26 14:03:38.807 Status: LUA: 486=target_index:23
2019-04-26 14:03:38.808 Status: LUA: 487=target_name:HTotal
2019-04-26 14:03:38.808 Status: LUA: 488=display:HTotal(Hrs)
2019-04-26 14:03:38.808 Status: LUA: 489=unit:Hrs
2019-04-26 14:03:38.808 Status: LUA: 490=target_key:HTotal
2019-04-26 14:03:38.808 Status: LUA: 491=text_cn:工作时长
2019-04-26 14:03:38.808 Status: LUA: 492=target_sn_six:null
2019-04-26 14:03:38.808 Status: LUA: 493=target_sn_seven:null
2019-04-26 14:03:38.808 Status: LUA: 494=target_type:null
2019-04-26 14:03:38.808 Status: LUA: 495=storage_name:null
2019-04-26 14:03:38.808 Status: LUA: 496=target_index:36
2019-04-26 14:03:38.808 Status: LUA: 497=target_name:RSSI
2019-04-26 14:03:38.808 Status: LUA: 498=display:RSSI(%)
2019-04-26 14:03:38.808 Status: LUA: 499=unit:%
2019-04-26 14:03:38.808 Status: LUA: 500=target_key:Reserved5
2019-04-26 14:03:38.808 Status: LUA: 501=text_cn:GPRS信号强度
2019-04-26 14:03:38.808 Status: LUA: 502=target_sn_six:null
2019-04-26 14:03:38.808 Status: LUA: 503=target_sn_seven:null
2019-04-26 14:03:38.808 Status: LUA: 504=target_type:null
2019-04-26 14:03:38.809 Status: LUA: 505=storage_name:null
2019-04-26 14:03:38.809 Status: LUA: 506=backup_pload_s:null
2019-04-26 14:03:38.809 Status: LUA: 507=backup_vload_s:0.0
2019-04-26 14:03:38.809 Status: LUA: 508=backup_iload_s:null
2019-04-26 14:03:38.809 Status: LUA: 509=backup_pload_t:null
2019-04-26 14:03:38.809 Status: LUA: 510=backup_vload_t:null
2019-04-26 14:03:38.809 Status: LUA: 511=backup_iload_t:null
2019-04-26 14:03:38.809 Status: LUA: 512=etotal_buy:null
2019-04-26 14:03:38.809 Status: LUA: 513=eday_buy:null
2019-04-26 14:03:38.809 Status: LUA: 514=ebattery_charge:null
2019-04-26 14:03:38.809 Status: LUA: 515=echarge_day:null
2019-04-26 14:03:38.809 Status: LUA: 516=ebattery_discharge:null
2019-04-26 14:03:38.809 Status: LUA: 517=edischarge_day:null
2019-04-26 14:03:38.809 Status: LUA: 518=batt_strings:null
2019-04-26 14:03:38.809 Status: LUA: 519=meter_connect_status:null
2019-04-26 14:03:38.809 Status: LUA: 520=mtactivepower_r:null
2019-04-26 14:03:38.809 Status: LUA: 521=mtactivepower_s:null
2019-04-26 14:03:38.809 Status: LUA: 522=mtactivepower_t:null
2019-04-26 14:03:38.809 Status: LUA: 523=has_tigo:false
2019-04-26 14:03:38.810 Status: LUA: 524=hjgx:co2:8.3190677000000015
2019-04-26 14:03:38.810 Status: LUA: 525=tree:455.921624
2019-04-26 14:03:38.810 Status: LUA: 526=coal:3.3710164000000002
2019-04-26 14:03:38.810 Status: LUA: 527=pre_powerstation_id:null
2019-04-26 14:03:38.810 Status: LUA: 528=nex_powerstation_id:null
2019-04-26 14:03:38.810 Status: LUA: 529=powerflow:pv:1036(W)
2019-04-26 14:03:38.810 Status: LUA: 530=pvStatus:-1
2019-04-26 14:03:38.810 Status: LUA: 531=bettery:0(W)
2019-04-26 14:03:38.810 Status: LUA: 532=betteryStatus:0
2019-04-26 14:03:38.810 Status: LUA: 533=load:1036(W)
2019-04-26 14:03:38.810 Status: LUA: 534=loadStatus:-1
2019-04-26 14:03:38.810 Status: LUA: 535=grid:0(W)
2019-04-26 14:03:38.810 Status: LUA: 536=soc:0
2019-04-26 14:03:38.810 Status: LUA: 537=socText:No battery
2019-04-26 14:03:38.810 Status: LUA: 538=hasEquipment:true
2019-04-26 14:03:38.810 Status: LUA: 539=gridStatus:1
2019-04-26 14:03:38.810 Status: LUA: 540=energeStatisticsCharts:sum:6.5
2019-04-26 14:03:38.810 Status: LUA: 541=buy:0.0
2019-04-26 14:03:38.811 Status: LUA: 542=buyPercent:0.0
2019-04-26 14:03:38.811 Status: LUA: 543=sell:0.0
2019-04-26 14:03:38.811 Status: LUA: 544=sellPercent:0.0
2019-04-26 14:03:38.811 Status: LUA: 545=selfUseOfPv:6.5
2019-04-26 14:03:38.811 Status: LUA: 546=consumptionOfLoad:6.5
2019-04-26 14:03:38.811 Status: LUA: 547=chartsType:4
2019-04-26 14:03:38.811 Status: LUA: 548=hasPv:true
2019-04-26 14:03:38.811 Status: LUA: 549=hasCharge:false
2019-04-26 14:03:38.811 Status: LUA: 550=charge:0.0
2019-04-26 14:03:38.811 Status: LUA: 551=disCharge:0.0
2019-04-26 14:03:38.811 Status: LUA: 552=soc:power:0
2019-04-26 14:03:38.811 Status: LUA: 553=status:0
2019-04-26 14:03:38.811 Status: LUA: last_refresh_time :--%
2019-04-26 14:03:38.811 Status: LUA: output_power :1036(W)
2019-04-26 14:03:38.811 Status: LUA: eDay :6.5
2019-04-26 14:03:38.816 Status: EventSystem: Script event triggered: Solar panels
2019-04-26 14:03:40.984 (RFXCOM) Temp + Humidity (Buiten Kas Daniu 3)
2019-04-26 14:03:49.892 (Thermosmart) Thermostat (Woonkamer Target T)
Therefore I had to change these lines:
output_power = getValue(splitString[533])
eDay = getValue(splitString[545])

And add brackets to:
current = tonumber(stripchars(output_power, "kwhKWH()"))

I cannot find the last refresh time in my list of values.

Re: GoodWe Solar: get data from the new semsportal (lua script)

Posted: Friday 26 April 2019 14:36
by mennohaeck
No matter what I do or how I name the script, there is no output. I checked all the rights and syntax. Funny thing is: no errors as wel. Is there anyone with a bright idea? :?

Re: GoodWe Solar: get data from the new semsportal (lua script)

Posted: Friday 26 April 2019 16:05
by Hielkep
nice work!

i did it a bit different then to copy script file to disk.
In Domoticz website, i opened Setup - More options - Events
added new script with + on top of screen,
selected lua > time.
and pasted script into there.
replaced [LOGINNAME] and [PASSWORD] with relevant data
and replaced [YOUR TOKEN] with my token (i left the quotes in place).
made a dummy switch with a different name, replaced in script on line 65 the dummy switch name.
saved it all, activated the script, and it ran! :D

Re: GoodWe Solar: get data from the new semsportal (lua script)

Posted: Friday 26 April 2019 16:07
by Hielkep
mennohaeck wrote: ↑Friday 26 April 2019 14:36 No matter what I do or how I name the script, there is no output. I checked all the rights and syntax. Funny thing is: no errors as wel. Is there anyone with a bright idea? :?
is the name of dummy switch and the declaration in line 65 of the name of the switch completely the same?

Re: GoodWe Solar: get data from the new semsportal (lua script)

Posted: Saturday 27 April 2019 1:49
by CrazyFrog
Hielkep wrote: ↑Friday 26 April 2019 16:05 nice work!

i did it a bit different then to copy script file to disk.
In Domoticz website, i opened Setup - More options - Events
added new script with + on top of screen,
selected lua > time.
and pasted script into there.
replaced [LOGINNAME] and [PASSWORD] with relevant data
and replaced [YOUR TOKEN] with my token (i left the quotes in place).
made a dummy switch with a different name, replaced in script on line 65 the dummy switch name.
saved it all, activated the script, and it ran! :D

Nice, i didn't know this possibility.
Thank you

Re: GoodWe Solar: get data from the new semsportal (lua script)

Posted: Saturday 27 April 2019 2:01
by CrazyFrog
mennohaeck wrote: ↑Friday 26 April 2019 14:36 No matter what I do or how I name the script, there is no output. I checked all the rights and syntax. Funny thing is: no errors as wel. Is there anyone with a bright idea? :?
Do you get data?
can you save the code below to your Raspberry Pi in a file testscript.lua
then execute this script with the command: lua testscript.lua
What do you see, do you get lot of data?

Before you execute this code, fill in your own Loginname, Password and Token

Code: Select all


-- To get you token use the curl command below, replace [LOGINNAME] and [PASSWORD] for your own id's
-- "curl -d 'account=[LOGINNAME]&pwd=[PASSWORD]&code=' https://www.semsportal.com/Home/Login";

-- copy the token from the response of the curl request above and place it below
token = "[YOUR TOKEN]"

-- replace [LOGINNAME] and [PASSWORD] for your own id's
command = "curl --silent --output /dev/null --cookie-jar - -d 'account=[LOGINNAME]&pwd=[PASSWORD]&code=' https://www.semsportal.com/Home/Login"

local f = assert(io.popen(command, 'r'))
s = assert(f:read('*a'))
f:close()

sessionId = string.sub(s, -26)
sessionId = string.gsub(sessionId,"[\r\n]", "" )
sessionId = string.gsub(sessionId, "%s+", "")

getData = string.format("%s%s%s%s" , "curl -v --cookie 'ASP.NET_SessionId=", sessionId, "' https://www.semsportal.com/PowerStation/PowerStatusSnMin/",token )

local f = assert(io.popen(getData, 'r'))
s = assert(f:read('*a'))
f:close()

print(s)

Re: GoodWe Solar: get data from the new semsportal (lua script)

Posted: Saturday 27 April 2019 14:41
by sandeman684
In my last post I mentioned I needed other key numbers, but that was partially incorrect because the domoticz log only showed me the last roughly 100 keys. I had to output them to a file to see them all.
In this way I found that for my case, each of the keys is one higher than in CrazyFrogs script. So now I have:

Code: Select all

    output_power      = getValue(splitString[183])
    last_refresh_time = getValue(splitString[189])
    eDay              = getValue(splitString[200])
This works well but indicates that the keys are not fully fixed.
Maybe we should not look for key number but for key value?
All times are UTC+02:00
Page 1 of 6

Powered by phpBB® Forum Software © phpBB Limited