Ginlong Solis inverter via mqtt to nodered to Domoticz

[sieth]

Ginlong Solis inverter via mqtt nodered Domoticz

This guide needs to be refined
But if you have skills enough this could already work for you

I've managed to readout the inverter and send the data to Domoticz.
Required:
* The inverter installed and connected to the internet
* Node-red
* The altered pythonscript
* Domoticz

Step 1:
check if your ginlong-solis inverter is connected to the internet:
https://m.ginlong.com/
The username (email) and the password are needed in the python script!

Step2:
install node-red
I used a Raspberry Pi for this project:
https://nodered.org/docs/getting-started/raspberrypi

Step3:
Install mosquitto if you allready haven't
http://www.steves-internet-guide.com/in ... tto-linux/

Step4:
I used the "ginlong-scraper.py" from https://github.com/dkruyt/ginlong-scraper
I copied the file to home/pi/ginlong-scraper
Then I've eddited the file as follows:

Spoiler: show

Code: Select all

#!/usr/bin/python
import requests
import urllib
import json
import datetime
import time
import os
import logging
import schedule

# Not all keys are avilable depending on your setup
COLLECTED_DATA = {
    'DC_Voltage_PV1': '1a', 
    'DC_Voltage_PV2': '1b', 
    'DC_Current1': '1j', 
    'DC_Current2': '1k', 
    'AC_Voltage': '1ah', 
    'AC_Current': '1ak', 
    'AC_Power': '1ao', 
    'AC_Frequency': '1ar', 
    'DC_Power_PV1': '1s', 
    'DC_Power_PV2': '1t', 
    'Inverter_Temperature': '1df', 
    'Daily_Generation': '1bd', 
    'Monthly_Generation': '1be', 
    'Annual_Generation': '1bf', 
    'Total_Generation': '1bc', 
    'Generation_Last_Month': '1ru', 
    'Power_Grid_Total_Power': '1bq', 
    'Total_On_grid_Generation': '1bu', 
    'Total_Energy_Purchased': '1bv', 
    'Consumption_Power': '1cj', 
    'Consumption_Energy': '1cn', 
    'Daily_Energy_Used': '1co', 
    'Monthly_Energy_Used': '1cp', 
    'Annual_Energy_Used': '1cq', 
    'Battery_Charge_Percent': '1cv'
}

def do_work():
    # solis/ginlong portal config
    username        = username from step1'
    password        = 'password from step1'
    domain          = 'm.ginlong.com'
    lan             = '2'
    deviceId        = ''

    ### Output ###

    # Influx settings
    influx              = 'FALSE'
    influx_database     = ''
    influx_server       = ''
    influx_port         = ''
    influx_user         = ''
    influx_password     = ''
    influx_measurement  = ''

    # pvoutput
    pvoutput            = 'FALSE'
    pvoutput_api        = ''
    pvoutput_system     = ''

    # MQTT
    mqtt                = 'TRUE'
    mqtt_client         = 'pv'
    mqtt_server         = 'ip adres where you run mosquitto from'
    mqtt_username       = '*********'
    mqtt_password       = '*********'

    ###

    if username == "" or password == "":
        logging.error('Username and password are mandatory for Ginlong Solis')
        return

    # Create session for requests
    session = requests.session()

    # building url
    url = 'https://'+domain+'/cpro/login/validateLogin.json'
    params = {
        "userName": username,
        "password": password,
        "lan": lan,
        "domain": domain,
        "userType": "C"
    }

    # default heaeders gives a 403, seems releted to the request user agent, so we put curl here
    headers = {'User-Agent': 'curl/7.58.0'}

    # login call
    loginSuccess = False
    try:
        resultData = session.post(url, data=params, headers=headers)
        resultJson = resultData.json()
        if resultJson.get('result') and resultJson.get('result').get('isAccept', 0) == 1:
            loginSuccess = True
            logging.info('Login successful for %s' % domain)
        else:
            raise Exception(json.dumps(resultJson))
    except Exception as e:
        logging.debug(e)
        logging.error('Login failed for %s' % domain)

    if loginSuccess:
        if deviceId == "":
            logging.info('Your deviceId is not set, auto detecting')
            url = 'http://'+domain+'/cpro/epc/plantview/view/doPlantList.json'

            cookies = {'language': lan}
            resultData = session.get(url, cookies=cookies, headers=headers)
            resultJson = resultData.json()

            plantId = resultJson['result']['pagination']['data'][0]['plantId']

            url = 'http://'+domain+'/cpro/epc/plantDevice/inverterListAjax.json?'
            params = {
                'plantId': int(plantId)
            }

            cookies = {'language': lan}
            resultData = session.get(url, params=params, cookies=cookies, headers=headers)
            resultJson = resultData.json()
            logging.debug('Ginlong inverter list: %s' % json.dumps(resultJson))

            # .result.paginationAjax.data
            deviceId = resultJson['result']['paginationAjax']['data'][0]['deviceId']

            logging.info('Your deviceId is %s' % deviceId)

        # get device details
        url = 'http://'+domain+'/cpro/device/inverter/goDetailAjax.json'
        params = {
            'deviceId': int(deviceId)
        }

        cookies = {'language': lan}
        resultData = session.get(url, params=params, cookies=cookies, headers=headers)
        resultJson = resultData.json()
        logging.debug('Ginlong device details: %s' % json.dumps(resultJson))

        # Get values from json
        updateDate = resultJson['result']['deviceWapper'].get('updateDate')
        inverterData = {'updateDate': updateDate}
        for name,code in COLLECTED_DATA.items():
            inverterData[name] = float(0)
            value = resultJson['result']['deviceWapper']['dataJSON'].get(code)
            if value is not None:
                inverterData[name] = float(value)

        # Print collected values
        logging.debug('Results from %s:' % deviceId)
        logging.debug('%s' % time.ctime((updateDate) / 1000))
        for key,value in inverterData.items():
            logging.debug('%s: %s' % (key,value))

        # Write to Influxdb
        if influx.lower() == "true":
            logging.info('InfluxDB output is enabled, posting outputs now...')
            from influxdb import InfluxDBClient
            json_body = [
                {
                    "measurement": influx_measurement,
                    "tags": {
                        "deviceId": deviceId
                    },
                    "time": int(updateDate),
                    "fields": inverterData
                }
            ]
            if influx_user != "" and influx_password != "":
                client = InfluxDBClient(host=influx_server, port=influx_port, username=influx_user, password=influx_password )
            else:
                client = InfluxDBClient(host=influx_server, port=influx_port)
            
            client.switch_database(influx_database)
            success = client.write_points(json_body, time_precision='ms')
            if not success:
                logging.error('Error writing to influx database')

        # Write to PVOutput
        if pvoutput.lower() == "true":
            logging.info('PvOutput output is enabled, posting results now...')

            headers = {
                "X-Pvoutput-Apikey": pvoutput_api,
                "X-Pvoutput-SystemId": pvoutput_system,
                "Content-type": "application/x-www-form-urlencoded",
                "Accept": "text/plain"
            }

            # make seconds
            tuple_time = time.localtime(updateDate / 1000)
            # Get hour and date
            date = time.strftime("%Y%m%d", tuple_time)
            hour = time.strftime("%H:%M", tuple_time)

            pvoutputdata = {
                    "d": date,
                    "t": hour,
                    "v1": inverterData['Daily_Generation'] * 1000,
                    "v2": inverterData['AC_Power'],
                    "v3": inverterData['Daily_Energy_Used'] * 1000,
                    "v4": inverterData['Consumption_Power'],
                    "v6": inverterData['AC_Voltage']
            }
#Python3 change
            encoded = urllib.parse.urlencode(pvoutputdata)

            pvoutput_result = requests.post("http://pvoutput.org/service/r2/addstatus.jsp", data=encoded, headers=headers)
            logging.debug('PvOutput response: %s' % pvoutput_result.content)
            if pvoutput_result.status_code != 200:
                logging.error('Error posting to PvOutput')

        # Push to MQTT
        if mqtt.lower() == "true":
            logging.info('MQTT output is enabled, posting results now...')

            import paho.mqtt.publish as publish
            msgs = []

            mqtt_topic = ''.join([mqtt_client, "/" ])   # Create the topic base using the client_id and serial number

            if (mqtt_username != "" and mqtt_password != ""):
                auth_settings = {'username':mqtt_username, 'password':mqtt_password}
            else:
                auth_settings = None
            
            msgs.append((mqtt_topic + "updateDate", int(updateDate), 0, False))
            for key,value in inverterData.items():
                msgs.append((mqtt_topic + key, value, 0, False))
            
            publish.multiple(msgs, hostname=mqtt_server, auth=auth_settings)


def main():
    global next_run_yes
    try:
        do_work()
    except Exception as e:
        logging.error('%s : %s' % (type(e).__name__, str(e)))
    next_run_yes = 1


global next_run_yes


schedule.every(1).minutes.at(':00').do(main).run()
while True:
    if next_run_yes == 1:
        next_run = schedule.next_run().strftime('%d/%m/%Y %H:%M:%S')
        logging.info('Next run is scheduled at %s' % next_run)
        next_run_yes = 0
    schedule.run_pending()
    time.sleep(1)

Step5:
install all this:
sudo apt install python3-pip
pip3 install requests
pip3 install schedule
pip3 install paho-mqtt

Step6:
automatic run the script at boot:
type: crontab -e
in crontab add the end:
@reboot /usr/bin/python3 /home/pi/ginlong-scraper/ginlong-scraper.py

Change the location "home/pi" if needed

Reboot the server/pi now please

---

Now the setup is complete, configure nodered and domoticz:
In domoticz add new hardware: "MQTT Client"
Fill in the 4 parameters from the MQTT server (Remote adress, port, username, password)

First lets test MQTT by downloading a MQTT explorer:
http://mqtt-explorer.com/
I use the portable

Open the app and fill in again the same MQTT parameters that you already used a few times.
Click connect and hopefully everything works. Wait 1 minute,
The pythonscripts runs every minute.
You should at least see a "domoticz" and "pv" topic.
the pv is the pythonscript unless you've altered the name in the script
Leave the MQTT Explorer open, it will help you to configure nodered

Go to nodered via the ip of the server like: 192.168.0.100:1880

In node red build the flow as follows:
"mqtt in"->"rbe"->"template"->"mqtt out"



When creating a MQTT block the first time:
The MQTT server is again to be defined with the same credentials.
So same server ip, port, username and password (from mosquitto server)

"mqtt in":
Server: Select the MQTT server
Topic: check the MQTT Explorer to select a topic
http://users.telenet.be/afbksu/1.jpg
http://users.telenet.be/afbksu/2.jpg

"rbe":
change only Modeto: "block unless value changes"

"template":
Property: msg.payload
Template: {"command" : "udevice", "idx": 778, "nvalue": 999, "svalue":"{{payload}}"}
Only change the idx to the idx you want to change in domoticz!

"mqtt out":

[palmerfarmer]

this is just what i need, does it still work with the new website that ginlong are hosting now?

[PeJeWe]

https://github.com/ZuinigeRijder/SolisCloud2PVOutput

Import directly from soliscloud to domoticz without mqtt
And for use with de new soliscloud.com
If you don't use pvoutput, you can set it disabled.

[palmerfarmer]

Thanks, got it working! (after a few rookie errors)