Need plugin example for simple GET request for JSON Topic is solved

Python and python framework

Moderator: leecollings

Post Reply
getSurreal
Posts: 19
Joined: Friday 31 July 2015 0:59
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Contact:

Need plugin example for simple GET request for JSON

Post by getSurreal »

I've created this GET request and tried setting the protocol to both HTTP and JSON, but onMessage never gets triggered. I've tried url with and without the full path. And I've tried it with and without the headers.

Code: Select all

            
            url = 'http://'+Parameters["Address"]+'/query/info'
            headers = {"Connection": "keep-alive", "Accept": "Content-Type: text/html; charset=UTF-8"}
            Domoticz.Send("", "GET", url, headers)
Looking at some of what others are doing, they are using urllib. Is that the correct way to go, or should Domoticz.Send() be used for communicating to the device?

I'm able to communicate and receive data using urllib.
User avatar
Dnpwwo
Posts: 820
Joined: Sunday 23 March 2014 9:00
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: Melbourne, Australia
Contact:

Re: Need plugin example for simple GET request for JSON

Post by Dnpwwo »

@getSurreal,

Using the built in functionality will make your plugin play well with others and make device updating more responsive because you will see disconnect immediately. I untroduced a couple of HTTP bugs last weekend unfortunately I will post a fix tomorrow and a simple sample plugin.
The reasonable man adapts himself to the world; the unreasonable one persists to adapt the world to himself. Therefore all progress depends on the unreasonable man. George Bernard Shaw
User avatar
Dnpwwo
Posts: 820
Joined: Sunday 23 March 2014 9:00
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: Melbourne, Australia
Contact:

Re: Need plugin example for simple GET request for JSON

Post by Dnpwwo »

@getSurreal,

Pull request #1336 will fix the remaining issues in HTTP from the upgrade a week ago. Should be in the next beta.

As discussed I've created a simple HTTP sample that connects, GETs handles a redirect and GETs again from http://www.google.com

Have a look and try the new version, if you have more problems let me know

Code: Select all

# Goolgle Home page example
#
# Author: Dnpwwo, 2017
#
#   Demonstrates HTTP connectivity.
#   After connection it performs a GET on  www.google.com and receives a 302 (Page Moved) response
#   It then does a subsequent GET on the Location specified in the 302 response and receives a 200 response.
#
"""
<plugin key="Google" name="Goolgle Home page example" author="Dnpwwo" version="1.0.0" externallink="https://www.google.com">
    <params>
        <param field="Address" label="IP Address" width="200px" required="true" default="www.google.com"/>
        <param field="Port" label="Port" width="30px" required="true" default="80"/>
        <param field="Mode6" label="Debug" width="75px">
            <options>
                <option label="True" value="Debug"/>
                <option label="False" value="Normal"  default="true" />
            </options>
        </param>
    </params>
</plugin>
"""
import Domoticz

class BasePlugin:
    
    def __init__(self):
        return

    def onStart(self):
        if Parameters["Mode6"] == "Debug":
            Domoticz.Debugging(1)
        DumpConfigToLog()
        Domoticz.Transport(Transport="TCP/IP", Address=Parameters["Address"], Port=Parameters["Port"])
        Domoticz.Protocol("HTTP")
        Domoticz.Connect()

    def onStop(self):
        Domoticz.Log("Plugin is stopping.")

    def onConnect(self, Status, Description):
        if (Status == 0):
            Domoticz.Debug("Google connected successfully.")
            data = ''
            headers = { 'Content-Type': 'text/xml; charset=utf-8', \
                        'Connection': 'keep-alive', \
                        'Accept': 'Content-Type: text/html; charset=UTF-8', \
                        'Host': Parameters["Address"]+":"+Parameters["Port"], \
                        'User-Agent':'Domoticz/1.0', \
                        'Content-Length' : "%d"%(len(data)) }
            Domoticz.Send(data, 'GET', '/', headers)
        else:
            Domoticz.Log("Failed to connect ("+str(Status)+") to: "+Parameters["Address"]+":"+Parameters["Port"]+" with error: "+Description)

    def onMessage(self, Data, Status, Extra):
        strData = Data.decode("utf-8", "ignore")
        Domoticz.Log("onMessage: Status="+str(Status))
        Domoticz.Log("Headers:")
        for x in Extra:
            Domoticz.Log("    '" + x + "':'" + str(Extra[x]))

        if (Status == 200):
            Domoticz.Log("Good Response received from Google.")
        elif (Status == 302):
            Domoticz.Log("Google returned a Page Moved Error.")
            headers = { 'Content-Type': 'text/xml; charset=utf-8', \
                        'Connection': 'keep-alive', \
                        'Accept': 'Content-Type: text/html; charset=UTF-8', \
                        'Host': Parameters["Address"]+":"+Parameters["Port"], \
                        'User-Agent':'Domoticz/1.0', \
                        'Content-Length' : "0" }
            Domoticz.Send("", "GET", Extra["Location"], headers)
        elif (Status == 400):
            Domoticz.Error("Google returned a Bad Request Error.")
        elif (Status == 500):
            Domoticz.Error("Google returned a Server Error.")
        else:
            Domoticz.Error("Google returned a status: "+str(Status))

    def onCommand(self, Unit, Command, Level, Hue):
        Domoticz.Debug("onCommand called for Unit " + str(Unit) + ": Parameter '" + str(Command) + "', Level: " + str(Level))

    def onDisconnect(self):
        Domoticz.Log("Device has disconnected")

    def onHeartbeat(self):
        Domoticz.Debug("onHeartbeat called")

global _plugin
_plugin = BasePlugin()

def onStart():
    global _plugin
    _plugin.onStart()

def onStop():
    global _plugin
    _plugin.onStop()

def onConnect(Status, Description):
    global _plugin
    _plugin.onConnect(Status, Description)

def onMessage(Data, Status, Extra):
    global _plugin
    _plugin.onMessage(Data, Status, Extra)

def onCommand(Unit, Command, Level, Hue):
    global _plugin
    _plugin.onCommand(Unit, Command, Level, Hue)

def onDisconnect():
    global _plugin
    _plugin.onDisconnect()

def onHeartbeat():
    global _plugin
    _plugin.onHeartbeat()

# Generic helper functions
def DumpConfigToLog():
    for x in Parameters:
        if Parameters[x] != "":
            Domoticz.Debug( "'" + x + "':'" + str(Parameters[x]) + "'")
    Domoticz.Debug("Device count: " + str(len(Devices)))
    for x in Devices:
        Domoticz.Debug("Device:           " + str(x) + " - " + str(Devices[x]))
        Domoticz.Debug("Device ID:       '" + str(Devices[x].ID) + "'")
        Domoticz.Debug("Device Name:     '" + Devices[x].Name + "'")
        Domoticz.Debug("Device nValue:    " + str(Devices[x].nValue))
        Domoticz.Debug("Device sValue:   '" + Devices[x].sValue + "'")
        Domoticz.Debug("Device LastLevel: " + str(Devices[x].LastLevel))
    return
The reasonable man adapts himself to the world; the unreasonable one persists to adapt the world to himself. Therefore all progress depends on the unreasonable man. George Bernard Shaw
getSurreal
Posts: 19
Joined: Friday 31 July 2015 0:59
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Contact:

Re: Need plugin example for simple GET request for JSON

Post by getSurreal »

@Dnpwwo

Thanks for your help. onMessage is now getting triggered. My problem now is parsing it. When using urllib, the response was clean json, but when using Data.decode("utf-8", "ignore") there is some extra information and json.loads does not handle the extra data being in there.

Is there some other Domoticz method that handles parsing the json? Otherwise, I guess I can trim everything before and after the curly braces.

Code: Select all

HTTP/1.1 200 OK 
Transfer-Encoding: chunked 

154 
{"name":"DOWNSTAIRS","mode":3,"state":0,"fan":0,"fanstate":0,"tempunits":0,"schedule":1,"schedulepart":1,"away":0,"spacetemp":69.0,"heattemp":68.0,"cooltemp":76.0,"cooltempmin":35.0,"cooltempmax":99.0,"heattempmin":35.00,"heattempmax":99.0,"hum":0,"hum_setpoint":48,"dehum_setpoint":0,"hum_active":70,"setpointdelta":2.0,"availablemodes":0}
0 
User avatar
Dnpwwo
Posts: 820
Joined: Sunday 23 March 2014 9:00
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: Melbourne, Australia
Contact:

Re: Need plugin example for simple GET request for JSON

Post by Dnpwwo »

@getSurreal,

Pull request #1336 has not been merged yet, as soon as it is you will just get the JSON in the data parameter.
The reasonable man adapts himself to the world; the unreasonable one persists to adapt the world to himself. Therefore all progress depends on the unreasonable man. George Bernard Shaw
getSurreal
Posts: 19
Joined: Friday 31 July 2015 0:59
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Contact:

Re: Need plugin example for simple GET request for JSON

Post by getSurreal »

@Dnpwwo

I had saw a new update available and thought it was in there, but now I see #1336 is merged and it is working for me. We all very much appreciate the work you're putting into the plugin module. I think a lot more devices will get supported much more quickly.

Couple more questions for you...

I'm looking for different responses when I send a heartbeat vs when I send a command. Is that by design that all responses go to onMessage? In this regard, it seemed easier to handle the responses with urllib at the time I was making the request.

My plugin is nearly complete, could or should we do pull requests to add plugins?
User avatar
Dnpwwo
Posts: 820
Joined: Sunday 23 March 2014 9:00
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: Melbourne, Australia
Contact:

Re: Need plugin example for simple GET request for JSON

Post by Dnpwwo »

@getSurreal,

Couple of answers...

The plan is not to add plugins to the Domoticz code base but to keep them separate. Gizmocuz and I have discussed hosting them somewhere and have search and install capability built in to the web UI. Still looking for good ideas on how to do this from users experiences...

It is by design that all messages go through a single callback. Event based coding requires a mind shift to use properly and initially it does seem harder particularliy when moving Python from an old cron script but it has benefits. Under normal running urllib is easier to use but it has one major weakness when running in a multi-plugin environment. When the device you are connecting to is not responding it can lock the Python framework for 10 to 20 seconds everytime you try and connect. This stops every plugin. Even if you put a 'reasonable' timeout on the connection of a couple of seconds, that is still time where other plugins are unresponsive. I already run 6 plugins and if you fast forward to when a lot of hardware is supported via Python this will leave to a lot of issues for users. A 2 or 3 second connection timeout is 20% to 30% of the time where all plugins are stalled. The built in connection is asynchronous so every plugin could be trying to connect at the same time and the framework will remain responsive.

I suspect when we get a repository up and running we will need to add a 'friendliness' rating to them
The reasonable man adapts himself to the world; the unreasonable one persists to adapt the world to himself. Therefore all progress depends on the unreasonable man. George Bernard Shaw
getSurreal
Posts: 19
Joined: Friday 31 July 2015 0:59
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Contact:

Re: Need plugin example for simple GET request for JSON

Post by getSurreal »

@Dnpwwo

Thanks for that explanation. That is very important to know and understand. In that case I would say the the plugin repository would need to review plugins and reject them if they don't meet the criteria.

That's kind of unfortunate that plugins won't be included with Domoticz.

If I wrote it in C++ instead like the other hardware modules, then they would accept it?
Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest