Rhasspy plugin

[Marathon2010]

I've launched a plugin for integrating Rhasspy voice assistant with Domoticz. Plugin can be found (including extensive documentation) on https://github.com/TurnOfAFriendlyCard/ ... spy-Plugin. Rhasspy documentation can be found on https://rhasspy.readthedocs.io/en/latest/.

The plugin should be able to process next dialogue:

Question (spoken): Hey Jarvis put on the kitchen light
Result in Domoticz: the kitchen light is switched on
Response (spoken): The light in the kitchen is swithed on

Main topics:

• Usage of MQTT to receive data from and send data to Rhasspy.

• Define in Rhasspy what device in Domoticz to process.

• Define in Rhasspy what base response to be spoken after processing in Domoticz.

• Translate state of the device from local language in Rhasspy to local language in Domoticz (and vice versa).

• Process requests via standard Domoticz API from and to the Domoticz database.

• Process only Domoticz related requests via this plugin (so non Domoticz requests can be processed in a different way).

The base concept is below.

Integration Rhasspy Domoticz.jpg (109.84 KiB) Viewed 1894 times

[wim57]

I tried to use Rhasspy and your plugin but setting all up is not quite easy. But I think I have it almost running.
One thing I do not understand is the use of a user variable for every device. Until now I did not use user variables at all and having to define one for each (new) device seems a lot of avoidable work. Why not define the right device names in Rhasspy sentences/slots?

Still, your plugin seems a good start for using speech control with Domoticz, thank you!


Wim

[Marathon2010]

Usage of user variables is optional, not mandatory. Assume next sentence.ini in Rhasspy:

[dzGetDevices]
what is outside temperature{device:Temperature Outside}(:){speakresponse:The temperature outside is}

The device "Temperature Outside" is listed in the Devices tab in Domoticz. So this will work definitely work.

But assume you've defined the user variable "TempOut" with a string value "Temperature Outside". This implies you can use this in the sentences.ini as well:

[dzGetDevices]
what is outside temperature{device:TempOut}(:){speakresponse:The temperature outside is}

So just two ways of working.

[wim57]

That is what I first expected but the plugin did not find my device until I added a uservariable with the device-name as name and value.
I will look again if something else was the problem.

[wim57]

Ok, I tried again but I do not think it is working as you described.
To test I added some debug statements in the plugin and also commented the 2 lines that lookup the uservar.

Code: Select all

       def domoGetIDX(domoticz_device):
            domoticz_idx = 0
            # first try to translate the supplied device name to the value of a user variable
            # so user variables names are logical device names and will containt the actual device names
            Domoticz.Log("Invoer device <" + domoticz_device + ">")
            domoticz_result = domoRequest("getuservariables")
            #if domoticz_result.ok and (domoticz_usrvar := searchJSON(f"result[?Name=='{domoticz_device}'].Value", domoticz_result.json())):
            #    domoticz_device = domoticz_usrvar        
            Domoticz.Log("Uservar device <" + domoticz_device + ">")
            # determine the IDX of the devicename        
            domoticz_result = domoRequest("devices_list")
            if domoticz_result.ok:    
                domoticz_idx = searchJSON("result[?name=='" + domoticz_device + "'].idx", domoticz_result.json())
                Domoticz.Debug("Device IDX <" + str(domoticz_idx) + "> (" + str(domoticz_device) + ")")
            return domoticz_idx

I have no uservars defined but the domoticz_device still gets updated after the SearchJson thus overwriting the value (with <None>) that was originial there. And then of course the device.idx can not be found. If I comment the 2 lines that lookup the device in uservars everything is working ok.
So I think there must be a bug in the statement:

Code: Select all

if domoticz_result.ok and (domoticz_usrvar := searchJSON(f"result[?Name=='{domoticz_device}'].Value", domoticz_result.json())):

It seems not to work ok when no uservars are present.

[Marathon2010]

Will investigate, raised an issue on GitHub to follow up (https://github.com/TurnOfAFriendlyCard/ ... n/issues/1).

[wim57]

I've changed

Code: Select all

if domoticz_result.ok and (domoticz_usrvar := searchJSON(f"result[?Name=='{domoticz_device}'].Value", domoticz_result.json())):
           domoticz_device = domoticz_usrvar        

into

Code: Select all

            if domoticz_result.ok:
                domoticz_usrvar = searchJSON(f"result[?Name=='{domoticz_device}'].Value", domoticz_result.json())
                if domoticz_usrvar != "None":
                    domoticz_device = domoticz_usrvar        

and it's working.

[Marathon2010]

Highly appreciated Wim

[wim57]

It wasn't a difficult fix.
I am going to experiment a little further, all my temperature sensors are temp/hum. I'll see if I can add them to the validation.

I did not try speech recognition yet for lack of a suitable microphone.

[waltervl]

I've launched a plugin for integrating Rhasspy voice assistant with Domoticz. Plugin can be found (including extensive documentation) on https://github.com/TurnOfAFriendlyCard/ ... spy-Plugin. Rhasspy documentation can be found on https://rhasspy.readthedocs.io/en/latest/.

The plugin should be able to process next dialogue:

Question (spoken): Hey Jarvis put on the kitchen light
Result in Domoticz: the kitchen light is switched on
Response (spoken): The light in the kitchen is swithed on

Main topics:

• Usage of MQTT to receive data from and send data to Rhasspy.

• Define in Rhasspy what device in Domoticz to process.

• Define in Rhasspy what base response to be spoken after processing in Domoticz.

• Translate state of the device from local language in Rhasspy to local language in Domoticz (and vice versa).

• Process requests via standard Domoticz API from and to the Domoticz database.

• Process only Domoticz related requests via this plugin (so non Domoticz requests can be processed in a different way).

Just a warning. The plugin Framework is not designed to support this kind of interfaces! From the wiki https://wiki.domoticz.com/Developing_a_Python_plugin :

The Domoticz Python Framework is not a general purpose Domoticz extension. It is designed to allow developers to easily create an interface between a piece of Hardware (or Virtual Hardware) and Domoticz. To that end it provides capabilities that seek to do the hard work for the developer leaving them to manage the messages that move backwards and forwards between the two.

This plugin does not make an interface with hardware, it interfaces with Domoticz and a voice assitant application.
I am not saying it will not work as a plugin but the better way would be to have it as a separate python script/service installed somewhere on your system and so run it outside Domoticz.

[wim57]

Good point Walter, I was already thinking if this has to be a plugin since it uses the API to access Domoticz.

[Marathon2010]

Interesting. I started using the local command option in Rhasspy (within Intent Handling) to integrate with Domoticz (and got that up and running). Next I switched to the Plugin approach. So my next move will be to stop the development of this Plugin and restart adjusting the Python script and make that script available for integrating with Domoticz. Will keep using the GitHub environment, so the Python local command approach remains available to download.

[wim57]

May be a good idea.

Just one other remark:
The plugin translates the state of a Domoticz device from the Domoticz language to the Rhasspy language. But I think device states are always reported in english and Domoticz uses the language setting only for the frontend.
In my case the Domoticz language is 'dutch' the webpage shows 'uit' (off) but the json reports 'off'. So it seems you can always use 'en' as the from-language for tts.

[Marathon2010]

Version 2.0 has been published on GitHub. Not a Domoticz plugin anymore, just a python script which is kicked off via the Local Command option within Intent Handling. MQTT is not used in this integration yet, will be next step to include as well as running as a service.

[waltervl]

Why the need for MQTT if you can run it without MQTT?
You only force people to use MQTT if they not use this already. Or is there an advantage in using MQTT?

[Marathon2010]

Rhasspy automatically installs Mosquitto MQTT broker for messaging between the Rhasspy modules running on that machine, called “Internal MQTT”. Because it is self-contained, it is set up with few settings; and is suitable where all the Rhasspy modules are running on the same machine.

So, either use that internal MQTT broker or use the external MQTT broker if it is already installed. And in v1 I've already enabled listening to MQTT messages triggering the intent of the Domoticz request. So I think it makes sense to apply the MQTT approach, however open for other opinions to go ahead.

[waltervl]

Well if MQTT broker is part of the Rhasspy install it does not matter of course.

[Marathon2010]

Version 3.0 of the integration between Rhasspy and Domoticz is published. Two concepts are now available: 1) via Local Command option in Rhasspy and 2) via MQTT. Schematic looks like below:

Integration Rhasspy Domoticz - LC - small.jpg (13.83 KiB) Viewed 302 times

Integration Rhasspy Domoticz - MQTT - small.jpg (15.01 KiB) Viewed 302 times

Besides both concepts available, next functionality has been processed:

• Extended speakreponse tag in Rhasspy sentence so own feedback can be configured combined with Domoticz data.

• Added setpoint device, so update of temperature is possible.

• Running MQTT script in background.

• Introduced specific translation approach (where deepl misses the context).

Plugin can be found (including extensive documentation) on https://github.com/TurnOfAFriendlyCard/ ... spy-Plugin. Rhasspy documentation can be found on https://rhasspy.readthedocs.io/en/latest/