Integrate OpenAI's ChatGPT chatbot with Domoticz

[Lemme]

I think that would be so cool. Alexa and Google Home are pretty dumb devices, even though they do the job well. But adding the extra AI layer where the assistant can understand the spoken context and handle multiple requests simultaneously is groundbreaking. Unfortunately, we might be looking a year or two ahead of this integration.

So I thought, why not make it yourself!?

**************** UPDATE ***********************************************************************
I have published the code on GitHub. It is the basics, but it will get you started:
https://github.com/LemmeDasker/Domoticz ... ntegration
**************************************************************************************************
I can speak to ChatGPT and give it commands like turning different lamps on/off in the same sentence. It can give me the status of all or individual doors if they are open/closed, and it can also function as a traditional digital assistant answering questions.

It still comes with minor errors occasionally, but it will get better as OpenAI matures the LLM model.

Right now, I have ChatGPT controlling Domoticz!

[kniazio]

Can you show us how to do it?

[Kedi]

Looking at https://openai.com/blog/chatgpt-can-now ... -and-speak Blog from 25 september 2023
It is really a miracle that you already integrated the latest ChatGPT into Domoticz.
Like the previous post: Tell us how you did it.

[Kedi]

I also ask chatGPT the question "connect chatgpt to domoticz"
The answer looks valid by using the node.js 'axios' library, but is based on text.
Currently I am using Google Home with dzga and that works to my satisfaction, but love to know how the voice connection between chatGPT and Domoticz will work.

B.t.w. I also asked Bard to connect Bard to Domoticz And the answer was that I should install HA and then via HA control Domoticz. So much for Bard.

[akamming]

Can you show us how to do it?

Why don’t you ask chatgpt?

[kniazio]

I asked

[Lemme]

Hi, guys!

This is how I did it:
I used a combination of Azure's Speech SDK and OpenAI's Python library. The hardware setup is based on a small Intel Compute Stick that is easy to hide, along with a Jabra Speak used as a microphone/speaker, which fits nicely on the shelf.

Here's the flow:

• First, I utilize Azure's 'Custom Keyword' feature to detect any action, similar to saying 'Alexa' or 'Hey Google'.

• Then, I use Azure's 'Speech to Text' to obtain the text string.

• This text string is passed to OpenAI's ChatGPT, along with additional 'system' instructions.

• The returned text from ChatGPT adheres to the provided system instructions. I manipulate and convert this information and pass it as an HTTP call to Domoticz's API.

• If the text is not related to Domoticz, I use Azure's 'Text to Speech' to generate the voice response, which is played through my speaker.

I hope this clarifies the process for you.

[kniazio]

Respect. Could you create a tutorial on how to do it step by step?

[Kedi]

I just wonder what do I get extra from this setup compared to a Google Nest with dzga connected to Domoticz?
Because every voice command to Domoticz or status inquiry of a device(s) or rooms works perfect.

[kniazio]

I just wonder what do I get extra from this setup compared to a Google Nest with dzga connected to Domoticz?
Because every voice command to Domoticz or status inquiry of a device(s) or rooms works perfect.

Not everyone has Nest.

[Kedi]

That's right, but they can be bought like a small Intel Compute Stick and a Jabra Speaker
I also see that Speech to Text is only 5 audio hours free per month.
B.t.w. The Intel Compute Sticks are discontinued.

[Kedi]

I know it is from the beginning of 2023, so chatGPT has progressed, but still.
https://www.reddit.com/r/GPT3/comments/ ... _while_it/
And especially the YouTube at midway in that post: https://youtu.be/rxUm-2x-2dM
In my opinion we are still far away from chatGPT or Bard or... really know and understand humans.

[Lemme]

I just wonder what do I get extra from this setup compared to a Google Nest with dzga connected to Domoticz?
Because every voice command to Domoticz or status inquiry of a device(s) or rooms works perfect.

I have created skills for both Alexa and Google Home (no knowledge of Google Nest). The drawback with them is that they can only handle one request at a time. And the commands often need to be very simple, so they are actually pretty dumb. But they work well at what they do.

If you have four lamps and you would like to handle them individually in one sentence, you need to create at least 16 individual hard-coded request entries for all combinations of on-off situations. If you add an extra lamp, you can increase this to 25. And you still have to give pretty precise commands.

With ChatGPT, it fully understands Danish now. It is also a language model that '...can follow complex instructions in natural language and solve difficult problems with accuracy.' You don't have to program every combination or use exact phrases; all this is handled by the ChatGPT model.

You can speak to it using natural language, and it will understand the context and act accordingly. I can say, 'Can you please turn on the pendant light, but turn off both the floor lamp and the lamp over in the corner.'

That's what you get extra.

[kniazio]

Make Tutorial. Please.

[Kedi]

I see. I my case I need 3 short sentences to get the same result.
All my lights are controlled by presence, ambient light and other charistics by dzVents scripts in a fully automatic way.
So I don't need that much voice intervention, but it would be nice to have a less strict way of communicating to get things done.

So please make a tutorial.

B.t.w. ChatGPT suggests that you should use "Hey Google" followed by the sentence above in case of Google Home, but that does not work.

[Lemme]

Make Tutorial. Please.

I think I could put something up on GitHub in a couple of weeks if there is a real interest.

[kniazio]

I'm interested

[Lemme]

Hi guys
The code are avaliable on GitHub:
https://github.com/LemmeDasker/Domoticz ... ntegration

Please share your experiences here...

Br
Lemme

[kniazio]

Thank you very much. I will try.