Home Automation Voice Commands

[Javier]

Hi All,

I have created a free and open simple voice commands app which allows for chained voice commands. I added Domoticz to the supported hubs today.

Instructions can be found here:
https://plus.google.com/+JavierRefuerzo ... bSM8WKAKse

App can be found here:
https://play.google.com/store/apps/deta ... cecommands

As of now it only supports domoticz devices (not scenes), If there is any interest in this app I will be happy to add scene support if anyone is willing to test, I use a different HA system but domoticz was very easy to add to the existing project. The project will also work with AutoVoice for "OK Google" commands although it is not required. Additonal Instructions are here: https://plus.google.com/+JavierRefuerzo ... 2ANQYXS3bU

[Scoobydoo]

Tried this out tonight and it works well.

The ui could do with a complete overhaul as it's not simple to understand but once setup you can have voice control for your domoticz devices

Well done to the developer, I look forward to seeing how it progresses.

[markk]

Hi. I'm trying to set this up on my Galaxy Tab S but I'm not getting the confirmation splash after inputting IP address etc. can you please let me know what format it needs to be entered in? I've tried leaving %DOMOTICZ_ there and removing it but nothing seems to work. Would be grateful for a confirmation please. Many thanks.

[Scoobydoo]

When your inputting the various domoticz settings like ip address etc, remove everything that's in there already so you just have the ip address. Then press the bottom right save button. You don't get any confirmation that it's saved but if you go back into the setting you'll see its there.
After you've entered all the various settings, click on refresh domoticz list and you'll get a confirmation that it's connected successfully to domoticz

[markk]

Got it. Thanks for that.
Also, thanks to the developer for this. I love it

[Thomasdc]

i will try this!

[Javier]

I'm Glad it is working! I will start to work on the UI and have updated the Instructions. I still need a tester to implement more items for Domoticz.

[Thomasdc]

I'm Glad it is working! I will start to work on the UI and have updated the Instructions. I still need a tester to implement more items for Domoticz.

if you need something tested you can always post it here or pm me, if i can i will help you out with testing! (scenes, groups, on/of devices, dimmers, temperatures, alarm, those are things where i can help you for testing )

[Javier]

I'm Glad it is working! I will start to work on the UI and have updated the Instructions. I still need a tester to implement more items for Domoticz.

if you need something tested you can always post it here or pm me, if i can i will help you out with testing! (scenes, groups, on/of devices, dimmers, temperatures, alarm, those are things where i can help you for testing )

Thanks Thomasdc,

I'll PM you with a link to beta when I have some time to add items.

[brandie21474]

I'm Glad it is working! I will start to work on the UI and have updated the Instructions. I still need a tester to implement more items for Domoticz.

no problem, send me a pm on the subjects you need to test.
I'm also trying to use another language (dutch) need some help on that, it doesnt work smoothly

[locogek]

Looks great! I've done it right know with Tasker and json but that takes more time for sure. I will try this when I'm at home tomorrow. Thanks for your work

Verstuurd vanaf mijn ONE A2003 met Tapatalk

[Javier]

no problem, send me a pm on the subjects you need to test.
I'm also trying to use another language (dutch) need some help on that, it doesnt work smoothly

Hi brandie,

Did you Change Settings>Domoticz_Settings>Domoticz_Device/Group Action Association, so the first word before the equal sign "=" is translated to Dutch, The word after the equal sign "=" should not be changed.

I used google translate for the following example so it may be incorrect

op=On,af=Off

Also Advanced_Settings>Change_Language you will need to translate and into Dutch.

en

You may also have to change "Word Corrections and replacements"

Hope this help,
Javier

[mschut]

Thanks to some great help of Javier I got it working on my phone AND Android Wear watch. I agree on the basic interface, but then again, it's the voice functionality that counts!

I am using Google Now in Dutch, as I really prefer voice interaction in my native language (also much more fun when my kids want to try). So in the app "Home Automation Voice Commands" I have added some dutch words in the "word corrections and replacements" part. For example, "uit" is replaced by off. Does feel a bit complex, but works fine. The option to completely switch to another language would be nice.

For now, a happy user for voice controlling switches. Would be nice to be able to extend to other devices, for example temperatures. Ask the temperature in a certain room, and get the answer spoken to me

Speeking of it, is there a way to speed up the time between speeking and performing the action? Feels like the system is waiting a few seconds before the command is executed. Wouldnt mind if it reacts a bit quicker.

[ro.]

Once set up, voice commands work nicely, but I am baffled by the weirdly minimal app interface!

Given that all domoticz devices (or scenes) already have meaningful names which get discovered in the initial set-up, can these not be automatically added to the vocabulary? Maybe because I already have my devices named with locations (e.g. "study desk light") I don't see any advantage of the "location-device" method - it just makes things seem overly complicated to set up.

[Javier]

Once set up, voice commands work nicely, but I am baffled by the weirdly minimal app interface!

Given that all domoticz devices (or scenes) already have meaningful names which get discovered in the initial set-up, can these not be automatically added to the vocabulary? Maybe because I already have my devices named with locations (e.g. "study desk light") I don't see any advantage of the "location-device" method - it just makes things seem overly complicated to set up.

Hi ro,

The reason I choose to used "location"-"Device" for naming items is to have the ability to chain together commands. At least in English and Spanish (the two languages I can speak) you can say things like "turn off the living room, dining room and kitchen lights", because the device is only said once the program needs to know the location, to trigger the correct command for each item. Also to possibly change the default room automatically based on indoor location tracking with items such as estimote beacons (eddiystone ble) or NFC.....As for setup as long as you have included all the locations and devices you can just say the command and if it is not a already linked the program will prompt you to link a new item, this way you only have to setup a device when it is used for the first time.


NOTES TO ALL USING DOMOTICZ

There is a bug in adding "Scenes" if you try to add more than one scene there will be data corruption with the scenes file and it will have to be deleted (on android device)....This will be fixed in the next update.

I am also working on translations but it will take some time as this is just a spare time project.

[ro.]

Given that all domoticz devices (or scenes) already have meaningful names which get discovered in the initial set-up, can these not be automatically added to the vocabulary? Maybe because I already have my devices named with locations (e.g. "study desk light") I don't see any advantage of the "location-device" method - it just makes things seem overly complicated to set up.

The reason I choose to used "location"-"Device" for naming items is to have the ability to chain together commands. At least in English and Spanish (the two languages I can speak) you can say things like "turn off the living room, dining room and kitchen lights", because the device is only said once the program needs to know the location, to trigger the correct command for each item.....

Fair enough, but could you not get this word-linking trick to work in a purely syntactic way without any human assistance in advance? If you hear "[VERB] 'the' [A] 'and' [C]", without knowing anything about the nature of A, B or C, could it not work out the intended meaning should be "[VERB] [A+C]" and "[VERB] [B+C]"? Indeed, with such a general mechanism, many similar patterns could be added to cope with other types of phrase which also make sense but don't fit location-device (e.g. "Turn on the garden lights and fountain", "Open and close the kitchen blinds").

In any case, lacking full AI support, I'd much prefer to be more long-winded and say "turn on the kitchen light and the dining room light", rather than have the hassle of dealing with the current app interface!

Perhaps my preference is skewed because I already have domoticz objects (groups or scenes) defined for where I think that kind of thing might be useful, so I could say instead "turn off downstairs lights" (action on a group) or "watch a movie" (trigger a scene). OK, so I've had to think about it in advance and put some effort in, but that effort seems more justified because the groups and scenes can be used in other useful circumstances (e.g. as the target of a button, timer or script action), and minimizes the number of places a change needs to be made when adding, moving or deleting a device (e.g. a new lamp gets added to the "kitchen" and "downstairs" groups).

As for setup as long as you have included all the locations and devices you can just say the command and if it is not a already linked the program will prompt you to link a new item, this way you only have to setup a device when it is used for the first time.

That's really cool, and in fact this is how I've been doing it since I fount it worked like that (on the phone - haven't tried on the watch yet). But I have a more than 100 devices already, so it seems unlikely I'll ever do them all - and even if I did, I can't imagine keeping everything linked correctly when I move things around!

[Javier]

I have to admit that the UI is my biggest problem and I appreciate your input...I think (and of course I have bias) the (code) voice commands works very well once setup....I have tried other Paid Voice programs and the setup(s) were much more difficult to setup and did not work as I desired.

As for pulling names automatically, that can be done very easily however my initial testers (including myself) had many devices and very few of them had (only) "location" and "device" in the name of the device/program/scene/group. I use the UD ISY and they have implemented and notes section for devices for voice purposes (mostly for people wanting to setup the Amazon echo), but I have observed my wife saying multiple "locations" for the same device, such as "counter" and "cabinet" lights, and "front yard" and "lawn" lights. So this single name for a device does not take into account how normal (non - automation) people say things. Also the need for indoor location tracking is a priority as my wife mentions to me "I'm in the kitchen, why do I have to say "turn on the tv in the kitchen"". Then there is the AutoVoice/Google Now integration, and the best way to improve latency and reduce errors in the program is to reduce the words that are needed tor trigger the program, this also improves false positives and reduces the number of word corrections needed, So Google Now is still useful as your personal assistant.

All that said I would like to improve the UI, and user experience, but please keep in mind this is a free app an I am developing in my spare time. It would be nice if every one named their devices/programs/scenes like you, but that is something the app can not control, and as obvious from this post, it is very difficult to force users to use a naming convention. I think it would be even harder to force users to rename there devices in their Home Automation Controller. Not to mention, implemented in this way, things like chaining and (future) indoor location may not work unless there is a delimiter (such as "-") between the location and the device, and if the delimiter was missing the program would know that this device does not require a location.

I only write this because I have been using voice commands for years, and was not satisfied by the offerings. The apps/programs that were (just) OK were too difficult and took too much time to setup. I hope you will help me make this app better with other UI suggestions, but please understand my thought process for not implement your suggestion even though it would be easy.

Best,
Javier

P.S. Most of the code is available as a tasker XML on GitHub, you are free to modify it to your needs, However there are things missing because of license agreements, and it must be exported as apk because of file size.

[ro.]

I have to admit that the UI is my biggest problem and I appreciate your input...

I don't know how much you can change given the framework it's built in, but it would be so much nicer to use if the menu structure looked more like the native Android Setting, which has everything in one long scrollable list, with bold subheadings for major topics, then multiple (option name, current value) pairs for that topic, which you can tap to change a value.

Obviously the voice input pop-up is something very special and you want that to be readily accessible, but maybe there are other ways to get there, maybe a home screen widget? That way, the app can present a more conventional front page a button to trigger voice input (so you don't have to do the magic "settings" business) next to some suggestions on what to say. What would be really great would be for it to dissect what you said (either live or from a history list) so you can see how it interpreted it, and make corrections for the future.

I think (and of course I have bias) the (code) voice commands works very well once setup....

I completely agree, it's amazingly good! I just don't like the "once setup" part...

As for pulling names automatically, that can be done very easily however my initial testers (including myself) had many devices and very few of them had (only) "location" and "device" in the name of the device/program/scene/group.

Not sure I understand what you mean - could you give examples where the existing device names would be unsuitable?
Admittedly, I'm only familiar with domoticz, but I'd have thought that any other system is going to need you to give unique, meaningful names to your devices. ...or are there some which are intrinsically structured as location-device?
But if that is the case, then domoticz does that too, with roomplans. I'd be happy to help get that working!

And on the subject of deeper integration, perhaps the app shouldn't have to figure everything out all by itself. If it would help to have voice-specific fields on a device/group/scene/roomplan (e.g. "spoken name"), then we can add that to domoticz.

I have observed my wife saying multiple "locations" for the same device, such as "counter" and "cabinet" lights, and "front yard" and "lawn" lights. So this single name for a device does not take into account how normal (non - automation) people say things.

True, but that's in addition to what I would think of as the "true name" for something. Isn't that what the "word replacement list" function is for?

Also the need for indoor location tracking is a priority, as my wife mentions to me "I'm in the kitchen, why do I have to say "turn on the tv in the kitchen""....

Getting a bit off-topic - I would really love to have this work - but I don't have a reliable way to do indoor location tracking. What are you using for that?

Then there is the AutoVoice/Google Now integration, and the best way to improve latency and reduce errors in the program is to reduce the words that are needed tor trigger the program, this also improves false positives and reduces the number of word corrections needed, So Google Now is still useful as your personal assistant.

Again, perhaps getting a bit deep in to the topic here, but to help me understand the constraints - how much flexibility do you have in defining what is a valid utterance? Do you have to give it complete phrases (I hope not!)? If you can give it individual words, can you tag them as parts of speech in a grammar?

All that said I would like to improve the UI, and user experience, but please keep in mind this is a free app an I am developing in my spare time.

Of course - I am also just scratching my own itch - but in the open source spirit, I am more than happy to make contributions (whether verbal or actual code) to help things progress.

It would be nice if every one named their devices/programs/scenes like you, but that is something the app can not control, and as obvious from this post, it is very difficult to force users to use a naming convention. I think it would be even harder to force users to rename there devices in their Home Automation Controller.

I completely agree, and I would not pretend that even in my system everything is named ideally. But would it not be a fair assumption that the name of a device in anybody's system is a good first approximation to what they would say when they tried to refer to it? Or to put it another way, surely it is reasonable to expect that if I say "turn on [x]" and I have exactly one device named "x" (of a type that can be turned on), that it should work without any further set-up beyond importing the devices. And this would just provide base functionality as a starting point - you could still do further set-up to reduce ambiguity (mark ineligible devices) or enable more convenient ways to accomplish the same thing (with locations etc.).

P.S. Most of the code is available as a tasker XML on GitHub, you are free to modify it to your needs, However there are things missing because of license agreements, and it must be exported as apk because of file size.

I'm not familiar with tasker but will see if I can make sense of it - it's certainly a huge file!

[Javier]

I don't know how much you can change given the framework it's built in, but it would be so much nicer to use if the menu structure looked more like the native Android Setting, which has everything in one long scrollable list, with bold subheadings for major topics, then multiple (option name, current value) pairs for that topic, which you can tap to change a value.

There is limitations to what tasker can do as far as the UI goes, A scrollable list does not have the option to change headings, but the (unmovable) screens can changed.

Obviously the voice input pop-up is something very special and you want that to be readily accessible, but maybe there are other ways to get there, maybe a home screen widget? That way, the app can present a more conventional front page a button to trigger voice input (so you don't have to do the magic "settings" business) next to some suggestions on what to say. What would be really great would be for it to dissect what you said (either live or from a history list) so you can see how it interpreted it, and make corrections for the future.

This again is a tasker UI limitation as widgets can not be created with child apps.

Not sure I understand what you mean - could you give examples where the existing device names would be unsuitable?
Admittedly, I'm only familiar with domoticz, but I'd have thought that any other system is going to need you to give unique, meaningful names to your devices. ...or are there some which are intrinsically structured as location-device?
But if that is the case, then domoticz does that too, with roomplans. I'd be happy to help get that working!

Names are all over the place from things like “No load living room light switch” to just “living room”, So this would cause every device to have a unique name in the app which could end up being hundreds of unique names, which may need hundreds of corrections for mis-recognized name words and parsing the command looking for hundreds of unique devices. When corrections are only needed for “locations” and “devices “ the app is looking for just a couple of dozen words so the odds for your command being executed correctly with very few word corrections is greatly improved. Also during initial tests people where adding extra words to devices such as “kitchen under cabinet lights”.

And on the subject of deeper integration, perhaps the app shouldn't have to figure everything out all by itself. If it would help to have voice-specific fields on a device/group/scene/roomplan (e.g. "spoken name"), then we can add that to domoticz.

Although we would like to think users would name things like we would like them named, I do not think that would be the case and the app be back in the place of possibly looking for hundreds of unique strings instead of a few dozen.

True, but that's in addition to what I would think of as the "true name" for something. Isn't that what the "word replacement list" function is for?

The word replacement list is for mis-recognized words and (in the case of autovoice) words that are mis-corrected. If used to force a new command we may again be back to looking through hundreds of stings instead of dozens, and it will start to get complicated....As a side not I have done this part in the past, it works for a small number of devices but the corrections get very hard to manage.

Getting a bit off-topic - I would really love to have this work - but I don't have a reliable way to do indoor location tracking. What are you using for that?

This is where the need for the location-device is the most important, and one of the main reasons I am doing this project. There are many ways (with much dificulty) to do this but the most promising is ble beacons. I have it working intermittently now but I'm just learning java (another reason the app was written in tasker)...my hope is to be able to implement the Estimote Indoor SDK http://estimote.com/indoor/ either through a tasker plugin or by developing the plugin myself...but because I am doing this is my spare time and I have a full time job and kids, if someone else beats me to making the plugin it would be less work for me to do and everything else is already in place...Side Note: The creator of AutoApps may have this plugin the pipeline.

Again, perhaps getting a bit deep in to the topic here, but to help me understand the constraints - how much flexibility do you have in defining what is a valid utterance? Do you have to give it complete phrases (I hope not!)? If you can give it individual words, can you tag them as parts of speech in a grammar?

It is very flexible and many Home Automation - Android enthusiast use it now, but this again goes back to the dozens vs hundreds argument mentioned above....Side Note: I used this method for years with unique device names and it worked OK, but changing to the trigger to “device” and “action” using REGEX greatly improved the reliability.

I completely agree, and I would not pretend that even in my system everything is named ideally. But would it not be a fair assumption that the name of a device in anybody's system is a good first approximation to what they would say when they tried to refer to it? Or to put it another way, surely it is reasonable to expect that if I say "turn on [x]" and I have exactly one device named "x" (of a type that can be turned on), that it should work without any further set-up beyond importing the devices. And this would just provide base functionality as a starting point - you could still do further set-up to reduce ambiguity (mark ineligible devices) or enable more convenient ways to accomplish the same thing (with locations etc.).

I could do that very easily but we get back to the hundreds vs dozens argument and this would make my main goals of indoor location and chaining commands not work as desired...Side Note. This project was originally called “home automation chaining voice commands” but google play has a 30 character limit.

I'm not familiar with tasker but will see if I can make sense of it - it's certainly a huge file!

This is another reason I decided to go with tasker. If you are familiar with any programing language you should not have a problem with tasker...except for the “Scenes” (UI screens). So anyone can make there own app or include some of my code in there app. Tasker does have its limitations, and they appear in the UI of this app.

All that said, I may create an android app without using tasker but it will be in the distant future, and anything I learn from this app will make the next better. I have already planned for this move by writing most settings (except user names, passwords, ect) to the internal storage so it will be avalible to a new app when it is created.

[ro.]

I had a play with the "Remove location dependency" option, but not knowing how it worked, I've now ended up with three actions all with the same name ("lights"). They all show up on the "Linked commands"/"Command associations" page, but I can't delete any of them with a long press.

The "Autowear instructions" page is blank. I presume this is for Android Wear?
"Enable/Disable autowear inergration" - spelling mistake
When selected, this goes to a page called "Autovoice intergration" - different spelling mistake, and shouldn't it say "Autowear"?

Is there any way to start the voice prompt if you set "App icon launch"/"Launch get voice from app icon" to off?

If you leave a menu page up long enough for the screen to go blank, when you wake up the screen, not only has the menu disappeared, but the whole app has vanished.