NEST Account migration

[Filip]

Hey,
Due to a new phone and re-installation of the NEST App, my account is migrated to Google.
And - of course - the NEST plugin ("NEST Thermostat/Protect") in Domoticz is no longer working (login problems).
I wanted to try the "NEST Thermostat/Protect OAuth" to bypass the problem, but unfortunately it seems that it is not possible anymore to create and API Key.
Does anyone found a solution?
Thanks for your help!
Filip

[dendoba]

Anyone? I'm also interested before i migrate!

[Lasermi]

Basically don't. There is another thread on the topic with not much progress either.

https://r.tapatalk.com/shareLink/topic? ... source=app

Unless Google provides some kind of API... there's little progress that can be done. Did anybody find documentation??

Enviado desde mi SM-G960F mediante Tapatalk

[Filip]

Finally I did some development work and created a new Python plugin for the NEST Thermostat and Protect that works with the Google Account. It is based on PHP code from https://github.com/gboudreau/nest-api.
It should detect all your devices. It works in my environment (1 thermostat and 1 protect).

However I have one request for help. I work with threading because couldn't get it working with the plugin connection system.
Normally I check at "onStop" if the threads are not running anymore before stopping. I followed the recommendations on https://github.com/domoticz/domoticz/bl ... hreaded.py. With the debug information, it shows that there are no threads running anymore. However, when "updating" the device in the "hardware" tab, it leads to problems with the plugin system and a reboot is required. So hope somebody can help me on this!

The plugin can be found here: https://github.com/FilipDem/Domoticz-NE ... ree/master

It is a workaround to get to the Google Account credentials (issue_token and cookies), but this can be done on the following way.

• Open a Chrome browser tab in Incognito Mode (or clear your cache).

• Open Developer Tools (View/Developer/Developer Tools).

• Click on Network tab. Make sure Preserve Log is checked.

• In the Filter box, enter issueToken

• Go to https://home.nest.com, and click Sign in with Google. Log into your account.

• One network call (beginning with iframerpc) will appear in the Dev Tools window. Click on it.

• In the Headers tab, under General, copy the entire Request URL (beginning with https://accounts.google.com, ending with nest.com). This is your issue_token.

• In the Filter box, enter oauth2/iframe

• Several network calls will appear in the Dev Tools window. Click on the last iframe call.

• In the Headers tab, under Request Headers, copy the entire cookie value (include the whole string which is several lines long and has many field/value pairs - do not include the Cookie: prefix). This is your cookies; make sure all of it is on a single line.

[Devious]

I'm a new NEST user. So I started out with a Google account.

I installed the new plugin but there are no devices showing up. Am I forgetting something?

Log showes:

2019-11-30 17:34:29.633 Status: (Google Nest) Stop directive received.
2019-11-30 17:34:29.633 Status: (Google Nest) Exiting work loop.
2019-11-30 17:34:29.834 Status: (Google Nest) Stopping threads.
2019-11-30 17:34:29.834 Status: (Google Nest) Stopped.
2019-11-30 22:11:30.681 Status: (Google Nest) Started.
2019-11-30 22:11:31.639 Status: (Google Nest) Entering work loop.
2019-11-30 22:11:31.640 Status: (Google Nest) Initialized version 1.0.0, author 'Filip Demaertelaere'

[Filip]

See also the problem with the threading mentioned in the text.
Normally this does not have any impact, unless you do an "update" of the hardware, eg, if you added the issue_token and cookie. Then it is better to reboot and normally devices should be added (anyway, in my case it works very well).
You can also activate the debug information to have more information in the logging (but again reboot is required when doing changes).

I know that the "reboot" is not very nice, but did not find a solution for it (I followed the guidance for threading). I opened a topic dedicated to the threading problem.

Hope it helps,

Filip

[Devious]

Rebooting doesn't help, and using cookies from other iframe calls gives the same result.
So I guess the problem is in the issue_ token and cookie part.

Could google 2 way authentication be a problem?

[dressie]

Tried it, doesn't work on this side. Did the reboot, but it's not creating devices.
I do believe this has potential, so hopefully we can get this to work. Just got my Nest V3, but it's not really working well with my OTGW.

Edit: I have noticed that after a reboot the hardware disappears.

[Filip]

No, i have also the two-step authentication activated.
Did you activate the debug mode?

[Devious]

yes i dit, this is whats logged:

2019-12-01 21:00:22.236 Status: (Google Nest) Started.
2019-12-01 21:00:23.265 (Google Nest) Debug logging mask set to: PYTHON PLUGIN QUEUE IMAGE DEVICE CONNECTION MESSAGE ALL
2019-12-01 21:00:23.266 (Google Nest) Images created.
2019-12-01 21:00:23.266 (Google Nest) Start thread
2019-12-01 21:00:23.263 Status: (Google Nest) Entering work loop.
2019-12-01 21:00:23.263 Status: (Google Nest) Initialized version 1.0.0, author 'Filip Demaertelaere'
2019-12-01 21:00:33.265 (Google Nest) Pushing 'onHeartbeatCallback' on to queue
2019-12-01 21:00:33.286 (Google Nest) Processing 'onHeartbeatCallback' message
2019-12-01 21:00:33.286 (Google Nest) Calling message handler 'onHeartbeat'.
2019-12-01 21:00:33.286 (Google Nest) onHeartbeat called
2019-12-01 21:00:33.287 (Google Nest) Start thread
2019-12-01 21:00:43.267 (Google Nest) Pushing 'onHeartbeatCallback' on to queue
2019-12-01 21:00:43.315 (Google Nest) Processing 'onHeartbeatCallback' message
2019-12-01 21:00:43.315 (Google Nest) Calling message handler 'onHeartbeat'.
2019-12-01 21:00:43.315 (Google Nest) onHeartbeat called
2019-12-01 21:00:43.315 (Google Nest) onHeartbeat called, run again in 29 heartbeats.
2019-12-01 21:00:53.268 (Google Nest) Pushing 'onHeartbeatCallback' on to queue
2019-12-01 21:00:53.284 (Google Nest) Processing 'onHeartbeatCallback' message
2019-12-01 21:00:53.284 (Google Nest) Calling message handler 'onHeartbeat'.
2019-12-01 21:00:53.284 (Google Nest) onHeartbeat called
2019-12-01 21:00:53.284 (Google Nest) onHeartbeat called, run again in 28 heartbeats.
...
2019-12-01 21:05:23.317 (Google Nest) Pushing 'onHeartbeatCallback' on to queue
2019-12-01 21:05:23.364 (Google Nest) Processing 'onHeartbeatCallback' message
2019-12-01 21:05:23.365 (Google Nest) Calling message handler 'onHeartbeat'.
2019-12-01 21:05:23.365 (Google Nest) onHeartbeat called
2019-12-01 21:05:23.365 (Google Nest) onHeartbeat called, run again in 1 heartbeats.
2019-12-01 21:05:33.318 (Google Nest) Pushing 'onHeartbeatCallback' on to queue
2019-12-01 21:05:33.333 (Google Nest) Processing 'onHeartbeatCallback' message
2019-12-01 21:05:33.333 (Google Nest) Calling message handler 'onHeartbeat'.
2019-12-01 21:05:33.334 (Google Nest) onHeartbeat called
2019-12-01 21:05:33.335 (Google Nest) Start thread
2019-12-01 21:05:43.320 (Google Nest) Pushing 'onHeartbeatCallback' on to queue
2019-12-01 21:05:43.362 (Google Nest) Processing 'onHeartbeatCallback' message
2019-12-01 21:05:43.362 (Google Nest) Calling message handler 'onHeartbeat'.
2019-12-01 21:05:43.362 (Google Nest) onHeartbeat called
2019-12-01 21:05:43.362 (Google Nest) onHeartbeat called, run again in 29 heartbeats.
and so on.

[Filip]

I agree. Together we can come to a workaround solution for many people...
I would advise you to start with a test outside Domoticz.

Change the file nest.py by copying the "issue_token" and "cookie". Just to repeat: follow well the guidance in the "how-to" what exactly to copy.
Run then the script with python 3: python3 nest.py.

This should normally output different data obout your devices. I suppose you have a nest thermostat?

Code: Select all

if __name__ == "__main__":

    issue_token = 'xxxxxxxxxxxx'
    cookie = 'xxxxxxxxxxxxxx'
    thermostat = Nest(issue_token, cookie)
    while True:
        thermostat.GetNestCredentials()
        thermostat.GetDevicesAndStatus()
        for device in thermostat.device_list:
            print(thermostat.GetDeviceInformation(device))
        for device in thermostat.protect_list:
            print(thermostat.GetProtectInformation(device))

[dressie]

I followed the instructions to a tee. I did correct the Cookie in the nest.py file and did a reboot. Only output I get is:

2019-12-02 13:14:20.684 (Nest) Pushing 'onHeartbeatCallback' on to queue
2019-12-02 13:14:20.729 (Nest) Processing 'onHeartbeatCallback' message
2019-12-02 13:14:20.730 (Nest) Calling message handler 'onHeartbeat'.
2019-12-02 13:14:20.730 (Nest) onHeartbeat called
2019-12-02 13:14:20.730 (Nest) onHeartbeat called, run again in 21 heartbeats.
2019-12-02 13:14:30.686 (Nest) Pushing 'onHeartbeatCallback' on to queue
2019-12-02 13:14:30.701 (Nest) Processing 'onHeartbeatCallback' message
2019-12-02 13:14:30.702 (Nest) Calling message handler 'onHeartbeat'.
2019-12-02 13:14:30.702 (Nest) onHeartbeat called
2019-12-02 13:14:30.702 (Nest) onHeartbeat called, run again in 20 heartbeats.
2019-12-02 13:14:40.688 (Nest) Pushing 'onHeartbeatCallback' on to queue
2019-12-02 13:14:40.725 (Nest) Processing 'onHeartbeatCallback' message
2019-12-02 13:14:40.730 (Nest) Calling message handler 'onHeartbeat'.
2019-12-02 13:14:40.730 (Nest) onHeartbeat called

[capman]

After execute the command 'python 3 nest.py' , I get following output:

Code: Select all

pi@Keuken:~ $ python3 nest.py
{'Temperature_scale': 'C', 'Where': 'Living Room', 'Heating': False, 'Eco': True            , 'Name': 'Dorp 22/206, Oosterzele', 'Target_temperature': 10.0, 'Current_temper            ature': 13.73999, 'Away': True, 'Humidity': 44}
{'Where': 'Kitchen', 'Serial_number': '06AA01AC301605BL', 'Smoke_status': 0, 'Co            _previous_peak': '0'}
{'Temperature_scale': 'C', 'Where': 'Living Room', 'Heating': False, 'Eco': True            , 'Name': 'Dorp 22/206, Oosterzele', 'Target_temperature': 10.0, 'Current_temper            ature': 13.73999, 'Away': True, 'Humidity': 44}
{'Where': 'Kitchen', 'Serial_number': '06AA01AC301605BL', 'Smoke_status': 0, 'Co            _previous_peak': '0'}
{'Temperature_scale': 'C', 'Where': 'Living Room', 'Heating': False, 'Eco': True            , 'Name': 'Dorp 22/206, Oosterzele', 'Target_temperature': 10.0, 'Current_temper            ature': 13.73999, 'Away': True, 'Humidity': 44}
{'Where': 'Kitchen', 'Serial_number': '06AA01AC301605BL', 'Smoke_status': 0, 'Co            _previous_peak': '0'}

But no extra devices are created in domoticz

[dressie]

I followed the instructions to a tee. I did correct the Cookie in the nest.py file and did a reboot. Only output I get is:

2019-12-02 13:14:20.684 (Nest) Pushing 'onHeartbeatCallback' on to queue
2019-12-02 13:14:20.729 (Nest) Processing 'onHeartbeatCallback' message
2019-12-02 13:14:20.730 (Nest) Calling message handler 'onHeartbeat'.
2019-12-02 13:14:20.730 (Nest) onHeartbeat called
2019-12-02 13:14:20.730 (Nest) onHeartbeat called, run again in 21 heartbeats.
2019-12-02 13:14:30.686 (Nest) Pushing 'onHeartbeatCallback' on to queue
2019-12-02 13:14:30.701 (Nest) Processing 'onHeartbeatCallback' message
2019-12-02 13:14:30.702 (Nest) Calling message handler 'onHeartbeat'.
2019-12-02 13:14:30.702 (Nest) onHeartbeat called
2019-12-02 13:14:30.702 (Nest) onHeartbeat called, run again in 20 heartbeats.
2019-12-02 13:14:40.688 (Nest) Pushing 'onHeartbeatCallback' on to queue
2019-12-02 13:14:40.725 (Nest) Processing 'onHeartbeatCallback' message
2019-12-02 13:14:40.730 (Nest) Calling message handler 'onHeartbeat'.
2019-12-02 13:14:40.730 (Nest) onHeartbeat called

Outside Domoticz I get this:

pi@DomoticzV2:~/domoticz/plugins/GoogleNest $ python3 nest.py
Traceback (most recent call last):
File "nest.py", line 227, in <module>
thermostat.GetNestCredentials()
File "nest.py", line 146, in GetNestCredentials
self._GetBearerTokenUsingGoogleCookiesIssue_token()
File "nest.py", line 101, in _GetBearerTokenUsingGoogleCookiesIssue_token
self.access_token = result['access_token']
KeyError: 'access_token'

Looks like I'm missing something?

[Filip]

I see already that you didn't included YOUR cookie/issue_token in the nest.py file...
Update the nest.py file with your issue_token/cookie (replace the one in the file by yours or download again the file from this forum above and change the xxxxxxxxxxx by your values). This part of the code can be found at the bottom of the nest.py. You can find the extract just below...

I forgot also to change my issue_token/cookie in the file so I will change this evening my login .

Code: Select all

if __name__ == "__main__":

    issue_token = '[u][b]xxxxxxxxxxxx[/b][/u]'
    cookie = '[u][b]xxxxxxxxxxxxxx[/b][/u]'
    thermostat = Nest(issue_token, cookie)
    while True:
        thermostat.GetNestCredentials()
        thermostat.GetDevicesAndStatus()
        for device in thermostat.device_list:
            print(thermostat.GetDeviceInformation(device))
        for device in thermostat.protect_list:
            print(thermostat.GetProtectInformation(device))

When using the Domoticz plugin, did you add YOUR issue_token and cookie in the hardware settings? I remind to the procedure how to obtain the issue_token/cookie:

• It is a workaround to get to the Google Account credentials (issue_token and cookies), but this can be done on the following way.

• Open a Chrome browser tab in Incognito Mode (or clear your cache).

• Open Developer Tools (View/Developer/Developer Tools).

• Click on Network tab. Make sure Preserve Log is checked.

• In the Filter box, enter issueToken

• Go to https://home.nest.com, and click Sign in with Google. Log into your account.

• One network call (beginning with iframerpc) will appear in the Dev Tools window. Click on it.

• In the Headers tab, under General, copy the entire Request URL (beginning with https://accounts.google.com, ending with nest.com). This is your issue_token.

• In the Filter box, enter oauth2/iframe

• Several network calls will appear in the Dev Tools window. Click on the last iframe call.

• In the Headers tab, under Request Headers, copy the entire cookie value (include the whole string which is several lines long and has many
  field/value pairs - do not include the Cookie: prefix). This is your cookies; make sure all of it is on a single line.

[dressie]

I did replace the 'XXXXXX' parts for the issue_token and cookie according the clear instruction.
The issue_token starts with 'https://accounts.google.com/o/oauth2/.......'
The cookie starts with 'NID=192=..........'

[Devious]

downloaded the zip file again en replaced the XXX with my token and cookie.

ran outside of domoticz:

pi@raspberrypi:~/domoticz/plugins/GoogleNest $ python3 nest.py
Traceback (most recent call last):
File "nest.py", line 228, in <module>
thermostat.GetDevicesAndStatus()
File "nest.py", line 170, in GetDevicesAndStatus
for protect in self.status['topaz']:
KeyError: 'topaz'

[dressie]

downloaded the zip file again en replaced the XXX with my token and cookie.

ran outside of domoticz:

pi@raspberrypi:~/domoticz/plugins/GoogleNest $ python3 nest.py
Traceback (most recent call last):
File "nest.py", line 228, in <module>
thermostat.GetDevicesAndStatus()
File "nest.py", line 170, in GetDevicesAndStatus
for protect in self.status['topaz']:
KeyError: 'topaz'

I have exactly the same outcome.

[Filip]

Do you have a nest protect (smoke alarm from nest)? Or only a thermostat?
I suppose not and this is then the problem. If this analysis is correct, delete than the following 2 lines in the function "GetDevicesAndStatus" and try again:

Code: Select all

for protect in self.status['topaz']:
    self.protect_list.append(str(protect))

I suppose this should solve your problem. Hopefully no side-effects because I cannot test it...
Please give feedback and if it works, I will adapt the script so that it take into account this scenario.

Filip

[dressie]

Do you have a nest protect (smoke alarm from nest)? Or only a thermostat?
I suppose not and this is then the problem. If this analysis is correct, delete than the following 2 lines in the function "GetDevicesAndStatus" and try again:

Code: Select all

for protect in self.status['topaz']:
    self.protect_list.append(str(protect))

I suppose this should solve your problem. Hopefully no side-effects because I cannot test it...
Please give feedback and if it works, I will adapt the script so that it take into account this scenario.

Filip

Yes! That did it! It works outside Domoticz right now, so time to see what happens in Domoticz!