Reverse engineering the Nest home/away API | Hacker News Reader

Reverse engineering the Nest home/away API

blog.emilburzo.com

80 pointsemilburzo20 comments

Commercial 'Smart Thermostats' bug the hell out of me.

A Thermostat is a temperature sensor, and a relay. Anyone who has a little bit of tinkering experience could probably build a smart thermostat that doesn't need the cloud and is totally private.

However, accepting that most people can't be bother to DIY, or even have the needed skills, I'm surprised there aren't 'Open Hardware' Thermostats available that are just as easy to fit?

I've not Googled, but does anyone know of any non-DIY 'open' alternatives ?

There are zigbee & zwave based platform which would tick this box. But I’m not aware of any from major brands in the West. But vesternet is one to look at.

This is probably the path of least resistance to get what it sounds like you want. I use a GoControl Z-Wave Thermostat and home assistant.

The gron tool mentioned by the author is incredibly useful for grepping json.

https://github.com/tomnomnom/gron

It’s great for flattening it too, so you can figure out the path to specific content in those larger gnarly docs.

So the author used someones else library https://github.com/derek-miller/nest-protobuf to "reverse engineer" the nest API, packaged it up and made a blog post...

That's not a library. It's just a tool to extract the protobuffer schema. It obviously saves doing 99% of the work. Once you get to this point, reverse engineering a gRPC API is easier than reverse engineering a JSON REST api.

I might have to look into that method for home vs away. Using Home Assistant and the HA app on my wife and I's phones solely for presence detection has been kind of a hassle.

What about just using the nmap homeassistant plugin, which doesn't require any kind of client device setup? Assuming you use wifi at home, and your phones automatically connect, and your wifi isn't too spotty.

Can also use a router integration. I have a mikrotik and it tracks DHCP clients which I use for presence detection using a bayesian sensor

Yup, I have HA setup with Unifi integration and it works perfectly for detecting if wifi clients. It gets alerted immediately upon connection (so I don't think it's polling the controller), and they get marked as away exactly 2 minutes after wifi disconnect.

If you want to go the other way, and have the phones ping the network instead of the other way around, there are applications that regularly publish to message queues. The Android app Presence Publisher [1] for example publishes to MQTT.

[1]: https://f-droid.org/en/packages/org.ostrya.presencepublisher...

Weird, I was wondering about this literally this afternoon! Nice write-up, and thanks for packaging it up nicely, hope to use it soon myself too.

Though I've been trying to think about being accepting of IPv6 and shunning static IPs (and DHCP) recently, maybe something can be done with MACs... Perhaps even on the broadcast when the device tries to join the network and self-assign an IP, I don't know a lot about it though.

Why use static ips? Wouldn't an ARP packet suffice? It doesn't require setting up static IPs, and also avoids the problem of "stealthed" ports, if android/ios decides to implement that sort of thing.

How would you use ARP here? ARP lets you find a MAC address assuming that you already know the IP address that you want to talk to. But without static IPs you wouldn't know the IP address of the phone that you're interested in. I'm not seeing how ARP will help here.

you don't need to know the IP address , you can just dump the ARP cache and look for the MAC address you need.

on Linux:

  $ arp -n | grep "MAC_ADDRESS_HERE"
  192.168.86.23            ether   MAC_ADDRESS_HERE   C   wlp4s0

if it's in the ARP cache it's "home" (or rather was, within the last 60 seconds, or gc_stale_time)

Convenience, mostly.

An ARP packet wouldn't work in the default Docker network configuration.

You would have to give up network isolation and use "host networking" (which doesn't work on the Mac/Windows docker version).

Did you look at possibly reversing the payload from the API into the device (vs. from the webapp into the API), and if so did it seem tractable?

Nice write up thank you for sharing.

Do you mean reverse engineering the API response from the "set home/away" command?

If yes, then I sort of[1] do this for the "status" part of the response, so I can give useful error messages back (e.g. wrong "structure id")

[1] https://github.com/emilburzo/nest-rest/blob/master/src/main/...

Clever use of static IPs, loved the write up!

Reply on news.ycombinator.com