Building an open source Nest
blog.spark.io
blog.spark.io
This allows me to measure the temperature inside, outside, and get the relative humidity (not nearly as accurate as the $20 honeywell sensor that they're using, but, it's close enough for my needs). I then built a simple website using mrtg (for temperature trending) and a ruby script that checks the temperatures versus what the set points are and mounted the raspberry pi's in various locations around my places.
My "Controller" nodes are a beagleboard with a 4 or 8 channel relay board attached that allow me to turn on or off the individual controls on the furnace. It works well with my two stage heat pump and fan at my home, but, I need some work to get it 100% at the hackerspace and at the farm.
I mainly did this because I needed something that allowed me to cover more rooms than the Nest (and I'm adding duct dampers and fans to my heating system, so I can selectively heat and cool more rooms to better temperatures).
That, or a put together a nice board that you can plug into the RPi to interface better with it maybe :)
https://community.sparkdevices.com/t/burning-down-the-house/...
I never thought I'd see a good use case for auto playing videos. (It kind of reminds me of Harry Potter too)
Having said that, an h264 source can have magnitudes more complexity for playback, and generally the greater the compression the higher the playback complexity. Having five or so high profile multi-MB decorative videos autoplaying on a page is excessive.
They shouldn't use autoplay. They shouldn't even initialize until scrolled into view. They shouldn't all play at once. Their bandwidth usage is going to be enormous from this HNing, a single page impression pushing 15MB+ (EDIT: I hadn't really delved into it at the time given the display issues, however I grossly underestimated when I said 15MB).
Sadly I won't be reading any of the content because even after five minutes it's killing my browser. To the spark.io blogging team, please don't assume unlimited wads of broadband. Let the reader decide whether they want to play your videos, and you know, I can only watch one video at a time on that page, so why start them all playing at once? This is no better etiquette than CNN or MSNBC or an adware farm where they start playing videos at you upon arrival. It's very rude.
Maybe using whatever https://mediacru.sh/ returns instead will help.
https://blog.mediacru.sh/2013/10/31/Add-media-hosting-to-you...
For no real reason either time. I'm not sure what's going there.
Edit: This is with crummy AMD FirePro 2270 graphics. Chrome 34.0.1789.0 canary was very smooth on the same box!
I'm sure YMMV depending upon OS, browser, underlying installed codecs handling the video (and their associated gpu acceleration or lack thereof), etc.
Now i'm thinking about upgrading and all that... :P
$('video').each(function() { this.pause(); })
Not sure what the others guy are talking about. Works fine on my MacBook Pro...not issues at all?
As I mentioned in another post, it works fine in Chrome 34 on the same hardware that it struggles dramatically with in Chrome 32. Both of them have hardware decoding enabled, and so on. The Chrome 32 instance has no problems at all playing video anywhere else.
Indeed, on that same line, the videos don't load at all on Chrome on the Nexus 5. The N5 of course supports h264, and has no problem on any other site that I've ever discovered. Maybe their server is now overloaded (EDIT: Okay they're using S3....going to be an ugly bandwidth bill from that...and it's fully responsive), but it also doesn't load on the iPad 3rd generation running iOS 7. Again just black boxes.
EDIT: I wonder if it's because they're (s3) setting the content type to octet-stream, rather than video/mp4. Some clients seem to be going on alternate decode paths or are refusing to display it because of that.
e.g.
Content-Length:6915107 Content-Range:bytes 0-6915106/6915107 Content-Type:application/octet-stream
There's a lesson in here somewhere.
It's sad to find out that they ignored that mp4 isn't supported by all browsers. I know: HTML5 video is just a PITA, and that's indeed a reason why so many still rely on gifs.
I hope this project grows, and that they will care a bit more than they did here.
I utterly disliked it. It made my browser slow and so I didn't read their page.
Poor degradation to those of us running NoScript, there.
Opening the page via eww in EMACS made it readable since it doesn't load videos if I don't ask it to.
I always hoped they would switch back to wood, but it's incredibly hard to do right in mass manufacturing.
Any particular reason? From the outside, wood seem easier to use than plastic.
There are "manufactured" woods which address many of these issues, but they also tend not to be deemed as beautiful / strong as the real thing.
The future is not made of wood.
One of the good things about wood is you don't have to worry about it sticking around for 1000+ years. The problem with plastics is most of it can't be recycled. Not the way we think of recycling.
Aluminum cans are melted down and turned back into aluminum cans. Polymers are ground up and injection molded into lower grade polymers. Once too many polymer chains get broken the grade is too low to be usable.
The problem with plastics is its cheap acquisition costs and expensive disposal costs. One of the good things about living in a developed country is we sorta manage the disposal costs well. In developing countries they don't manage plastic disposal at all. In all my travels one of the overriding themes I've seen is plastic on the sides of road, in rivers, on the beach.
When I removed the wooden door from a house we didn't need to call the city to send a special truck to dispose of it. We chopped it into pieces and used it for bbq. Then we mixed the ashes into the compost. Disposal was $0.
While plastic is cheap in the short term I think there are a lot of long term costs that get ignored.
Great point. Simple and head-smack obvious when you read it. Hits right at the core.
Immediately after reading, I generalized to "cheap up-front, expensive back-end costs."
http://en.wikipedia.org/wiki/Pyrolysis#Plastic_waste_disposa...
http://en.wikipedia.org/wiki/Thermal_depolymerization
(Those processes mostly just mean that it all comes down to whether you want to put energy into something. Whether they are economical today doesn't factor into the usefulness in eliminating mountains of plastic.)
It's quite profitable, just the required amount of upfront investment is high.
- I just want to say one word to you. Just one word.
- Yes, sir.
- Are you listening?
- Yes, I am.
- Plastics.I'm only involved in manufacturing as a hobby for my custom keyboard projects. But my bet would be that it's about injection molding [0].
Once you got the molds machined you can mass produce plastic parts extremely cheaply. Need more plastic parts? Get a few more molds made. It scales pretty well because one mold can produce multiple parts at once and the injection molding presses itself are relatively inexpensive and pretty much fully automated.
Wood while being great for prototyping doesn't scale that well in mass production. You would machine it using CNC routers and if you need more produced you need new expensive CNC routers which can only produce one part at a time. Also the CNC routers I know you have to baby sit.
Again: That's only from my limited experience with limited runs (a few thousand pieces at most). Maybe someone who knows more about real mass production can elaborate a little more on this.
Unfortunately, with my electric heat the thermostats sit inline with the heater's power source so I need devices that can safely handle 120v.
electric heat refers specifically to heat generated by heating an element with current.
The issue is that standard programmable thermostats don't come close to the price of a Nest... about $20.
These wifi light bulbs are 3 for $199: http://store.apple.com/us/product/HA779VC/A/philips-hue-conn...
I mean, I just don't get why people are so angry that someone would possibly want to spend some extra money to have a cool thermostat. If it was a cool video card for $249 that just lets them play games, no one would blink an eye. But because it's for a house, but for a part of the house that is supposed to be utilitarian, it's a sin.
http://www.homedepot.com/p/Honeywell-Wi-Fi-Programmable-Touc...
I could buy a new fridge, a new stove, and a new water heater for that kind of money. Nice ones, too.
One problem with the Nest is that if your thermostat is in a poor location (more of a problem with central heating), the motion sensor won't see you. With the ones I suggested above, you can programmatically link those to ~$40 motion sensors installed in the correct locations that will actually work.
The financial payback time is unfortunately infinite but it was interesting.
One advantage of programming an insteon thermostat is its just plain old off the shelf technology to all end users including repair guys, although how its temp is set is magic and done by computer.
Most of the software work is already done by the misterhouse system, its not like you have to write your own insteon drivers or write your own sensor drivers or whatever. Misterhouse's support for Insteon has varied a bit over the decades. Last time I had to mess with anything it was a little fuzzy. I would imagine things have advanced considerably in the last half decade or so. I do not currently have misterhouse controlling the insteon thermostat mostly out of lazyness, I'll eventually re-enable it.
This doesn't solve your problem of the $250/room cost, but if you'd like to use some other thermostats in your rooms this would work great.
The only complaint I'd have about mine is that the relay is very loud. You might want to search for a quieter one.
http://www.aubetech.com/products/list.php?noLangue=2&noFamil...
They apparently pay attention to how loud it is.
We've moved since then and took the thermostats with us. I still have two if you want to buy some slightly used aube programmables (nothing with wifi, but better than nothing).
I can build all kinds of things with my arduino and all of those awesome little one-off function boards you can snag on ebay from china theses days. I can't build 10000 of any of them.
I missed the technical challenges - this seems trivial, and exactly how easy that I would imagine it to be. The only challenge that I see is figuring that people would want a thermostat controlled by a phone app.
Since that's been figured out, I'm going to be very surprised if within 2 years 10 vendors don't have $50 versions sold at Wal-Mart, and there aren't 2-3 different open source software stacks competing to support a few of them.
I remember when the iPod was still crazy expensive and people were abandoning their much cheaper Rios in droves.
Btw, I've purchased a few cheap home devices. For example, a few smart light switches. I've ended up switching back because they made my life more complicated and difficult, the opposite of the intended effect.
I think that the lesson from iPod example would be: Marketing matters.
Judging the quality of a tool by its price is the reason that Monster can sell $5 cables for $50.
edit: I'd be happy to hear of any improvement that iPods made to mp3 players other than making them into expensive status symbols that refuse to play anything but a proprietary vendor format that all music has to be lossily converted to by a buggy, slow, invasive application (at the dawn of iPods, not whatever condition iTunes or iPods are in currently.)
Interesting, because the first iPods I bought for my wife and I were specifically to address the fact that everything else on the market that I tried at the time was a pain in the ass to use. I found the first-gen iPod Nanos in conjunction with iTunes to be the opposite of "more complicated and difficult". "Worst of breed"? I can name a half dozen examples that were far worse than anything Apple has ever produced. Obviously YMMV.
> I think that the lesson from iPod example would be: Marketing matters.
I often read the "it's all marketing" screeds as "a lot of people like this thing, and I don't. I am bothered by this fact. Therefore the only reason the sheep buy them is marketing." Could be that the sheep truly find that it's a way to trim their wool that suits them better.
It works the same whether the product is good or terrible.
>I often read the "it's all marketing" screeds as "a lot of people like this thing, and I don't. I am bothered by this fact. Therefore the only reason the sheep buy them is marketing." Could be that the sheep truly find that it's a way to trim their wool that suits them better.
This is more a screed than anything I've said. I don't care if you use an iPod.
>"Worst of breed"? I can name a half dozen examples that were far worse than anything Apple has ever produced.
You're right - it was a bit of an exaggeration. Microsoft's "PlaysForSure" cert was nearly as bad, and all of Sony's lines were garbage (IMO.) What I mean to say is that there was never a point when you couldn't find a $50 player that offered a far better UX than an iPod.
> You're right - it was a bit of an exaggeration. Microsoft's "PlaysForSure" cert was nearly as bad, and all of Sony's lines were garbage (IMO.)
Aaaand, you hit on the top two candidates for "Why Did mikestew Buy an iPod?" MSFT's offering was broken from the start, and later yanked (buh-bye, music!), and the company that brought you the original Walkman obviously lost their magic. But of the $50 players, I never found one worth keeping. Maybe I just never learned about them, hence making your point of "marketing matters". :-)
The iPod's storage capacity was massive in comparison to nearly everything else. The firewire connection made uploading huge amounts of music crazy-fast in comparison to the old USB connections other players were using. Look at digital players of the same vintage -- stupid complicated interfaces & displays -- the clickwheel was a revolution in UX. So much so that it remains, largely unchanged, on all but touchscreen ipods.
>the clickwheel was a revolution in UX
I guess. It's pretty, at least.
>So much so that it remains, largely unchanged, on all but touchscreen ipods.
That isn't revolution, that's branding. The product had a single differentiating feature, and they've maintained it.
And iTunes was a revolution in awful, unnecessary experiences that one can have with a computer.
iPods have always had the ability to play standard MP3s. I've never converted a file to AAC for actual use.
Yes, their marketing was brilliant, but it won't cause people to buy things they don't want in perpetuity. Eventually, even the most brilliantly marketed product will fail if customers don't want it.
Those things can't be separated from the product though, which required iTunes and was expensive. Replacing the firmware on an iPod makes it perfectly fine. The reasons that the iPod was worse than many of its competitors weren't due to incompetence, they were by design.
>Yes, their marketing was brilliant
I'm not even saying that their marketing was brilliant, just pervasive. When the iPod came out, a large portion of the population wouldn't have even known and/or experienced that your music could be easily put on a tiny device at all. Their first exposure to the concept would have been an iPod ad, or seeing someone with one.
I thought the phone app would be gimmicky, but it's nice to be able to get off the plane and un-autoaway the thing so by the time you get home it's comfortable.
The second problem is that it's lacking of systematic design. Think about we are going to have some many "Intenet of the things" in our home or office, we have to download some man apps, each of them needs cloud services which usually not free. This is how they make money after the hardware sales.
- Eggs should not be washed.
These issues are definitely resolved in Nest, but not quite in Spark. I guess they are bragging.
- Eggs should not be washed.
EDIT: Yep, the Common Questions section of their website[1] says that they'll be releasing an open source version of their Cloud. Awesome.
In order to ship a widely used operating system, you need a support infrastructure, consumer research, drivers for lots of hardware, warranties, marketing, payroll, operations, accountants, regulatory compliance. The product is almost the easiest part.
I imagine that Nest understands all this. Putting a piece of hardware in someone’s house – one that’s connected to a furnace or which claims to protect against fire – means a lot of liabilities, broadly defined.
I’d love to see an open source version get to that level of maturity and support. It does happen but it takes a lot of people.
–
(Tangent, but when I started at Stack, a lot of people said they could (and did) build a clone in a weekend. Sure, as an approximation of the technical product. But that ain’t the ‘retail’ product, which is actually comprised of community, goodwill, SEO, quality control, and a lot of other things.)
The thesis of spark.io is "you can trust us with your data" not you have control of your data.
The spark is built on a cloud connected platform. even if you can see and control outputs from your board you still exist as part of their ecosystem. Which is basically the functional equivalent of using the dropbox api to build something instead of google drive.
I won't be excited about home automation until someone goes the way of an open protocol for these types of devices that doesn't require a centralized pass through.
Because if history has been any kind of teacher, it shows us that spark.io will probably get sucked up by google or somebody in the near future.
From the "Common questions" section at the bottom of https://www.spark.io/:
If you want the simplicity of the Cloud but you want it on your own server, we'll be releasing an open source version of the Cloud designed for quick and easy deployment.
(They also don't seem to restrict the device very much)
Of course, I might be wrong and had Nest actually been a Google product from day 1, then it might have been even bigger. Who knows.
http://docs.spark.io/#/examples/local-communication
Eventually they promised to open source their entire cloud stack. Eagerly awaiting when they do.
Going from a few examples that obviously cause harm, beyond dispute, I can immediately think of several that do not lend themselves to quantification.
(I hesitate to list them, lest I incur low-brow "Do you really think what google is doing is that bad?!?" comments (hint: harm, while not necessary quantifiable, is not all made equal), but if you really can't think of any examples I can list a few.)
That said, there's microcontrollers and firmware in all programable thermostats (not to mention the heat pump). Most people today already rely on third parties to heat and cool their homes.
Without that constraint, it's a much easier problem.
The control circuitry in the heating unit is sensitive to a range of voltage values. It has to be, because V=IR, and there's no way to know exactly how much wire (and thus R) will be strung through your house. The Nest can thus pull some current off the wire to charge its battery but keep the signal strong enough to flip the heater on and off.
My house has a new electronic "zone box", which, because it's all semiconductors, only needs to run a very small amount of current through the wires. That's not enough for the Nest to charge. Fortunately, the standard home heating wiring includes a "common" wire, with a low amount of constant voltage suitable for devices like the Nest.
Yep, the Common Questions section of their website[1] says that they'll be releasing an open source version of their Cloud. Awesome. [1] https://www.spark.io/
- Eggs should not be washed.
"At Spark, we’re making it easier to bring connected
devices to market with the Spark Core, our Wi-Fi
development kit, and the Spark Cloud, our cloud service for
connected devices."
It would be nice if you had the option to host your own cloud service. You could protect your business model at least partially by using an open source license that requires people to change the name if they decide to fork it and productize it, such as the Artistic License v2.
The Spark Thermostat is great minus the fact that you need their web api for communicating with it. But for a 1-day build, how can anyone disappointed! Great job Spark team.
In regards to my own devices, I am definitely going to have to take a look at Spark now. Cool hardware.
They don't provide any specifics in the docs either, only this:
"Security is hard. It’s especially hard on an embedded system, because encryption is resource intensive. But it’s also important, because you don’t want anyone turning on and off your lights, or worse, locking and unlocking your front doors.We hand-picked a set of rock-solid security protocols that are secure and efficient, so they work great on an embedded system. They’re baked into the Spark Protocol, which is open source and ready to be extended to other products."
I get that encryption may be difficult on embedded systems, but I would also argue that if a small embedded system can't handle strong encryption then it's not ready to connect devices to the web. I can't find any links to source code - anyone know what sort of encryption they use?
Return message from cloud is decrypted with an RSA private key on the Core: https://github.com/spark/core-communication-lib/blob/master/...
An HMAC signature is verified: https://github.com/spark/core-communication-lib/blob/master/...
If everything checks out, AES-128-CBC session key is saved and IV is rotated with every message exchanged:
encrypt: https://github.com/spark/core-communication-lib/blob/master/...
decrypt: https://github.com/spark/core-communication-lib/blob/master/...
Cheers, Zachary Crockett Spark CTO ;)
I actually found your GitHub profile and should have realized that core-communication-lib is what I wanted, but it appears new and doesn't have many stars yet, so I missed it.
Nobody has ever offered this in a thermostat with the possible exception of the RadioThermostat CT30/3M50, which has a JSON API and lets you manually control the relays.
As I understand it any temperature control designed for big refrigerators or freezers has this features. Some even allow you to pre-configure fan cycles (i.e. on for x minutes, off for y, repeat) and turn the cycle on or off depending on temperature or compressor state.
(not including the Spark Core at $39)
Also if you're in the bay area, you should check out this meetup group run by my friend Nick Pinkston: http://www.meetup.com/HardwareStartupSF/
I've now extended the idea into a fully working DIY DJ Controller. My first big electronics project... I've been planning to open-source the build documents for quite a while now. : http://www.youtube.com/watch?v=BFhLQzisx90
The software is partly open sourced.
I think for something to truly be open-source and beneficial for everyone, everything about it must be open, including the data. The data from all the connected devices globally could be stored on an open database that anyone can access and use. Its one thing to 'learn' with the limited data that one device might generate, but for a machine to 'learn to learn' it should be able to study ALL the data that might be useful.
This kind of organization could lead to a type of opensource corporation where anyone can be an 'employee'. Employment and compensation could be based off a public list of contributions to the project. To each according to his contribution.
This idea could be applied to anything that's used in public and generates data. Autonomous cars, home automation, drones, (NSA data, slightly more complicated but still could be open sourced). But as long as we're tricking ourselves into thinking we need 'money' to survive, the organization or company with the most of it wins.
There are lots of microcontroller-that-you-program-via-usb communities. Arduino is a good search word.
Also, this discussion:
https://news.ycombinator.com/item?id=7071080
included this tumblr:
I built out my XMPP-controlled office thermostat on a BeagleBone Black, which costs a bit more than the Pi but has way more pins, including analog inputs, which are pretty important for temperature sensors. Plus using a Linux machine meant being able to choose your language instead of being stuck with C, so I took the chance to learn some Erlang: http://technomancy.us/171
The $40 wifi-included board featured in this story sort of has a nice installation story though (maybe fussy setup, but one less cable).
A web connected[1] ultrasonic sensor unit, that you put[3] on the side of your car , and while driving, it recognizes[2] empty parking spaces and sends the location[4] to a cosm.com map.
[1] through your phone hotspot.
[2] since an ultrasonic sensor senses distance, it can sense if you have a car near you or an empty space.
[3] Maybe with vacuum stick units.
[4] You'll need a gsm shield for that.
[1] http://www.theconnectivist.com/2013/07/how-google-tracks-tra...
Servo big enough to close central heating valve (the big knob next to your heater).
Automatic light switches when you leave house or going to sleep.
Smart monitors, esp. water leak alert when you are not home.
I think a really great improvement would be GSM module, delivering connection over Whispernet (aka hassle free).
Is it possible to run your software on other dev boards?
Does it have enough processing power for HTTPS POST? I see someone complain here: https://community.sparkdevices.com/t/how-to-send-http-post-r...
https://community.sparkdevices.com/t/burning-down-the-house/...
(Sure, it would have to have most of the functionality of a thermostat, but so what.)
If your Internet/wifi is spotty (very common)... Does your thermostat not work? Are you stuck being hot or cold?
After all, one wouldn't want to have to charge their smoke detector every day like a smartphone, right? (Not once a month either).
Edit: It's doing the same in Chrome too
Looking back to 5-10 years ago you would have had a really hard time building this in a week let alone 1 day.
Also the firmware definition bugs me.
neat product/concept tho.
But for me, it's the tinkerer inside. I would rather build one of these than buy a nest.
http://i.imgur.com/8fFkdQ9.jpg
In that dashboard of mine, you see devices from 6 different companies that I've integrated. 3 of them required reverse engineering some kind of website for remote control, even though all that website does is pass on a command from my home network back to my home network.
This would break if the website goes down, the website rate limits me, the website blocks my IP, or the website owner decides to sue me under CFAA because their TOS probably didn't authorize remote control outside of their official apps. Nest is one of those websites. There's no official public API.
I consider alternatives where I could just talk to the device directly a selling point, and it has nothing to do with privacy. I don't care about Google having my temperature history at all.
Seriously, the case was possibly the least important part of this and they had no troubles manufacturing it.