The day my smart vacuum turned against me
codetiger.github.io
codetiger.github.io
1. Has the technical skills to disassemble this device, trace circuit boards, design his own boards and custom software to interface with components to substantially reverse engineer this device.
2. Is totally mystified when his internet connected device stops working after he blocks its communication, and rather than try unblocking it and seeing if it works again, sends it out for repair repeatedly.
Something here doesn't add up. Tastes like bullshit to me.
It sure sounds like they were aware of the relation, just not how or why one thing led to the other.
I've done restrictive or invasive things to a variety of devices I own. But if something isn't working the way it should, "reset back to a clean default state and test again" always comes before trying to engage a warranty service process.
The device might not be designed with a publicized tool to restore to a "clean state", and there's also the business signaling factor of "the device stopped working, so I will make sure sure it costs them money to handle a warranty claim".
Were they even able to see what was inside the traffic they blocked? Or are they just assuming it’s telemetry?
Also the very frequent use of `—` gives me ChatGPT vibes, but may just be for editing or a personal style. Still enjoyed reading it.
Whether or not the author used AI to write it, it’s a valid criticism that it sounds like it, it makes people not want to read it and the author should consider a less offputting style if they want more engagement.
Edit to add, it’s worth flagging AI articles if you don’t want to see them, just commenting on it ends up making for a poor discussion - this thread is littered with talk about how it’s AI written. Better just to vote for it to flagged/dead.
I think, charitably, what might have happened here is:
1. Author left out the diagnostic step that restoring the connectivity didn't fix the robot, because it didn't work.
2. Author did the technical analysis mentioned (there is a repository attached. I haven't verified that it actually has the level of technical analysis indicated).
3. Author took some creative liberties (possibly involving some AI-assisted punching-up) when writing the blog post story to make it more compelling in a way that made it feel a bit off and left me and others questioning its veracity.
Skimming the post and skipping past the fluff, I thought it was an interesting situation and bit of debugging.
The post really makes it unclear how permanent the disablement was, or how exactly "one script had been modified to prevent the main application from launching". Would really love to see some details here. Could author undo that change? Did they try to?
Perhaps so, but it's easily confirmed by another owner of said device going through the same or similar procedure.
If there's any truth to this then everyone should know about it. (Frankly, from what I've seen lately I'd not be a bit surprised that manufacturers would stoop so low).
That said, given the comparatively small number of readers here on HN the chances of another experiencing the same issue would be small. As the story will have alerted everyone the truth will eventually out itself.
Does it really? In my opinion, if it stops working and it's under warranty, why not send it out for repair? They did no changes to the actual device, and apparently it was working fine for a few days without network connection, so if it suddenly stops working and it's under warranty that's the manufacturer's/store's problem, not theirs. Trying to fix it/reverse engineer it takes time, and I can see someone with these kinds of skills wanting to spend it on something else than trying to figure out how the manufacturer bricked their vacuum.
In addition, _someone_ is paying for the repairs under warranty, so if enough people were to do it, hopefully it would disincentivize completely blocking devices just because they can't reach a server.
https://valetudo.cloud/pages/general/supported-robots.html#i...
> Cloud replacement for vacuum robots enabling local-only operation
Middle would imply there being another end still.
This is a distinction that is worth making because the robot is still running and relying on all of the on-robot proprietary code; it's just the in-cloud code that has been replaced.
I think it's also not quite correct to say the low-level firmware is unmodified, because with vale tudo you rely on the project author to provide a minimal rootkit that gets customized on a per-serial-number basis for the initial rooting.
from a high-level though, it delivers what it says on the tin - cloud features without any requirement of packets leaving your network or even the robot itself.
here's a talk from the author discussing his research https://www.youtube.com/watch?v=AfMfYOUYZvc
Why on earth would you do it?! It was literally the comment I visit HN for - a solution for the problem we basically all have, already tested by someone, starting a thread with people's experience of it.
It's pretty amazing. Valetudo is perhaps the most opinionated software I've ever used, which comes with the good and the bad. But overall, it works and it does what it says it will do.
I don't really need to access it remotely, though it has come in handy: when heading out of town I can turn off the scheduled cleans and just run it once on the day I'm returning home. Which is really the only functionality you would need the manufacturer-provided cloud connectivity for.
It's been fun explaining to people that it's "declouded", but I can access it from anywhere. Melts non-tech peoples' brains a little bit.
Unless it's Saturday Night and you're drunk. Then go for Demon Seed.
Don't give Svengoolie any ideas!
Alas, lately it's mostly been 60's and earlier, when at one point it was a lot of 70's-90's stuff.
Sven is one of the few things I miss about living in Chicago.
When I am away from home for extended periods, I like to watch Sven to get a taste of home. It's especially nice for after I return from travel and want to unwind.
PSA: There's full broadcasts of Svengoolie up on Archive.org! Some uploaded by the man himself! ETA: A lot more than I realized; the last ~year some fellow has been recording and uploading them regularly.
Former Illinoisan, and happy that Sven is national on the MeTV network.
I've heard he's now in commercials with Howard Ankin, which seems strange since he gets thrown out of the back of the hearse in the middle of each show!
With ARM Memory Tagging Extension becoming common on phones now it's getting borderline impossible to hack them.
I expect there are oodles of sci-fi references to this already, but in particular I remember an audio-log from the game Horizon: Forbidden West, where a hacker is critiquing some shady corporate research he found, implying a project to weaponize domestic robots:
> There's the Moldova brain-hack of course, but also up and coming little devils like the know-it-all MEMEr, or my personal favorite, Sovereign 7482.
> Now that's an apex predator. Assuming control of them Ti-D-O bots and arming 'em with household appliances? Imagine tidying up after that! Gotta admit, it'd be fun to see 'em hunt in the wild.
Given the complete lack of relevant technical details, it could be something as simple as "internal log storage full, refusing to start up until logs uploaded". We'd never know.
Someone (or a company) does something bad - yes, pay it back, but there needs to be some punishment for doing evil. Pay back X 100. Or pay purchaser + pay fine.
Just paying back the cost (or fraud) is saying "it's fine if you don't get caught, and if you do get caught, there's no real cost to you."
1. Like a few comments mentioned here, the remote event happens for multiple reasons. Especially triggered by their app when you start or stop cleaning etc. I missed mentioning it in the article. The one I showed was suspicious as the device got bricked just after that event.
2. I fixed it by reseting the firmware and it worked for 2 days now without any issues. After second day cleaning it went back to charge as usual and never turned on. The bricking happened again this time, and I see a similar remote event again. This time am 100% sure there was no action from my side on the App or remote control.
3. What I found so far, after cleaning completes, the device uploads the map (in PNG) and some more data (in binary) to their server (There are very clear logs for this). After this upload, it receives a remote event and stops working.
4. How did I fix the device? I've the backup of all files in the device. Rebooting the device after replacing the files do the trick.
5. Now it reports the device is not on flat surface. Probably a loose connection with sensors which am yet to figure out. This time I suspect the bricking technique has changed, we shall be updating after more research
Note: I am not sure if I can publicly share all techniques used to get access to the device. But it is very straightforward as mentioned in the article and very easy if you have some knowledge on adb (Android Debug Bridge) tools and a USB to micro usb wire.
Sadly, because of (2), most (all?) companies don't bother with local connectivity at all. Much easier to debug one codepath (via remote server) rather than two (remote server and direct connection).
So yeah, if you are worried about device being remote controlled by its manufacturer, don't buy devices which say "Can be remote controlled" right on the box. But of course then you are back to ancient tech, setting physical virtual wall devices or bounding the clean area with overturned chairs.
This is when you only want to clean only left half of the room, for example. In the old robotics vacuums, your choices were either enough physical "virtual wall" units (those little stubby boxes you place on the floor) or physically fencing it, with overturned chairs or boxes or something.
With the smart vacuums, you get the nifty map and you can draw the lines/boxes around the area you want cleaned. Very convenient, but naturally requires some sort of device with a screen (to show map) and position input (to draw the boundaries), like a phone.
Why couldn't that just be over Bluetooth?
I recently moved into a new home and decided to take the opportunity to replace everything; it’s been surprising how many things are just coming to life. TVs, vacuums, kitchen appliances, etc. Some of my new TVs won’t even let me use the microphone on the remote until I give it my WiFi password. It’s quite ridiculous the world we’re creating for ourselves.
What brand?
Do you have any specific examples of a product doing this?
Couple days later, we had moved belongings and setup an old dumb TV for my kid. I was a bit surprised when then Apple TV had the WiFi credentials but I was busy moving and just shrugged it off.
Couple days later, I replaced the old dumb TV with a new Samsung. It had connectivity too, without doing anything specific. This isn’t my first smart TV but I never give any TV my WiFi and instead always use the Apple TV devices. So was a little more shocked on this one but thought I’ll figure it out when I setup my mesh.
Over the next few days, the JennAir appliances showed signs of connectivity. I wasn’t sure how. Hadn’t intended on giving them any Internet access. Don’t really care about having a smart oven.
About 2 weeks later, I get around to setting up the mesh. Start going device by device and having it forget the ATT routers Wi-Fi. None of the devices know my mesh info, so set them up individually. I need to just disable the ATT routers Wi-Fi network, haven’t yet.
I’m not really a networking guru. I just never had this happen before. I think I either accidentally allowed my phone to share the information or it’s possible my wife did. I had never used QR to login, so maybe that exposed something.
Sorry, what? That's not a documented thing.
There are limited ways in which you can share credentials (like between iPhones), and there have been some weird things in the past related to sharing with friends' accounts (the controversial Windows 10 Wi-Fi Sense in 2015), but your consumer devices aren't just sharing Wi-Fi credentials with each other. There isn't even a protocol for that kind of automatic discovery.
As for the microphone on the remote, you need to enable Wi-Fi on the TV so it can perform voice recognition on remote servers. The TV doesn't have the chips/software to do voice recognition. So that makes complete sense.
Also, pretty sure it’s undocumented, but I’ve been on a handful of reddits and HN conversations in the past where people are all experiencing this. I think a simplified theory is that they broadcast the WiFi credentials over Bluetooth and other unconnected devices can then get online. I remember one discussion of this theory where the guy found out his TV had connected to his neighbors WiFi somehow and this was a theory in the thread. It’s foil hat type stuff but very believable if you’ve experienced having devices know your WiFi despite having never granting it.
Another theory is similar where the unconnected devices send the connected device data to relay. This would allow the unconnected device to remain stealth, as it wouldn’t appear in your WiFi logs. There’s even a deeper level theory on this one where devices (particularly cheap Chinese toys and gadgets) come preloaded to exploit common vulnerabilities in other devices to establish this relay. Imagine a smart oven or fridge where the manufacturer has no incentive to keep up the firmware for security purposes. So now you have a baby monitor sending data to China through your air fryer’s WiFi connection.
There's no protocol for that over Bluetooth. One could be created, but it hasn't been yet. Nor for relaying data. These don't exist. Like, there are limited relays like what iPhones do for AirTags, but not for a vacuum cleaner to transfer data via your fridge to your neighbor's open WiFi.
In every case where someone's TV has mysteriously connected to a neighbor's WiFi, it's kids or the babysitter or someone else who actively tried to connect to watch a show.
The real madness is to think that data harvesting is not happening.
"2024/02/29, 14:06:55.852622 [LogKimbo][CAppSystemState] Handle message! cmd_id 501 RS_CTRL_REMOTE_EVENT, len 8 serialno 0"
Note something being named RS_CTRL_REMOTE_EVENT
I'd have been tempted to explore this further - does sending fake or repeated telemetry satisfy it?
I’m not really defending it but pointing out this is still on a different plane than outright maliciousness.
It might be a malfunction caused by his blocking, but the idea that someone in HQ was like "guys, we've got someone blocking telemetry!" "disable his vacuum, the bastard".
Or in some design meeting they were like "what do we do if a handful of privacy nerds block our telemetry?" "well.. I guess we should automatically disable their vacuums in a weird way so they repeatedly send them in for repair and it costs us loads of money".
Come on, at least try to live in the real world.
He posits that some low-level support person triggered a remote "kill switch" because he dared to block some telemetry servers which is, frankly, ridiculous.
The article is obviously AI-written, and also I very much doubt that these conclusions were reached without a sycophantic AI in their ear.
1. I didn't see any obvious AI ticks in the article.
2. If you want to claim that some slop is AI then please bring reasons. Even if they are the stuff of "there is too many em-dashes" then fine at least you brought something.
I do see a lot of em dashes throughout the opening, but at least one of them seems proper. "Inside, the iLife A11 wasn’t just a vacuum cleaner; it was a small computer on wheels." is also kind of an AI tick phrasing. And there's pretty heavy use of bullet points for listing things beyond what I would normally expect from a tech blog.
(Also a lot of random lines are in block quotes for emphasis, but that could be a writing quirk. Kinda weird to read though)
If you go through there's at least enough of a smell I suspect someone had an AI polish or edit their actual blog post here?
Maybe he's using a Mac?
Those of us who have been professional writers are quite comfortable with pressing ⇧⌥-
> 2024/02/29, 14:06:55.852622 [LogKimbo][CAppSystemState] Handle message! cmd_id 501 RS_CTRL_REMOTE_EVENT, len 8 serialno 0
> Someone—or something—had remotely issued a kill command.
Uuuuh are you sure that you're not reading a bit too much into the word "REMOTE" in that logline?
These are some very strong accusations and opinions that to me don't feel like they're being backed up with equally strong evidence. At least not evidence that is part of that post.
What even is a RS_CTRL_REMOTE_EVENT? Did you maybe check with e.g. Ghidra?
I mean it does, but not like shell commands but probably IR remote? The CRL-200S can be controlled via an IR remote, so it is possible that it saw something. The sun, perhaps?
Feel free to prove me wrong on this of course.
If last connection time < N days ago and last M tries connecting were unsuccessful, then: brick myself.
Still shitty, no doubt (and very similar to planned obsolescence), but the customer can un-brick by resetting to factory like they did in the service center.
He's insisting that they repeatedly remotely disabled his device in retaliation for blocking their data collection...
...yet they paid for the device to be shipped back and forth and inspected several times under warranty, presumably costing them $$$$?
It makes zero business sense to break your customer's products intentionally, which will lead to 1-star reviews and expensive support.
Plus, I hate to be the "this sounds like it was written by ChatGPT" guy, but this does. People don't write like this:
> Deep within the robot’s startup scripts, I discovered the smoking gun.
> It came back to life instantly. They hadn’t merely incorporated a remote control feature. They had used it to permanently disable my device.
> I may have lost my warranty, but I won back my autonomy.
Also, the idea that someone would waste months (?!) of their life on some broken vacuum cleaner until they "had a complete understanding of how the hardware was designed, down to each chip and wire connector" is just not real life. This is more like someone with a mental illness related to obsession, unless they're starting their own smart vacuum company.
I'm guessing this is 100% fiction from ChatGPT. Complete with the AI-generated image.
A vacuum cleaner that doesn't suck ? Is this from Microsoft ?
- Ten em-dashes
- "not just A, but B"
- wasn’t just a vacuum cleaner; it was a small computer on wheels
- they didn’t merely create a backdoor; they utilized it
- they hadn’t merely incorporated a remote control feature. They had used it to permanently disable my device
- And an overly dramatic/promotional tone
Obviously the image is AI as well, but /shrug
What hostname/s did you block? What filename prevents auto-reboot? What firmware version is your device? Were any credentials necessary to access your robot’s internal syslogs? Was the remote always precisely 8*86400 seconds after you powered on the repaired model?
The repository contains only the barest “how to repurpose this device” details with no supporting material evident for your post’s topic, “what the OEM OS was doing”, which makes the final paragraph either wrong or misleading. Do you have a timeline in mind for when that will be published to GitHub?
The story is marginally interesting, but without the technical details, it’s more “this is completely unsurprising, see also nearly all in-home smart devices” and less “this is novel and interesting”. (I concur with the outrage, but outrage alone does not satisfy.)
I can agree, however, that refusing to work without internet is be too much for the device which can support offline operation.
So, given that, why are you worried about rtty specifically? It's likely a redundant debugging channel in case the main app crashes. It does not add any special functionality that main app does not have.
Now re "disabling the device" - I wonder what command means? Could it be something like "local logs buffer full, pausing operation until upload is done"? Thinking about this more, your blog basically says:
1. vacuum works fine
2. you disable half of the ports on the firewall
3. vacuum stops working
4. you send it for warranty repair
I was very surprised to see that 4 was "send it to warranty repair", instead of "re-open ports on firewall and see if it starts to work now". Did you try this? If not, then it's pretty likely the vacuum was not "bricked" in any sense, but rather was waiting forever for its logs to get uploaded.
In what way is this mundane? The writer purchased a device, and after purchase the device was remotely disabled.
Terrifying - that it happened is alarming but that it is now "mundane" is utterly chilling