How did Roomba-recorded photos end up on Facebook?
technologyreview.com
technologyreview.com
> iRobot … confirmed that these images were captured by its Roombas in 2020. All of them came from “special development robots with hardware and software modifications that are not and never were present on iRobot consumer products for purchase,” the company said in a statement. They were given to “paid collectors and employees” who signed written agreements acknowledging that they were sending data streams, including video, back to the company for training purposes. According to iRobot, the devices were labeled with a bright green sticker that read “video recording in progress,” and it was up to those paid data collectors to “remove anything they deem sensitive from any space the robot operates in, including children.”
Seems like the real story is that training data was leaked, rather than the attention getting “they’re watching you” narrative the title suggests
But yes, the real story is that training data (and “real” data) does leak all the time and that most companies don’t take insider risk as seriously as they should.
The meat of the article is that what technology and tech companies are doing is divorced from the expectations that we have as a society.
It couldn’t have been done from any roomba, but it could happen to almost everyone who didn’t understand the exact ramifications (which we click through several of every year to try to get the vacuum up and running). That’s why a lot of ppl on HN put masking tape over their laptop webcam. Or are you calling those people paranoid?
Is it a bad thing if people err on the side of being too private, for a change, instead of blindly trusting big tech tracking everything they do, or being completely uninformed about this sort of thing?
Precisely how the "Zoom is Chinese spyware!" stuff spread around.
I did not expect to see actual photos of the woman sitting on the toilet in this article. But damn, they're real and published dead center. It's awful and voyeuristic to feature, but in a way it brings to life the freakishly perverse Orwellian horror of all of this.
This piece hits hard, as it should.
How did neither Roomba nor ScaleAI have safeguards against PII of this nature? This is inside people's intimate spaces. It could have been sex. Or children. How did they not think of this?
This sort of disregard for privacy should be punished, and this woman should be able to sue Roomba and ScaleAI for a handsome sum.
Maybe they did have some kind of internal data privacy policy or 3rd party policy, but it was wholly inadequate.
My team once had a certain perennial Billboard chart topper's login credentials due to suspected mishandling by one of their team (I'm still afraid to say whom), but you'd better believe we treated it - and all of our customer data - as sacred taboo. Mishandling PII was fireable at minimum, and could probably land us in litigation with a permanent mark against our careers.
We need GDPR/CCPA++ protections here. As an added bonus, the companies that play nice will get a comfortable moat in the form of their compliance.
It's also absurd to think they didn't face safeguards. We can only speculate if the individual was fired, or if stronger policies were put in place since 2020, but it's naive to expect that whatever policy is put in place will stop a human data labeler from smuggling PPI for personal reasons.
But as a guy, I don't know if girls pull them down farther if they think they're not being watched. She might be the one who agreed to it.
Still, the owner of a space is making the decision for all people who come into that space.
Well, you'll find a Roomba spokesperson saying that, anyway.
I don't mean to imply that the spokesperson was lying and these were plain old roomba's instead of special R&D Roomba's.
I do mean to imply that I think a roomba spokesperson would describe "included in paragraph 12 of a 23 paragraph terms of service that they clicked through" as "specifically aware", but I would not consider that any kind of "proof" that the people actually were "specifically aware". At best it is an argument they ought to have been, which is a different thing (and it's an argument).
Therefore I don't agree with your implication as I just don't see any evidence to support it. Even the article's author, with the evidence they were given doesn't push this point.
You are making claims about what the owners "specifically knew". What is the evidence about what the roomba owners (or, uh, holders) did or did not know, "specifically" or otherwise?
> They were given to “paid collectors and employees” who signed written agreements acknowledging that they were sending data streams, including video, back to the company for training purposes.
So it hardly seems likely that they would not be aware of this.
I opened the article (on a phone) and no fewer than 3 separate popovers appeared over the content. “Hey! This is our cookie policy” “Happy holidays! We have a special subscription price!” And something else that was covered by the first two before I had a chance to read it.
Thankyou for summarising. I noped right out of there out of disgust.
Roomba iOS app refuses to go past its welcome screen unless its granted access to the location info.
This is unreasonable, they don't need this info for their app to function.
However their devices are all but unusable without an app, so they ultimately blackmail people into giving location data to them.
Meaning they don't really give a sh#t about users' privacy, so it's not that "they are watching you", but that they won't think twice about hooking up to a random Roomba and shooting a video with it. Consent or not.
Out of spite I took the iPad to a friend's house, enabled location, clicked through the welcome screen and disabled it again. Then came back home and hooked up the device.
So, no, it's not linked to its BT needs at all.
Does anyone know, is this like Android where some bluetooth functionality is behind the location permission?
There’s a specific “local network” permission nowadays so maybe it was the older permission API because you can deduce location from MAC addresses?
And I am not sure I buy the “anyone who does one bad thing will necessarily do all of the bad things” view.
And that's without getting into why it must have an Internet connection (to enable a large chunk of its functionality). It has a Bluetooth, it is perfectly capable of talking to the smartphone using it, but it can't be controlled that way.
Re: bad thing - not a "bad thing" per se, just a very cavalier attitude, which is as troubling.
I've returned several "smart" devices that required an app and an account. One example is GoVee smart blubs. As soon as they refused to work without an account I sent them back. Another is some Meross smart outlets. I retuned them for Eve Energy outlets.
The burden is on them to not perform those actions, not on me.
I should be fully entitled to make use of the product without agreeing to any additional contracts or permissions.
Either way, Android (and iOS) should be stricter about these things; apps should work without any permissions. I mean a navigation app without location access won't work very well and permission can be denied by accident, but that can be resolved.
If I disabled the permission it would stop working.
So I switched to the home assistant integration for it and removed the app from my phone.
Best thing it's he didn't even know at first. He bought a new washing machine that had app support, so out of curiosity he installed the app and scanned for devices, but the app only found his stove.
Technology was a mistake.
Another smart scenario I can think of, link it with energy prices and a real-time dynamic energy contract, and/or solar panel output or battery charge levels, to only run it when it's cheapest to do so.
Of course, that means having it stand-by to run at all times, meaning it may run half-empty and you need to keep the door closed which will cause mold etc.
Should be, but here's the thing: they don't want to build the best app or customer experience through convenient features, they want your (location) data, because aggregated, that shit is worth more than the stupid dishwasher.
They don't make money on selling dishwashers anymore; components, shipping, marketing and middle men will swallow up any profit. But location data and subscriptions will bring in the real money, over a long and continuous period of time.
See also this story from a Twitter engineer who talked to mobile phone companies that were prepared to pay big money for location data; I can't find the source anymore, but it's been reposted on various outlets: https://hindupost.in/media/an-ex-twitter-engineer-reveals-ho...
You can't pair anything with an app unless you turn the location on. Main feature of the Roomba app is to pair a vacuuming robot using phone's wifi and a special wifi network between phone and vacuum.
So while it's not a perfect solution to block you from going further, you can't really be that paranoid about them asking for that access. And ofc, I'm sure your location flew to the iRobot servers along with your other details that could've been scraped from starting the app on your phone.
edit: added paragraphs for readability.
I participated in an iRobot Study in 2019, since then I have received multiple invitations that I have declined specifically because they mentioned streaming video or cameras.
Here's the following para:-
"In other words, by iRobot’s estimation, anyone whose photos or video appeared in the streams had agreed to let their Roombas monitor them. iRobot declined to let MIT Technology Review view the consent agreements and did not make any of its paid collectors or employees available to discuss their understanding of the terms."
Was the HN title changed. I am having trouble seeing how one who did not read the article would interpret the HN title to suggest a "they're watching you" narrative. For example, one could interpret it as posing a question about leaked photos, e.g., who leaked them and/or why.
The idea that a "they're watching you" narrative is "attention-getting" is interesting seeing that many HN commenters repeatedly tell us that people outside of HN do not care about privacy or surveillance. If that is true, then why would a journalist/publication seeking the largest possible audience try to advance such a narrative. No one cares. That's what they tell us, anyway.
NB. I am not suggesting that I believe the HN comments arguing that no one outside HN cares. They could be correct. However I believe the commenters making them could be biased if they are invested in the survival of the so-called "tech" industry, i.e., targeting internet users with data collection and surveillance and selling online advertising services as a "business model".
The actual title is: "A Roomba recorded a woman on the toilet. How did the photos end up on Facebook"
This website is a publication of MIT. A strange source for "they're watching you" narratives.
Seems like the narrative's just as accurate as it was prior to reading the title.
And the sequence seems to be:
1. iRobot hires people to use special development versions of the Roomba in their homes to collect training data. These are clearly labeled, and the participants are informed that the images are being sent to iRobot for training. This seems fine - if you want to exchange some degree of privacy for money, that should be your right as long as you're clearly informed about it.
2. A contractor posts some of these photos to a private Facebook group used by other contractors on the project. This is obviously bad, but at the same time, it's limited in scope to people who would have had access to these photos or similar ones.
3. The MIT Technology Review gets a hold of these images and decides to publish them on the Internet for everyone to see, just to get more clicks on their article. This feels like the most egregious privacy violation in the sequence.
That's where it went wrong. Everything else seems reasonable for a visual AI training project, well signalled to the participating users and the data securely communicated.
Thereafter, the data was mismanaged.
There is clearly no such things as a "private" Facebook group. So called "contractors" [1] using a disservice like Facebook to communicate beggars belief.
[1] people with the unremarkable skill of being able to spot ordinary household objects and label them - so someone probably had the bright idea of creating a CAPTCHA "Find all the women on toilets".
And this is the problem with most of the technology spying on us. Once the data is in someone else's hands, there is nothing you can do to prevent it from being "mismanaged". It can exist forever and you aren't allowed to know who has it or what they're doing with it.
Bugging your own home with any device designed to spy on you is just a terrible idea and I'm amazed at how many people are oblivious to the harm they risk bringing on themselves and everyone else around them.
All of these are reputable, popular SaaS applications, widely used to collaborate at work. All of these are equally trustworthy (read: not that much), and give you the same privacy guarantees. Using either in a company setting without a contract in hand is unwise, IMO, but that ship has sailed long ago.
GP is correct. #2 is bad, but not significant. #3 is the bigger violation here.
You didn't fail. Because there isn't one.
> All of these are reputable, popular SaaS applications, widely used to collaborate at work.
As you admit yourself;
"these are equally trustworthy (read: not that much)"
No one would ever sign up for their pictures being taken in a bathroom.
https://nypost.com/2017/07/25/roomba-maker-wants-to-sell-you...
Considering that this is how poorly they protect the sensitive data their devices collect (even those used for development purposes) I guess it's a good thing that public backlash over their spying plans forced them to reconsider.
This feels like pearl clutching for no reason.
It's not hard to find out the layout of a house. In a given neighborhood there are probably dozens of houses for sale, which would have their floor plans publicly available as part of their listing. At the same time, there is a high chance that multiple homes in a given neighborhood share a floor plan. Therefore the layout of any given house isn't exactly some sort of a secret.
As for figuring out whether a house is occupied, there are far easier ways. Any internet connected laptop/phone has built-in cameras and microphones, which also allow you do determine occupancy.
Of course, none of this matters. The typical robber isn't going to do some ocean's 11 heist shit where they're scoping out your house's layout in advance and hacking into megacorp's servers so they can learn your daily routine. They're far more likely to make an educated guess based on whether your car is parked and whether the lights are on. If they're really prepared they'll maybe drive through your neighborhood several times a day to get a better sense of occupancy.
Continuously updated maps of your home and its contents over time is a lot more revealing than just a basic floorplan.
> All of them came from “special development robots with hardware and software modifications that are not and never were present on iRobot consumer products for purchase,” the company said in a statement. They were given to “paid collectors and employees” who signed written agreements acknowledging that they were sending data streams, including video, back to the company for training purposes. According to iRobot, the devices were labeled with a bright green sticker that read “video recording in progress,” and it was up to those paid data collectors to “remove anything they deem sensitive from any space the robot operates in, including children.”
> The data labelers found this work “really uncomfortable,” she adds.
This is an interesting point - the article seems to present that this was not done out of malice (as the woman's face was pre obscured).
> Labelers discussed Project IO [another assignment by Scale] in Facebook, Discord, and other groups that they had set up to share advice on handling delayed payments, talk about the best-paying assignments, or request assistance in labeling tricky objects.
It's clearly against policy,
> But such actions are nearly impossible to police on crowdsourcing platforms.
> When I ask Kevin Guo, the CEO of Hive, a Scale competitor that also depends on contract workers, if he is aware of data labelers sharing content on social media, he is blunt. “These are distributed workers,” he says. “You have to assume that people … ask each other for help. The policy always says that you’re not supposed to, but it’s very hard to control.”
I'm honestly not that suprised that something like this happened, where similar things happen for mturk.
1. Society demands this kind of automation 2. Companies reacting to this demand have to hire humans to perform manual labelling 3. Humans that perform labelling don't always follow the rules and policies in place 4. Data leaks occur 5. Article like this are written 6. Demands for automation don't really change
(repeat)
We see similar stories all the time, whether it's about companies leaking data that was collected via consent, data collected without consent, or data collected without anyone knowing about it gets leaked.
Even Apple has been caught recording via Homepods without consent.
Even apart from the privacy stuff, the fast local web interface and open standards integration support (mqtt, homeassistant etc) are brilliant.
there's a major turnover in their federal team...
Lidar, in theory, could create a photo-like image, but that resolution costs money and none of these robot vacuums are anywhere near that. Plus they map depth, not texture, so anything it does create is somewhat abstract.
The few robot vacuums that have only a front lidar sensors perform poorly compared to the camera only or lidar augmented camera in the latest Roomba and Roborocks.
That goes for the device manufacturer as well. They couldn't possibly prove the fidelity of their statements except on a witness stand under the penalty of perjury. So unless you think they are conducting the type of conspiracy that could see some of them sent to prison, we might just have to trust that they don't defraud the public on a regular basis.
It is simple for any person with even basic knowledge of networking to independently come to the conclusion that Roombas are not uploading video streams (or photographs) to the internet.
I know the IP (10.0.0.11) and MAC (50:14:79:1E:AB:6B) address of my Roomba and using the Insight Netflow Analyzer for OPNsense I can see how much data it has sent to the internet. In the last six months it has sent approximately 72MB of data outside my network. That's about 600KB per day.
It has received much more, presumably firmware downloads.
This is consistent with firmware update checks, notification traffic, and me periodically adjusting its schedule remotely.
That's just me clicking on some tabs in my router's web UI. Hundreds if not thousands of people globally are constantly reviewing and monitoring Roomba network traffic in fine detail in order to understand and/or reverse engineer it for research and other purposes.
So one of three things is happening:
1. All Roombas send photo and video streams to iRobot and they have thus far managed to hide this from the public and the thousands of eyeballs constantly monitoring the network traffic of their products, or
2. A subset of Roombas send photo and video streams to iRobot and they have thus far managed to hide this from the public and the subset of eyeballs monitoring the network traffic of their products
3. These are development devices like they claim.
Based on my own experience we can eliminate 1, based on the images accompanying the article option 3 is highly likely.
A non-trivial number of them. Thousands, out of millions, at least.
The reporters tried to validate and were blocked. That’s where my suspicion lies.
> The remaining training data comes from what iRobot calls “staged data collection,” in which the company builds models that it then records.
> iRobot has also begun offering regular consumers the opportunity to opt in to contributing training data through its app, where people can choose to send specific images of obstacles to company servers to improve its algorithms. iRobot says that if a customer participates in this “user-in-the-loop” training, as it is known, the company receives only these specific images, and no others. Baussmann, the company representative, said in an email that such images have not yet been used to train any algorithms.
As a bonus, they seem to be far ahead of iRobot at vacuuming/mopping.
It's not a great option, but might be the best if you are looking for something that will map out a house and clean designated rooms, and that will function when not connected to the internet.
Most of the home devices like vacuum robots really have no valid reason to connect to any cloud let alone send any pictures there. Such a robot can run all the necessary code [ran in the cloud normally] on itself. If only the consumers would not be so naive to accept the cloud bullshit for a norm.
Why are they labeling furniture in home that Roomba can't possibly reach from the floor?
That's not the only approach though. You can look forward (or just use lidar), but this navigation approach seems to be less sensitive to, say, furniture been moved around.
https://www.reddit.com/r/roomba/comments/qrs5e5/roomba_beta_...
and, of course, iRobot is in the middle of being acquired by Amazon, who have a long history of giving police, access to customers' camera feeds without proper opt-in processes.
The sort of live-view functionality described in that post doesn't necessarily require Roomba to store/transmit video/mean it was part of the set of images leaked by ScaleAI contractors.
So can people with credentials of compromised iRobot accounts.
So can Amazon, post acquisition.
People that purchased these devices weren't signing up for a system with such terrible privacy implications.
[0]: https://www.theatlantic.com/ideas/archive/2022/08/amazon-roo...
Even then, who made the consent? Did they verify the person who did legally had the ability to do so? What happens if the person on the toilet is a child?
I hate lawyers and I’m not one myself but even I can tell you this is a bad fucking idea.
> All of them came from “special development robots with hardware and software modifications that are not and never were present on iRobot consumer products for purchase,” the company said in a statement. They were given to “paid collectors and employees” who signed written agreements acknowledging that they were sending data streams, including video, back to the company for training purposes. According to iRobot, the devices were labeled with a bright green sticker that read “video recording in progress,” and it was up to those paid data collectors to “remove anything they deem sensitive from any space the robot operates in, including children.”
At the very least, companies should have sign an oath to protect their customers and employees - not to abuse them… similar to how health professionals have an oath to do no harm. Is that too much to ask in this world.
...aha!