The solution to this hack is simple: Shut this company down, because it's a bad idea.
Ideally it would store video locally, then encrypt it and upload the last minute to the cloud.
I can entirely get why a company would outsource IP cameras to a third party cloud, even with storing data on-site. Business runs on contracts. It's entirely normal to contract out everything except your core competencies, if it's cheaper this way. It's how you turn CAPEX, complex OPEX and high risk into simple OPEX and low risk. A contract is in big part a risk shifting tool. This works well in practice... outside IT.
The problem is, with IT and data, there's a mismatch between expectations and reality. An enterprise should feel safe buying their video surveillance from Verkada, because between the contract and the legal framework, Verkada should be bankrupt now, and their management possibly facing jail time. That's the part where contracts work as Cover-Your-Ass tool: if you shift risk and liability to outside party, the liability is not on you.
However, this only works as long as the other party actually internalizes the risk and liability. Since there are no consequences for mishandling data, operating IT services you're not structurally competent to operate, and eventually having your crown jewels stolen - the contractor doesn't really internalize risk, has no incentive to mitigate it.
All this to say: Verkada should go down after this, and their customers should be named and shamed widely - the latter is so that future customers of IT services put more care into vetting companies they contract IT out to. You shouldn't get to CYA with a contract where assumptions around contracting are broken.
If you have an on-site system (like I do) it's way more effort for the average consumer.
edit: not talking here about active monitoring security cameras used by guards.
Theft of recorders/loss of recorded video is much much less of an issue than it is generally made out to be, and most systems have various means for auto backups, dual recording, etc. Many also have the ability to send select segments of video, based on motion or analytics, to cloud/FTP/email for free, which adds some extra resiliency if you are really worried about random arsonists :)
(I work in the industry)
Ohhhh, you are so in trouble now! :P
I am pondering moving to a rather remote estate with my wife but we're both city kids and have zero clue how to defend ourselves in such conditions (I am even thinking we should take shooting lessons and bring guns to the estate as well).
Part of our plan is a surveillance system -- I planned to have an on-site ZFS storage cluster with several cameras that periodically encrypts the last 5 minutes and sends them off-site. Not sure how complex such a setup might turn out to be though... Maybe there's a way to make the storage cluster itself auto-replicate remotely often enough? I'm not that educated yet.
Do you have any recommendations? I don't want to spend $5000 due to paranoia but I don't want to be defenseless in case of a robbery either. What's the middle ground?
Still not sure about guns but the dogs would indeed be a very good investment.
I think you might be overthinking this. The dogs don't even need to be large. If the property is more than a few acres just get 2 or 3 herding dogs. They're upbeat, energetic, and above all very noisy if a stranger wanders into your yard. I very much doubt anyone will ever bother you.
> Still not sure about guns
How remote is it? What's the police response time? If it's really in the middle of nowhere then it seems like a good idea to have something on hand just in case.
Exactly because of that: remote and small country (Costa Rica is one idea so far), we would prefer a sea-side house which is NOT very close to a city (we think 20-30km away from a city is optimal) and we have no clue about police response time -- but I can't imagine they'll be there 2 minutes after I make a panicked call that I am tracking 3 armed burglars, especially if not in a city.
> I think you might be overthinking this.
Very possible. Right now I don't even have the money for a down-payment, but me and my wife both share this dream and started brainstorming it and doing some preliminary planning.
Above all, I really want the dogs to not be able to be bribed with food. Not a fan of dogs that bark at every single small disturbance but oh well, if that's the price you pay for living in a remote area and having good security then okay.
Oh, I was imagining remote rural in the US or a similar country (Canada, Scandinavia, etc). I have absolutely no idea what public safety concerns might exist in less developed places.
> want the dogs to not be able to be bribed with food
It's more that a solid looking fence, a few dogs, and some visible security cameras are highly likely to deter any attempts in the first place. If someone is armed and willing to shoot your dogs then you have much larger problems to deal with (and probably don't want to be living there to begin with).
Downsides include being extremely smart but not very trainable (smart like a wolf) -- they decide what's a threat and what isn't, and they don't care about your opinion. They guard naturally. Also, you may find yourself having to bury dead coyotes on occasion, as they will murder any that are stupid enough to intrude.
I often have debates with my wife about country vs city living, but I don't think buying a gun / attack dog / big security system ever comes into the equation. (UK based).
Am I not paranoid enough, are you over paranoid? Is there a Bayesian equation we need here?
Just to put up an alternative I just read this https://news.ycombinator.com/item?id=26411899
The guy cycled through Europe and India, met some guy in a bar in Italy and ended up sleeping on his couch and eating home cooked Pasta lunch the next day. A different approach would be to drive with a gun under the seat. I think something is missing.
Do you also forgo a fire extinguisher in your kitchen?
Often what's "country" in the UK (outside of Scotland) would be considered exurban in the USA. You might possibly feel different about security at a truly remote home in N America.
The concept of having a predictable recurring bill, and known costs can have a lot of value. In the opex hardware model, there is usually a hardware refresh built in (a lease of sorts) and also covered replacements for any failures before the refresh.
This makes no sense to me, because almost no OpEx-based services operate on a predictable billing model. Usage based pricing means they are subject to change.
If I buy a $5,000 server, and it lasts me five years, I can plan around needing $1,000 a year set aside to replace it. Meanwhile, my AWS bill can suddenly be $20,000 with no warning.
The hardware refresh model you describe... I've actually only ever seen in a CapEx appliance, where paying for the support contract got you those refreshes regularly, replacement for failures, etc. but had an initial upfront purchase cost.
I'm not someone who's crazy about privacy, but this is a pretty dark indicator for a company housing DNS query records. Maybe its time for someone to build a proxy for tunneling Cloudflare DoH/DoT over tor or some other free mixing network.
I think you give up any sense of privacy as to where you're located in an office or where you've been in an office when you decide to work in an office owned by some employer. I don't know why there'd be any expectation there.
It isn't. https://en.wikipedia.org/wiki/United_States_Office_of_Person...
If it's really secure there will be monitoring of all entrances, including corridors. (And there will still occasionally be people successfully tailgating, usually for perfectly innocent reasons like forgetting their badges at their desks etc. Real security is all sorts of fun.)
For the uninformed - badge in to open the entrance door. The room then locks and you use your badge to open the exit.
Any access control system has the capability to integrate with a fire system and allow this.
In secured areas where they want you to swipe out or places where they might get tripped accidentally, they sometimes have like a 15 second lockout before actually tripping the door.
I've been in just one server room and they just had a motion-deactivated maglock tied to an electric strike so in the case of a power outage a simple mechanical lock could be opened but otherwise you need to badge in/out.
This seems like plenty of due diligence for the store not to be liable is someone gets locked inside.
Not at all in the paranoid "are you slacking off?!" sense, but just security information like knowing when I've been in a server room, or knowing if my work computer sent traffic to a known botnet C&C. If there's a security or theft incident and they don't know who's been in their building or what their computers are doing, it's pretty much impossible to investigate anything.
I understand that in places like Europe there's a very different culture and workers have a lot of protections from things employers may want to do, but not everyone around the world feels that way. Basic record-keeping of when badge-restricted doors and computers are authenticated to doesn't feel invasive to me in the slightest, even if others may strongly feel it is invasive.
There are many things I would find egregiously invasive, such as a manager inspecting all the websites someone visits to assess how productive they are, or timing people's bathroom breaks, but I just avoid such companies.
Maybe what we should prevent is employer keeping months of proof and only bringing it up as inappropriate later, but if the employer uses the camera to tell an employee within 24hrs that he needs to ramp up, it feels ok. Maybe we should impose rules like “24hrs max” and “can’t be used legally, just orally.”
On some level it depends on what 'slacking off' means.
I've had employers where 'slacking off' meant actively doing some %mundane/repetitive/unnecessary% task with every moment of my free time. We were literally pulling the finish off the counters; there was no need to keep dusting them.
I've had software shops where reading integration documentation was 'slacking off'.
An interesting data point; In Germany, MS Office doesn't track how long you have been editing a document. My understanding is this is because the law there more or less says if you pay someone to do a task, you aren't supposed to (i.e. can't) care about how long it took them to actually do it as long as it was done on time.
So I guess that's my problem. There's a very fine line between employers using surveillance to catch 'bad actors' and employers using surveillance as another tool to bully substandard work conditions onto people.
I'm sure they have a legal right to check (in the US), but I really wouldn't want to work for such a company and would immediately start looking for a new job if it happened to me.
We were tracked by contract (badge into building, badge into area). We couldn’t leave the work area un-attended which was a pain, so there were “processes in place” (last person badge etc..).
Generally we knew they left you alone unless you were cheating. (Having someone badge you in when you weren’t there was a fire able offense).
I don’t miss it, but it wasn’t that bad. Of course having the work network not on the internet what else could we do but work...
I really doubt Cloudflare is the type of company to be tracking where each employee is and whether they are taking too many bathroom breaks. It's definitely an area abuse is possible, but probably not an area it's likely in Cloudflare's case.
I absolutely agree. The thing that concerns me is these cameras sitting on the internet. It says something about how overworked the security team is. I trust that they have good faith, but I don't know if they have the resources they need.
Actually, if you want to have IOT access outside of the network, the best approach is to close all ports and for the device to initiate connection with a control server. The device is dark when scanned while a heartbeat signal will ensure connectivity. This will require a good security on the control server, but that is okay because server security is much better understood and does not suffer from the constraints of the embedded software.
We do not use that feature and do not intend to.
The majority of real camera systems (100+ cameras, 10+ locations) I have interacted with are monitored by outside vendors.
It wouldn’t be as newsy, though because people put these in their own homes.
The only way this group raises awareness instead of angering people is by targeting companies and institutions, rather than forcing people to confront their own compromises with technology, time and effort.
I am very impress with their features. I am planning to buy couple of them, but I would never point them inside the house. Probably will put them in their own wifi network.
I built a home grow solution with RPi + Webcam + Google Drive with some python script. But the performance to upload Pics to G Drive was slow and the way viewing the pics uploaded to G Drive was not very convenient.
If you want something like a video of that opossum sneaking in the attic, coyote peeing in your rose bushes, that kid across in the street with the razer scooter and german shepard not cleaning up the poop off your lawn or what your iguana is up to when you're not home then it's great for that.
The UI is probably worse than your gdrive setup. You have to use the wyze app to access it, which has a terribly slow scrubbing interface. The more convenient part is to check the motion/sound triggered 15 second clips which are upload to their cloud servers. Other than that I find myself frustrated and just take out the microSD card and browse it on my computer.
Also don't buy the sense kit. It just doesn't work. I've even tried experiments with it right next to the camera and it's not consistent enough.
If you want to use the wyze cams are generic IP cameras they offer unsupported firmware for that but then you'll need to roll your own DVR and I'm not sure if that supports all the new features the V3 offers or follows the genric protocol for remote pan control.
I've never used (or trusted) the Wyze firmware anywhere on my network, but I use a couple of Wyze cameras pointed outwards from my house connected to a a DVR with motion detection.
The first thing I do with a wyze cam is physically remove the microphone, because it can only otherwise be turned off in software which is configured in the cloud.
I'd consider using them with the OS firmware change, but hadn't thought of also running them on an independent network.
But it suggested that Wyze can remotely disable all encryption and so on whenever they feel like it.
And that for a "subset of users" they collected: "Height, Weight, Gender, Bone Density, Bone Mass, Daily Protein Intake, and other health information", which was included in the breached data.
[0] https://www.nytimes.com/2019/12/30/business/wyze-security-ca...
Cloud security footage makes sure that you have at least the footage up until they disable your network.
In my case, I've got a camera in my elderly parent's back yard as there have been a few falls in the back yard, and one break-in some years back.
Being able to remote in and go "ah, everything's fine" basically can instantly turn such anxieties off, rather than say, having to live with it until you finish out the work day and can head home.
I agree there are trade-offs, but especially for large enterprises with security teams, on-prem is definitely more secure.
It's a necessity for some cases. I commented on this elsewhere, but for me it's having an eye on my elderly parents back yard in case of falls. Past history of not knowing is precisely why we got the camera.
Also with the cloud being used as the de-facto off-site backup location, most data these days will end-up (hopefully encrypted) in some kind of cloud service or another.
Exactly the sort of people you want to trust with highly sensitive surveillance.
I tend to agree, but ...
If you're a small business or manufacturer, IT is a pain in the ass. The "cloud" is a benefit because you don't need to maintain any servers yourself and can just get on with your business.
The problem is that these companies don't face any consequence for claiming that they're secure and then not actually being ... you know ... secure.
If this stuff was simply encrypted at rest, that would have mitigated most of this breach.
Small businesses have either a box they bought at Costco or use something like Comcast's service which they just add on to their cable modem.
Anyway, I think we're at the point with software/digital systems where food manufacturing was in 1900: there's been a gold-rush due to new technology, and we're reaching peak negligence in the pursuit of profit, and I think we'll soon get to a threshold where regulators will finally step in and lock down the wild-west and impose some real standards. Between SolarWinds, Exchange, and now this, it's to a point where it's dragging down our whole society. Something has to give.
Perhaps that's the real reason behind the anti-FDA lobbying we are seeing here. The FDA demonstrates that government-mandated safety standards work. We can't have those in the IT field because they would diminish profits!
Also: Hepatitis C antivirals. One pill and you are cured, whereas in the past you'd be looking at a liver transplant. If that's not innovation it's impossible to say what that would be.
For anything large or distributed, you’re probably going to spend less for better security with a cloud system.
It’s sensitive data, but probably in a second tier of sensitivity. The integrity of the on-prem system depends in your remote access security and operational practices. Solutions like this often really suck.
I was able to send them links to the video clips on the Nest site, with embedded timestamps.
If this was a local system there would be no way to prove I hadn't just faked the timestamps.
Showing a ticket? Parking lots have been using them pretty successfully for 50+ years
Who should shut it down?