Update: UniFi Phone Home/Performance Data Collection | Hacker News Reader

Update: UniFi Phone Home/Performance Data Collection

community.ui.com

204 pointsbhauer132 comments

We've seen two episodes of companies who enjoy the goodwill of the enthusiast/advocate sector committing obvious foot-gun maneuvers in as many weeks. First it was GitLab, and now Ubiquiti. Both should have known better than to hoist telemetry on self-hosted software. Doing so betrays a willful ignorance or disdain (or both) for the very reasons these enthusiasts self-host.

The good news is that in both cases, the community has been clear in its reprimand, and the companies have reversed course (or promised to do so, in the case of Ubiquiti). I'd like to see more companies realize this is an opportunity rather than a threat: an opportunity to create products and services that deeply respect privacy and market this advantage over the competition.

GitLab's CFO was kind enough to leave public evidence of his incompetence:

"I don’t understand. This should not be an opt in or an opt out. It is a condition of using our product. There is an acceptance of terms and the use of this data should be included in that." [1]

In the previous thread people were referring to the HiPPO problem (Highest Paid Person's Opinion). I'm willing to bet that it's the same story at Ubiquity. But despite occasional backpedaling, there are rarely any real consequences for the irredeemably stupid managers who push for crap like this. Unfortunately, that's probably the only way to make it stop.

In my opinion, if the outcry is bad enough for a company to backpedal on a decision like this, then it's already bad enough that they should fire the manager responsible for making the decision in the first place. These aren't line employees. The company doesn't pay them more than everybody else for fun -- they're paid to understand the business well enough to know better. People in these positions tend to be natural risk takers. If there are never real consequences, they're going to continue to bet anything they can get their hands on for a shot at personal gain, and the company and their customers are going to foot the bill.

[1] https://gitlab.com/gitlab-org/gitlab/merge_requests/14182#no...

Don't penalize GitLab too harshly for having public issues and discussion. The obvious fix will be to just not have all this public, which is what all other companies of non-trivial size do. I think the CFO's comment was illuminating, but not really worse than I would expect from any misc C-level in any industry.

That's a very valid point, and I don't want to go too overboard in my criticism. However I don't think my assessment is wrong. If he really was the one pushing for this decision then I think he should be fired. It's not just about getting it so wrong when he should have known better, it's about ignoring all of the internal push-back from employees who were trying to prevent him from fucking it up the first place. Bullies in the C-suite should be removed.

> but not really worse than I would expect from any misc C-level in any industry.

This is my take as well, unfortunately, but I think it needs to change. There are loads of competent people who can perform these executive jobs. The bar should be higher.

I don’t think making an executive decision is necessarily “bullying.” The nice thing about public comments and development is that you would see some non-hypothetical evidence of such action.

I don’t think it’s unnatural for an OSS product dev team to think this way. I don’t like that they implemented it, but I think it’s positive they rolled it back.

Firing a CEO for such a situation will result in worse software, I think. It won’t make the next CEO less likely to make a dumb mistake. But it will make them less likely to be public or to roll back changes. If the consequence of a fairly minor mistake (given GitLab’s history of super smart decisions in the past 10 years) is firing, then management will make sure nothing is ever considered a mistake again.

>then management will make sure nothing is ever considered a mistake again.

but it's ultimately the customers who decide if something was a mistake by switching to different products... hence company foots the bill

I think you’re making an assumption that there are executive level people who never make such mistakes. I like to look at this from a different angle - is this person capable of taking feedback and correcting their course. If yes, then you can work with them.

If you fired every manager that every made a bad call because of some lack of understanding then every company would be empty. Everyone screws up due to a lack of understanding at some point. The key is how they react and course correct. The ability to make the decision and re-evaluate as needed is the reason they are are in the role they are in. Having been a IC, MTS, Manager and Exec I can tell you it is not always simple to transfer your views as an individual contributor up the stack when faces with choices that effect the bottom line of the business. On the face of it saying "hey we should auto collect crash data" seems like a reasonable thought process. I am happy with how they handled this.

I recommend their products. I have them installed at work and at home. A few weeks ago I sent an email to them about getting an RMA for something that is clearly their issue. The support person was unhelpful. Lucky someone else reviewed my quite strong reply and said, please RMA it. I plan to do so tomorrow (I have been traveling for 30 days so..). Lets see what happens. No company is perfect. Most suck. So far over the last 5 years I have been over all happy with Ubiquity.

They are violating the GPL:

https://news.ycombinator.com/item?id=21432134

They might offer good products, but they're benefiting from the work of thousands of other people, improve on it and don't give back. If you think this is fine then I guess so is Chinese companies stealing western IP.

If they're violating the GPL, and no one sues them for it, why should I care? Clearly the rights holders don't.

In [1] it's even worse that you have Gitlab's "VP of Legal" and "Director of Global Risk and Compliance" chasing after the CFO's comments telling him that's illegal. That seems kind of messed up to an outsider, like the CFO is completely disregarding and ignoring critical staff that know more than he does in specific regulated fields.

> In [1] it's even worse that you have Gitlab's "VP of Legal" and "Director of Global Risk and Compliance" chasing after the CFO's comments telling him that's illegal.

That doesn't shock me though, and I would say is expected. After all, it's their job to know what's legal and watch over the decisions the company makes.

Ubelievable honestly. While I have a high opinion on Gitlab and am sure they will rectify this error, a CFO should know the audience of the products they offer.

These statements endanger the success of the whole company. I would think a CFO should keep that in mind. At that paygrade I would expect nothing less.

The CFO isn’t incompetent. You agree to all sorts of awful terms by default, including automatic acceptance of changes, severability, etc.

It’s less obvious with GitLab given the nature of the business, but in most cases the vendor has the power in the relationship and can basically tell the customer to fuck off. Where are you going to go, Github?

Same deal with Ubnt. Their core consumer market is WiFi. The competitors are Google, Amazon, Spectrum/Comcast/etc, and a bunch of random companies. Backpedaling is dumb — they are better off letting the angry internet people spew for a few weeks. If you care about privacy, you’re not buying Google WiFi or Eero.

There are companies like Oracle where you pay them boatloads of cash despite knowing that their business is designing new and innovative ways to screw you. The decision to give money to companies like these are almost never technical... it's political or driven by fear or incompetence.

Then there are companies like GitLab and Ubiquity. Most of their customers give them money because someone on the engineering team is discerning, persuasive, and trusted by management. These companies are not capable of pulling an Oracle without losing a significant amount of business, and everybody in the C-suite needs to understand this. Deciding to implement telemetry because everybody else is doing it is not going to fly here.

It is worth saying that half of GitLab's "crime" was the telemetry, the other half was the completely unprofessional way they rolled it out. They essentially disabled people's paid for hosted GitLab accounts, until they agreed to the collection. See [0] from GitLab:

> Starting with GitLab 12.4, existing customers who use our proprietary products (that is, GitLab.com and the Enterprise Edition of our self-managed offerings) may notice additional Javascript snippets that will interact with GitLab and/or third-party SaaS telemetry service (such as Pendo). [...] as we roll out this update you will be prompted to accept our new Terms of Service. Until the new Terms are accepted access to the web interface and API will be blocked. So, for users who have integrations with our API this will cause a brief pause in service via our API until the terms have been accepted by signing in to the web interface.

At least Ubiquiti had the basic courtesy of not breaking people's paid service/devices to roll out their questionable telemetry. Even if it should be opt in, not opt out.

[0] https://gitlab.com/gitlab-org/gitlab/issues/34833

They're not reversing course since it's not an opt-in, but still an opt-out. I don't want to inspect every hidden option on my hardware to be sure that nothing is spying on me.

For me it's still not solve here. I'm sad, because now I have to find another hardware provider that I can trust.

I hear you, and I agree I'd prefer all telemetry to be strictly opt-in. I'd be comfortable with software prompting me to make that decision at installation time to have a chance at me answering affirmatively (I'll virtually never opt in after installation).

But I consider an opt-out a workable option in this case, as long as they realize this mistake should not be repeated, which remains to be seen.

Is there any alternative that provides the elegance and affordability of UniFi SDN?

Thank you. This is an important distinction to make that I wasn’t aware of.

Maybe they will end up making it opt in. In the mean time, would an open source tool to automate ubiquiti opt outs be useful?

Indeed. It's especially odd when companies seem to fail to realize their entire market is comprised of people leaving larger, more established companies largely because of things like this.

The complaint seems unfair. Every piece of commercial software I've ever used has crash reporting. You can't develop software without knowing how it breaks.

Even Debian has popcon.

I mean, the complaints are a bit rich coming from the very people that dump countless of third-party "crash collection" SaaS and other marketing SDKs into their apps or have marketing insert whatever script tags they want into websites. Where is the personal responsibility gone to?

Chances are, if you are reading this, you are working at a company that does this very thing! For much more meaningless things than enterprise hardware with custom OS.

Crash collection is common, so personal responsibility should be to just accept opt-out dumps in a security device? That's a weird argument. This is a pretty clear cut case as the users have made clear; it's good that they make it so clear because it might change the mentality of businesses in which it's still perceived as perfectly fine practice.

Still not enough. Needs to be opt in. As others have stated, crash dumps are pretty much data leaks.

By pulling stunts like this you force all operators to review the contents of every update. You actually reduce security across the ecosystem because you reduce the likelihood individuals will update.

> For any further questions related to this, please review our EULA, Terms of Service and Privacy Policy.

May as well be: "For more information, please re-read."

Come on.

For any further questions related to this, please review our EULA, Terms of Service and Privacy Policy.

As someone who was literally about to order a whole set of Ubiquiti gear for his business this month, having never been a customer before: no, I don't think I will.

Ubiquiti, you blew it once by trying to push this out in the first place. Now you've blown it again, by thinking some token opt-out button was enough to fix the problem, instead of realising that what you're dealing with here is a major trust issue and you're fundamentally on the wrong side of it. You don't get a third shot. I need to protect the security of my business data, and I need to be confident that the professional gear we're buying isn't going to actively undermine that.

As someone who was literally about to order a whole set of Ubiquiti gear for his business this month, having never been a customer before: no, I don't think I will.

Here's what I don't get: Ubiquiti has always been a bit of a dumpster fire. From the wilful GPL violation that introduced security vulnerabilities to shipping a product based on an end-of-lifed version of Debian to shipping known vulnerable OSS software. Yes, I know that they've (recently) shipped a new version of "EdgeOS" that is based on a supported version of Debian.

From what I can tell the love affair with Ubiquiti was more about the cost than the quality.

Previously I had heard primarily good things about Ubiquiti on the grapevine, mostly about the performance of the WiFi equipment and the ease of use and central management via the GUI. Now that I'm looking properly in connection with a specific purchase, the picture doesn't seem to be nearly as pretty, sadly.

This is a little ridiculous, and it sounds like anything they could have done would have lost your business (not that they lost your business - they never had it in the first case). They made a decision, and rolled back on it almost immediately.

That's not really true. However, we do have very strong views on privacy and security, and in connection with those, on mandatory telemetry. These aren't just on a whim. They arise as a practical consequence of the types of client and data we often work with, and our formal obligations and ethics in connection with those. Ubiquiti isn't special in this respect. We don't have any Windows 10 systems either, for example.

The problem here is only partly the original decision. It's actually much more concerning that they don't really seem to understand why that decision was a problem for some people or that they did anything wrong. How can anyone trust an organisation that would pull a stunt like this and then think a simple opt-out added under protest was sufficient to address the issue?

They have not at all rolled back.

Please let me know what you are going to get instead....

I bet is phones home as well, and probably does not even have a opt-out at all.

Sadly this is almost certainly true. Ubiquity was really the only reasonable player in this market for people who aren't just throwing money away.

No they weren't.

Comments like this, especially when someone in the thread has already asked for alternatives, isn't particularly useful. On the contrary, it is the coward's approach of shutting down other people position but not be willing to stake out one of your own to likewise be subject to scrutiny.

Comments like this, especially when someone in the thread has already asked for alternatives, isn't particularly useful. On the contrary, it is the coward's approach of shutting down other people position but not be willing to stake out one of your own to likewise be subject to scrutiny.

I've already outlined the many ways in which Ubiquiti was not a reasonable player on this and other posts, their deliberate GPL violations that exposed users to added vulns was covered in the initial post about their spyware. I'll take it a step further and posit that Ubiquiti has done little to nothing in good faith.

Their offerings are largely thin veneers over open source software with minimal engineering effort (likely because Ubiquiti has little to no actual engineering staff). Look at the packet corruption bugs in the switches, the overheating, or the RADIUS issues that their WiFi gear has (and Ubiquiti has zero knowledge on how to debug). For fucks sakes Ubiquiti shipped their EdgeRouter lineup with an EOL'd version of Debian for ages (specifically the MIPS support had been EOL'd and Ubiquiti largely didn't build or ship security updates). EdgeOS, of course, is merely a thin HTML based wrapper on top of Vyatta (and an exceedingly buggy one for a while).

I don't think there are a ton of great options (although NetBSD on the Oceteon hardware seems like a good solution) once you've restricted yourself to that price point. Lack of better options doesn't mean that Ubiquiti is good or reasonably behaved.

Ubiquity, for all its faults, is kit I can buy and it works today. I had to look up what the fuck Oceteon even is and it looks like an embedded platform. I can't see any ready-made solution for the things Uni-Fi does, so that means it's roll-your-own. No thanks.

Ubiquity, for all its faults, is kit I can buy and it works today.

Until you enable hardware acceleration and it corrupts your traffic mysteriously. Or maybe you don't want your network gear to phone home. Maybe you don't want to overheat and kill its internal storage. Maybe you want the PoE to be configured in a way that you're going to fry things accidentally.

I had to look up what the fuck Oceteon even is and it looks like an embedded platform. I can't see any ready-made solution for the things Uni-Fi does, so that means it's roll-your-own. No thanks.

If you don't know what the fuck I'm talking about maybe instead of swearing and whining like a jackass you should look it up.

Edit: If anyone else out there is interested in defending Ubiquiti without knowing anything about their products - Cavium's Oceton is a MIPS64 SoC for network appliances that Ubiquiti uses in their EdgeRouter Lite lineup (and potentially others). It's also found in a variety of hardware from vendors like D-Link and SonicWall. It's a bit power hungry (you can cook an egg on an ERL), but is supported by NetBSD and OpenBSD just fine.

Ubiquiti is like the Mongo of networking appliances. They have a lot of checkboxes on their datasheets, but not all of the features are well developed.

While I totally get wanting the option to opt-out of phone home, literally every enterprise hardware vendor has phone home and expects you to have it enabled if you want proactive support. Claiming they need to dump the feature entirely is more than a bit overly dramatic.

The option to disable it was absolutely the right move for them to make and it should’ve been there from the start. Everyone claiming they should not have any phone home at all hasn’t spent much time in the enterprise hardware world...

While I totally get wanting the option to opt-out of phone home, literally every enterprise hardware vendor has phone home and expects you to have it enabled if you want proactive support.

No, they don't. We work with some clients who are in a very similar area, on the development side. Anyone who thought this was a good idea would get bounced out of the room so fast their feet wouldn't touch the floor until they were in the corridor. It is absolutely not the norm to upload data from within the customer's network without their knowledge or consent.

Read more carefully.

Almost all of your enterprise vendors do this today. Some of them even give Sales Engineers real-time access to performance and utilization metrics. It’s a great way to get competitive intelligence about other companies.

You misunderstand. My business develops software, including software that runs on networked devices, and we have never had a client ask for phoning home like that. In fact, it has often been a requirement that the device could operate with no Internet access at all, because a lot of security-conscious customers such as those in finance or healthcare have strict rules about this.

Can you name one?

IBM

Cisco

Dell/EMC

HPe-Aruba

Arista

All phone home. Who is this magical enterprise company that has no phone-home?

This farcical: claiming there are issues with healthcare and finance is complete rubbish. They aren't phoning home patient data, they're phoning home telemetry and health status of the hardware. I can tell you with 100% certainty that both segments have literally hundreds of thousands of hardware devices phoning home to their respective manufacturers without ANY issue.

There is nothing magical about it. If you were correct and no-one made enterprise networking gear without phone-homes, quite a few of our clients' customers wouldn't have a network, since they are strictly prohibited from buying any equipment that does. You seem to be extrapolating from limited experience, possibly of a few specific product ranges or licensing models, and assuming that the whole industry works the same way. It does not.

That’s an awfully long-winded response without actually listing a single vendor. It would lead me to believe you don’t know what you’re talking about.

I’ve worked with basically every vendor in the space and phone-home is table stakes which is why your story doesn’t really hold water...

You've worked with basically every vendor in which space? Surely not enterprise-level networking, where there are hundreds if not thousands of sources for numerous different types of device?

Let me bring you back since you seem to be trying to change the discussion now that you're stuck in a corner. We're talking about vendors that provide wireless networking hardware. Your claim was that any wireless hardware networking vendor that had phone-home wouldn't fly in hospitals or finance. I can literally walk into any hospital in the US and prove that wrong, so I asked you for a list of these magical vendors. There aren't hundreds, there aren't thousands, there are dozens at best at any given moment. Name one that is an enterprise wireless networking vendor that doesn't have phone-home.

We're talking about vendors that provide wireless networking hardware.

You might have been, but the comment that I'm replying to is the first one in this thread to include the word "wireless". Ubiquiti make various other kinds of network equipment as well, and enterprise networking is a vast industry all of its own, of which wireless is only a small part.

I did. Mea culpa

>>It is absolutely not the norm to upload data

Yea it really is, All the major Players in WiFi do it, most even need to talk to a licensing server at regular intervals (which include Telemetry data) and there is no Opt-Out (looking at you Meraki )

Citation very much needed. I do a lot of work in this part of the industry. If you want to even get into the conversation with some of the big financial services, healthcare providers and so on, you need to comply with their standard rules, which might have a checklist hundreds of points long, mostly non-negotiable. You're not going to get those deals with mandatory telemetry no matter who your boss plays golf with.

They need it to be opt-in, such that the hardware I have already bought does not begin sending data arbitrarily upon an update.

Since when does Uni-Fi have proactive support?

That's kind of the point, isn't it? If they get crash dumps they can fix bugs that aren't reported by the (likely very large) install base not savvy enough to realize that there's a bug they should be sending crash reports for.

I have run into this at literally every job I've ever had. Make notes, gather evidence, and file bug reports is just a foreign concept to some people.

One issue is that it's a lot of work that, in my experience, usually doesn't go anywhere. You found the problem and worked around it, gave their people what you had, but they keep asking you to try things and gather more data for 3 weeks while you're trying to get shit done.

> By pulling stunts like this you force all operators to review the contents of every update.

Shouldn't you be doing that anyway?

They say the data is GDPR compliant. If the data is anonymous (and the linked page specifically says "completely anonymized"), the GDPR does not apply, as far as I know. See Recital 26. [0]

> This Regulation does not therefore concern the processing of such anonymous information, including for statistical or research purposes.

Crash data does not mean it contains memory dumps; it could just be assertions and stack traces.

[0] https://gdpr-info.eu/recitals/no-26/

The data includes the users IP address which is personal information. It is not anonymous.

Where does it say the IP address is stored with the data?

Recital 26 says:

> The principles of data protection should therefore not apply to anonymous information, namely information which does not relate to an identified or identifiable natural person or to personal data rendered anonymous in such a manner that the data subject is not OR NO LONGER identifiable. [emphasis mine]

So as long as the data becomes no longer identifiable, recital 26 seems to apply.

in their privacy policy. Section 1b.

The Usage Data that we collect may include information such as ...... Internet Protocol address ......

https://www.ui.com/legal/privacypolicy/

Fair enough. It does say "may". :) It's still possible for other data to be "anonymous" if it's stored independently of an IP address, device identifier, etc. since recital 26 allows for data that is "no longer identifiable".

"May" is synonymous with "will", as far as complying with privacy regulations is concerned.

The moment the device connects to Unifi, without your permission, they have your IP.

Then again, they had it when you downloaded the update, and when your devices check for auto update, so, you’re not getting away from this data collection with an opt out button.

in such case, it might be pseudonymous, but to make it pseudonymous specific effort is required. I'll copy an old comment which explains the terms. ID in this case could be the IP address, device identifier etc...

--------------------------

A person is identified, if the ID references only one user in the whole dataset[1]. This also makes any information linked to the ID PII.

the ID would be pseudo-anonymous if one would need some extra data, to which they don't have access to, for linking the ID to one specific user in the whole dataset[2].

So to answer your question, RTB ID is not pseudo-anonymous as it only references a single user out of all of them.

[1] It's also important to understand the definition of PII in GDPR context. Which is any data that relates to an identified or identifiable person. Identifiable is the same as distinguishable. Knowing this helps to understand where the line is. https://www.lexico.com/en/definition/identifiable

[2] Definition of pseudonymisation, 5'th bullet-point: https://gdpr-info.eu/art-4-gdpr/ sheds some light on this.

I didn't mention anything about GDPR.

I don't want any data to be sent to Ubiquiti. This forces me to now treat my network hardware as an untrusted device.

I'm confused how they keep saying this is in compliance with Grpd when auto opt in is not allowed.

And the idea that they force everyone in, and will add an opt out button later seems like they are gaming the system.

If they don't opt everyone out they didn't ask to opt in as a default, their actions to me seem hollow.

Most likely because as an American company they have no clue what the GDPR really does. Many European companies are still struggling to be fully compliant.

Furthermore, if they transmit crash logs, and those contain partial memory contents (core dumps), chances are actual content from network packets are transmitted to UBNT as well, and those contents may contain PII (IP addresses, MAC addresses, usernames, passwords, real names, addresses, credit card information, etc.), meaning they’re most certainly violating the GDPR by making it opt out.

With a significant fraction of Internet traffic (and all eCommerce traffic) end-to-end encrypted, is it really realistic that they would have sensitive data (usernames, passwords, real names, addresses, credit card information, etc.) in memory of an AP? This is the point of Let's Encrypt, after all, to close the exposure that we give to our ISPs and our ISPs ISPs.

Macs tend to use your name as their network name from what I've seen, and corporate logins usually are some variation of your first and last names. The APs absolutely know those.

But APs are not limited to internet traffic.

Maybe some company somewhere runs an internal server, and because it's internal they might think that encryption is not needed. Hell, it might even use GET arguments for username/passwords in the URL.

The crash dumps would contain these data as well.

AFAIK ubiquity also does routers, those can hold quite some sensitive data in memory.

Don't think sensitive data, think personal data. An IP is personal data.

I wouldn't assume "crash logs" means "core dumps". I'd assume they mean stack traces, and perhaps register contents.

And they never hold IP addresses in registers?

Oh come on. It's extremely unlikely, and there'd be no reliable, automated way to extract them anyway.

You can absolutely automate retrieving information from a core file. I'm not saying that UI is going to purposefully extract sensitive information, but of course they'll setup automated processing of the core dumps in order to categorize the types of crashes into buckets so that engineers can examine them more closely.

Next time you come across a core file, use strings on it - you'll be surprised what it will find!

We were talking about registers+stack trace, not a core dump.

Anonymous data is not subject to the the GDPR. See recital 26. [0]

IANAL, but as far as I know, companies can collect anonymous data ("what versions are our customers running?") without involving the GDPR.

> This Regulation does not therefore concern the processing of such anonymous information, including for statistical or research purposes.

[0] https://gdpr-info.eu/recitals/no-26/

the thing is, it's pretty difficult to guarantee the data is indeed anonymous. In this specific case, as they are not willing to explain what data is actually processed and say it's somewhere in the Tos, privacy policy[1] etc... we'll take that as the gospel.

----------------

The Usage Data that we collect may include information such as your device data, including your mobile devices, sensor data, device signals, device parameters, device identifiers that may uniquely identify your devices, including your mobile device, web request, Internet Protocol address, browser type, browser language, referring/exit pages and URLs, platform type, the date and time of your request, and one or more cookies that may uniquely identify your devices or browser.[1]

----------------

This data is certainly not anonymous by GDPR standards. Identifiers that uniquely identify a device will in some cases(ex, single-user devices) also be unique identifiers to a single person. That is enough for the data to be considered Personal Data in the context of GDPR.

‘personal data’ means any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person[2]

[1] - https://www.ui.com/legal/privacypolicy/

[2] - https://gdpr-info.eu/art-4-gdpr/

I bet they will not actually be GDPR compliant. They will likely get away with it because enforcement is lacking, but anonymization is hard to get right and I would be surprised if they did get it right.

Is there a way to file GDPR complaints? Enforcement might be lacking but with enough data and support surely it would be possible or be sufficient to make Ubiquiti move to opt in.

contact your local DPA[1] and file a complaint. There's plenty of info on the original thread[2] on what they're doing wrong.

[1] - https://ec.europa.eu/info/law/law-topic/data-protection/refo...

[2] - https://community.ui.com/questions/UI-official-urgent-please...

There is no such thing as completely anonymous data. If the data is of interest is has some uniquely identifying feature. Even anonymized data always contain some bits of information that can be correlated to de-anonymize a user.

Opt in is only one way to justify data collection and be in compliance with GDPR.

They may be basing their collection on ‘legitimate interest’ for instance.

I don’t think anyone is in a position to predict these things but I’d bet they have a pretty good justification there for IP collection given the product they provide.

given that there are very little references to GDPR in their ToS and privacy policy I'd say they haven't even thought of it properly.

Also in the original thread someone quoted this cool tidbit from their ToS .[1]

> Special Note to International Users. If You are accessing the Services from the European Union, Asia or any other region with laws or regulations governing personal data collection and disclosure that differ from United States laws, please be advised that Your continued use of the Services will be governed by United States law

this makes it seem even more likely.

[1] - https://www.ui.com/legal/termsofservice/

I hope Ubiquiti (or one of their lawyers) reads your comment.

Contrarian opinion/question: Out of the software developers reading the comments here: Who is offering an op-out (much less an opt-in into) from their crash reporting solution they have included in their software?

And of those who do actually offer an op-out: Who is annoyed about not being able to debug an issue because the crash they know is happening only seems to be happening on clients who have opted out?

And my second question to all the non-developers out there: Who here is annoyed about crashes on their machine that "never seem to get fixed" as if the vendor "didn't care about getting crashes fixed" in software that offered and explicit opt-in and they didn't opt into "spying" by the vendor? Who remembers not opting in and remembers to rectify that when the crash happens?

It's very easy to stand on the hill of righteousness and yell at a vendor implementing crash reporting, but before doing so ask yourself: Are you collecting crash reports? Do you wish you had crash reports? Are you unhappy about crashes on your machine not being fixed?

If any of those are true, then you better not complain about a vendor adding crash reporting to their products.

Do I think this should be opt-outable? Of course. Do I think crash reporting needs to be opt-in? No. Absolutely not because next to nobody will opt in and when crashes happen, nobody will remember their decision.

Make it easy for your customer to get a crash report when one occurs. Make it easy for them to contact your support, whether it is a Github issues page or a professional phone support desk.

Make the report readable by a simple text editor. Let the users have a look at the content inside. If you do need the report, ask them to check it then send it through trusted and encrypted ways like Firefox Send.

There is no need to make an crash report automatically sent to you. No need to fix crashes for users that never cared for reporting them to you. Orient your bandwidth to users that care.

> Make the report readable by a simple text editor. Let the users have a look at the content inside.

how do you explain the concept of a backtrace to an end-user so they can make an informed decision whether it's fine to submit the data or not? How do you do the same with a a full memory dump (if you need one?). How do you expect users to analyze the memory dump for potentially compromising data?

> If you do need the report, ask them to check it then send it through trusted and encrypted ways like Firefox Send.

this works for you and me. It doesn't work even for my coworker. Also: What is the advantage of going through the extra hoop of Firefox Send compared to over plain SSL when you intend the recipient to be able to read the dump anyways?

> No need to fix crashes for users that never cared for reporting them to you

Judging from metrics available to an app I'm working on I would say that users are much more likely to stop using my app completely rather than even bothering to press the "report this issue" bug.

It really depends on your target audience.

But in case of UniFi (as opposed to their AmpliFi line), asking for individual reports to be reported could possibly be feasible given the product audience.

Save the crash report, show me a notification where I can review the data and then decide whether to send it or not.

Yes. Getting this right is that simple.

Where do you show the notification when an AP crashes?

How do you handle users not familiar with the concept of a crash? How do you make sure you get their crash reports? How much are you willing to spend on dealing with edge-cases (like crashing loops and reporting fatigue)?

But yes. I agree that your proposed solution is the friendliest for the subset of total users who are also frequenting HN.

Problem is also that they don't even give you an opt-out button yet:

> If you do not wish to participate/provide this data, we will add an opt-out button in upcoming versions that will make it easy to opt-out of providing this data. In the meantime, you can block traffic from UniFi devices to trace.svc.ui.com.

Wonder what the hurry was to launch such a "feature" that couldn't wait for the opt-out button to be ready. In the meantime they will definitely be able to collect some data from users that still wait for the opt-out button.

The notification can be shown in the unifi controller, and optionally send an email. If you are using the controller you would be expected to understand the concept of a crash report.

If a crash occurs, you can ask to send a report. Maybe after a restart of the application. This is the way I do it and many other software vendors as well. It is a non-excuse for default active telemetry.

I've been saying for several years that, eventually, we will all end up having to put in "default deny" rules for our outgoing traffic, just as we do today for incoming traffic.

(Either that, or hosts will have to be configured without a default route and will need to use a proxy server for all outgoing connections.)

When every piece of hardware and software you use is spying on you, blocking everything by default and only allowing explicitly whitelisted traffic will be your only option to stop it.

I think we're already there.

Site I manage run separate VLANS for admin (switches, APs, etc) and Internet-of-Things, and both of those VLANS are default deny on outbound. That rule blocks a fair amount of traffic.

I wonder if part of the answer is better tools in firewalls for this sort of thing. Easily tagging "unstrusted device" or something perhaps.

Pihole, a router that forces anything on port 53 back to the Pihole and careful device selection seems to be very much required already.

My privileged-access work laptop does that today—whitelisted applications and whitelisted network access.

We need to start intercepting their traffic and redacting the information we don't want them to have. Just like how corporations basically MITM everyone in order to inspect outgoing traffic and prevent data leaks.

If the thing needs the internet to work, we should be able to whitelist just the data that's essential to its function and have the firewall automatically strip the remaining data from the packet.

On the level of the individual machine (as opposed to your router), this is basically what Little Snitch does.

For engineers reading this who work at a tech company, or well, any company. Watch this keynote talk by Patricia Aas.

It's called Embedded Ethics. And it is crystal clear many of you need to watch it.

https://www.youtube.com/watch?v=HfNIiitVFtcalk

Opt out is good. But I just want to weigh in for the statistics that I have almost zero concerns about anonymous crash reports. I worry a lot more about whether products requires internet connections to their manufacturers to work ten years later, than whether they send home crash data.

If it’s clearly indicated and has an opt out I’m fine with it. I’d go so far as preferring it over opt-in, because I want my product to have the improvements of as many crash reports as possible.

Obviously, by “crash log” I hope they gather call stacks - not memory content, since memory content could contain data that I wouldn’t want collected.

Their "protect" camera software doesn't work properly without it being able to connect to the net. It won't let you login on the localhost!

Now this. What is a good alternative for AP's and cameras?

After trying several inferior solutions, including the Unifi cameras, I switched to Blue Iris in combination with high quality Dahua cameras (IPC-HDW5231R-Z). It's been very reliable and overall just works great.

Commercial wired cameras that write to a bunch of hard drives?

I'm glad they will add the option, but considering their user base background it was really naive to don't foresee this was going to be detected and that it was going to be an issue.

Does anyone know if they've announced the same thing for their EdgeMAX gear?

This is really dumb, these are the kind of shenanigans that people buy non-consumer gear to get away from.

Safe assumption is they'll add it eventually to all their hardware. But given the reaction they got here, they'll get the opt out setting done before they spread it to other devices.

Thanks for your thoughts. I’m hoping that the more “serious” bent of the EdgeMAX stuff (much of the functionality requires dipping into CLI) will cause them to hold off there. I’m assuming most businesses would find this unacceptable. But maybe you’re right.

Well, I think the amount of CLI config required, means most EdgeMAX users won't think much of having to add a "disable telemetry" command in their configs: They already have to add like all the basics manually to begin with.

But the other thing is, since UNMS devices like EdgeMAX and AirMAX are intended for mass deployment by ISPs, there's a decent chance UNMS will reveal and aggregate that telemetry data and allow UNMS users to see it themselves for their own troubleshooting and diagnostic purposes.

Neat, thanks again, you clearly have dug more into these devices than I have :-) I haven’t tried UNMS (mostly for multi-site deployments/massive overkill for home use?) but hopefully that means that it’ll be fairly easy to disable.

Power outage corrupts Mongodb in the UniFi Cloud key. They won't fix that.

There is no option to remove the dead/unplugged devices from the device list. The only way to remove is to reset and reconfigure everything from scratch.

> Power outage corrupts Mongodb in the UniFi Cloud key. They won't fix that.

The Cloud Key 2 has a battery inside it and shuts down gracefully if you pull the power.

So that's a kind of fix. Although it does involve buying a new device.

Avoiding dedicated hardware and running it on a machine with a UPS is another way. I can recommend Docker, Synology and a small UPS. And that’s how the equipment cost got multiplied by 10.

How about the software controller that installs to a users profile with no way to change the install directory on Windows

I have 2 cloud keys and have suffered power outages on both multiple times. Never had any issues with corruption.

Re: your second point... I just removed an unplugged switch today. Maybe it’s a new feature?

I posted this as a reply to a comment, but I will repeat it here,

If they transmit crash logs, and those contain partial memory contents (core dumps), chances are actual content from network packets are transmitted to UBNT as well, and those contents may contain PII (IP addresses, MAC addresses, usernames, passwords, real names, addresses, credit card information, etc.), meaning they’re most certainly violating the GDPR by making it opt out.

The GDPR doesn’t care how you came by the PII, it only dictates how you should behave when you handle it.

Edit: I should clarify before somebody trips over their pitchfork and sets the house on fire, the GDPR cares a lot about how you come into possession of PII, and also if you should have that information in the first place, and also cares about how long you hold on to it.

When it comes to protection of those data though, it doesn’t care if it was collected through telemetry, emailed to you (yes, also the spam folder), or it’s an old archive of core dumps on an old server somewhere. The same rules apply to how those data are handled.

This is simply not true. Opt in is only one of many ways to be compliant with GDPR covered data.

Legitimate interest is the obvious way to not require opt-in and while I’m sure the regulators are getting tired of seeing ad networks claim that exception a router company claiming it for crash support seems like something that will likely be deemed in bounds. Especially if that company has made a good faith effort to show its not repurposing that data.

that would only be valid if crash support is requested and provided. As far as I know ubiquiti is pretty hands-off in terms of support, so the case isn't too valid.

>This is simply not true. Opt in is only one of many ways to be compliant with GDPR covered data.

And yet, Article 32 (https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELE... the GDPR states:

    Silence, pre-ticked  boxes or inactivity should not  therefore constitute consent. 
    Consent should cover all processing activities carried out for the same purpose or purposes. 
    When the processing has multiple purposes, consent should be given for all of them. 
    If the data subject's consent is to be given following a request by electronic means, 
    the request must be clear, concise and not unnecessarily disruptive to the use of the service for which it is provided.

It is of course also possible that UBNT is fully GDPR compliant if they can guarantee that the don't send PII. The problem is we don't know.

You are misunderstanding. You only need to collect consent via opt in if consent is your basis of collection.

There are several other basis as well including ‘legitimate interest’. You can see all of the basis in article 6(1).

If you use a different basis you don’t need to collect consent at all whether opt in or otherwise.

Hardware data from crashes used just for cause analysis and properly stored/secured is a fairly straightforward ‘legitimate interest’ argument.

Does anyone else observe hourly connection attempts to "unifi-report.ubnt.com" from their Unifi Controller? I'm running the controller locally with all cloud features, error reporting and live support disabled. Not sure what other options user-facing options I can turn off to stop this from happening.

The biggest concern for me is not opt-in vs opt-out, and not even data mining. They create another side channel which can and eventually will be exploited by the party they can't imagine now. They weaken the security of their otherwise good products.

"transmitted using end-to-end encryption"

Technically correct but sounds either ignorant or misleading.

In other words: over HTTPS.

All encryption is end-to-end if there's only two ends.

For "enterprise" hardware/software I would expect them detailing exactly what metrics they collect and what could be in a dump that gets sent.

How do you anonymise a dump with user info especially for a routing device? Like that requires some sophistication, you'd imagine they'd love to talk about it to generate blog clicks.

But it seems that some keen user will have to try reverse engineering their data collection to actually give the users information of what is leaving their network.

As a user of their products, it has really soured me on their product. When supposedly if they're so "GDPR compliant" they just need to explain themselves to win people over.

It's unclear from the post, does this impact the firmware on my unifi devices, or is this part of the controller software? ie. is my raspi phoning home, or are my wireless APs phoning home?

Most likely a follow up on this posted 22 hours ago on HN: https://news.ycombinator.com/item?id=21430997 (Ubiquiti adds phone-home to the access point firmware)

This has already been added to the excellent host block file of StevenBlack.

https://github.com/StevenBlack/hosts/issues/1083

I do not understand how somebody at UBNT thinks the data is worth the hassle. They are taking a reputation hit, risking getting PII and getting rolled over by the GDPR steamroller, angering some of their best customers (corporations with strict security rules). What for? Is the crash data really so valuable? Do they intend to pore over every report with dedicated manpower? (they really should if it matters so much).

I would say they should first get their basic support to work in a top-notch manner, carefully reading, thinking about, and responding to every bug report. Then get into automated data collection, and then ASK FOR USER CONSENT. This isn't difficult. No "opt-out" crap or corporate lingo, TELL ME CLEARLY WHAT YOU NEED, WHY YOU NEED IT, AND ASK ME IF I AGREE.

"If you do not wish to participate/provide this data, we will add an opt-out button in upcoming versions".

Reply on news.ycombinator.com