Hacker News Reader

This is similar to law enforcement saying: "You mean we cannot track everything you say and do. How in the world are we supposed to do our jobs safely and efficiently?" Answer is, you basically can't. Not safely and not (as) efficiently. Because the world will turn dystopian and will destroy the very thing we are trying to create/maintain. (Happiness, etc)

It's nice to be able to A/B test your blog or product. It's cool and efficient, but it also hurts and to me that hurt outweighs your company's marginal benefits although it is a nuanced and difficult discussion to be fair. I'm being simplistic here to make the point clear. I hope.

> it also hurts

How is simple analytics or A/B testing that's NOT internet-wide tracking (that is, only for the website you're on) or sold (which would be outright illegal without explicit consent) hurting you? Genuine question, because I don't see it. Internet-wide tracking across many sites: sure. But that's a very different thing – it's the difference between "I'm home Darling, I saw Sander at the mall today" vs. "Hello everyone, here is everything Sander did this week".

It's like like being issued a state mandated always online tracker and then police would say verbally that they promise not to abuse that data.

And before people would nitpick my comment, metadata is surveillance. https://www.schneier.com/blog/archives/2013/09/metadata_equa...

I bet HN keeps server logs; would you say you're being surveilled by viewing this page and posting that comment? If not, why not? What's the difference between that metadata and any other?

Surveillance is a deliberate action. If some incidental metadata is temporarily stored at HN and then evicted and deleted permanently in reasonable timeframes then it is not a surveillance. But as soon as it's analyzed, and/or specifically stored for prolonged periods, and/or unnecessary metadata is collected (not required for basic tech support and server administration), and/or linked to the user profiles more than needed to save and display our posts - then it's surveillance.

If I passed by your house and saw you walking your dog at 10:34 18/05/2023 and then promptly forgot about it, it is not a surveillance. If I set up a permanent observation post across the road from your door and will tail you daily everywhere you go - it's a surveillance. Even if I pinky promise that it is my own internal information and I would not share it, sell it, exchange it etc.

> set up a permanent observation post across the road from your door and will tail you daily everywhere you go

But I don't really see how that is happening here? I agree that would be bad, but merely checking "a user visited my website" is a very thing than seeing what someone does all over the interwebz. It's very questionable this minimum of information can even be tracked to you when done right.

I'm not claiming the relation is causal. I don't know what happens between happy world -> happy world with constant surveillance -> dystopian hellscape. The same is true for HN logs. I'd say, yes those logs are a form of surveillance. If that matters is where the debate is. It's hard to say, because these things are slippery and take (a long) time.

Of course they’re being surveilled on Hacker News. Keeping a finger on the pulse here is the epitome of low hanging fruit for law enforcement and intelligence. Why would anyone suspect otherwise?

The data is that of "I have a person who likes monkey paws" and then they can sell that to the monkey paw conglomerate. Next thing you know you're getting monkey paws in the mail!

The entire premise of my question is that data is kept private between the two parties (the website owner and the visitor). I don't see how this connects with that at all.

And how do you even get someone's mail address without an extensive internet-wide profile (which, again, isn't what the premise of my question was in the first place)?

It’s one thing to say that you’ll keep the data private between you, but from the user’s perspective there’s no guarantee. Whether that’s because you change your mind and decide to sell the user’s data, or there’s some sorta data breach, or or or.

I get the feeling. But to make things very simple, it's again like law enforcement tracking everything you say and do. Let's say it's a single agency, with a single employee doing the tracking to keep it easy.

How is that hurting you? How is your every movement being tracked by a single agency such an issue? They said they would keep it private, pinky promise.

Law enforcement tracking everyone can have serious and unintended consequences in away that a SaaS tracking visits are unlikely to have.

An extreme case is on the other side, is to compare simple site limited tracking to a coffee shop being able to see who enters the door. Why should they track how I look and the way I'm dressed? Does it make service harder if you have to work blindfolded? Though luck, my privacy is more important than your wish to provide your customers with good service.

“Unlikely” being the keyword here. I know it’s unlikely, but just the act of collecting all this data makes it dangerous. IMO the benefits are small compared to the longterm risks so I lean towards, don’t track, just deliver “good coffee”. No need for hyperoptimalization of the coffee experience.

In your example they are only looking. It would be different if they were recording their observations in a structured manner (aka cameras). Even writing it down by hand would raise flags, would it not? Sure, being a human means looking at me and my coffee to brew it, but recording everything I do in (semi)permanent storage? Is that human? Is that necessary to brew coffee?

On it's face it is harder for baristas to be blindfolded. It would literally be a worse coffee shop at blindfolded baristas than normal coffee.

Yes, but that's my point. A large number of companies uses analytics data to provide better service and products without any sinister motives.

The difference is that keeping cookies is a user choice.

You can set your browser to not store cookies at all.

Or to discard cookies when you close it.

Or you can delete cookies whenever you feel like.

The problem is that there are good and bad cookies:

- session cookie: good

- tracking cookie: bad

I remember a time when browsers would ask you for every website if you wanted to keep the cookies and because most cookies were good cookies, everybody agreed to accept them. Later came the ad industry and started to introduce bad cookies.

My violin is probably small-medium sized for this problem.

You can talk to your users in person and ask them, or poll them via email. Do usability tests etc. I guess it just costs more.

Other industries have to do this, they can't just default-spy on their customers.

Yeah so I run/build/maintain a platform in the public survey space and we have a feedback box for users at the end. This box is a really good source for feedback, don't get me wrong, but a lot of the feedback is "When I hit the button, nothing happens" or "Totally broken, completely unusable" both of which are useless to us. These users clearly want us to fix something, but we have no visibility on what that is.

I'm 85% sure the problem they have is a problem you can't fix.

> or poll them via email

Marketing survey emails are infinitely more annoying than cookies.

Have you ever actually responded to an email poll and not just unsubscribe/block the emails?

Yes

> You can talk to your users in person and ask them

doesn't scale.

> poll them via email

doesn't work.

> default-spy on their customers

analytics is not spying.

>doesn’t scale

Yes it does. Have you never heard of a survey?

>doesn’t work

It absolutely can. For instance, polling via newsletter is rather effective.

>analytics is not spying.

If it contains any personally identifiable information that is not required by law, then yes. Yes it fucking is.

I'm not in agreement with the top comment's cookie needs, but no. Surveys are not comparable to using analytics/tracking to determine user behavior.

Surveys measure what people say and are sporadically filled out. Cookies measure what people do with granularity and accuracy and will have near 100% coverage.

People aren't concerned about spying via survey because they're not even in the same realm of effectiveness.

> If it contains any personally identifiable information that is not required by law, then yes. Yes it fucking is.

wahwahwah

>doesn't scale.

Much of the math behind surveys is based on finite samples taken from infinite populations. It scales really, really well. You just have to make sure your samples aren't totally biased, which is a little more difficult. But there are ways to reduce those problems, if you're aware of how the bias is introduced.

You can poll them via pop ups.

Ive seen this done a couple of times. The sites took care to make it unobtrusive, sensibly placed and with short, meaningful questions.

Frankly I think the data they got drove better decisions than analytics.

I liked that they let you write a message too.

I like this method. Earlier today I filled in some gov forms online, and at the end it asked if I'd like to do a short survey on my experience - I was happy to oblige, because I have a personal interest in service improvement.

you don't need scale, you need good data

Good data requires statistical significance which needs scale

you only need 5 people to test poison

This is all bullshit. We literally did this sort of thing for years on the Internet before all of this tracking was common.

Every company claims that they are doing this stuff to make things better for users, and it never is.

Genuinely asking - what is the easiest method for doing this other than cookie tracking? I’m inclined to agree with the commenter above and it feels like a community of developers would sympathize with this but it seems like everyone is acting like it’s not a big deal.

> We literally did this sort of thing for years on the Internet before all of this tracking was common.

You mean back when your average website was hot garbage? The yearning for the days of static HTML pages is childish atavism

The average site today is hot garbage. But now it requires several orders of magnitude more bandwidth and processing power to render. Not sure this is progress.

You are literally using a website that was created by someone who started an online storefront app in 1995. Amazon started around the same time. Google a few years later.

Who said anything about static HTML pages? Keep beating that strawman though.

I'm not saying it's easy or perfect, but Wikipedia does tracker-less analytics, and yet still produces https://stats.wikimedia.org/#/all-projects .

We do use cookies for part of the analysis, but they're not unique user-tracking cookies. Instead there's one that simply tracks a self-reported datestamp on when the user last visited, which looks something like:

WMF-Last-Access=2023-01-01,Expires=Wed, 01 Feb 2023 12:00:00 GMT

(we send a Set-Cookie for this with the current datestamp only to the 1-day accuracy, which expires ~32 days after it's set (but rounded to 12-hour accuracy), and is replaced constantly).

There's another more-recent one we use that's explicitly about differential privacy, which send back info on the hashes of the 10 most recent URLs you've visited on the site, IIRC. None of them are unique tracker hashes for a given user, though.

>which send back info on the hashes of the 10 most recent URLs you've visited on the site, IIRC. None of them are unique tracker hashes for a given user, though.

That seems highly unlikely to never be unique for some users.

The hash is keyed and rotates once a day, so it's not a long-term history. And in any case, nobody's trying to build a unique tracker out of it on the other end, which is the main thing (even in our internal analytics, we even throw away the PII we get directly from request attributes after 90 days).

You could ask people! There is hardly a site I use day to day that I couldn't send a ten paragraph rant about poor usability to the company, if that was socially permissible and felt like they'd care. There's hardly a site I use where I think the developers have ever used it themselves, or ever seen someone trying to use it, for that matter.

> "Because I can't see a user journey."

Today's worst offender was trying to follow an invite and register an account within a company account, it took me two goes through the signup form, two attempts to edit my profile, trying to login to three different subdomains, one useless search of their documentation site, three error messages, two rounds of asking my coworker to check their admin side, before I saw the right thing to do. "You" (big silicon valley company) don't need tracking cookies to "see a user journey" or to tell you that "I really love your site because I keep coming back to it and looking around", you need to grab someone in a hallway and push them through the workflow and watch them fall on their face over and over.

> "I think for a commercial site, where a degradation of 10% in user experience can tank the business"

Imagine how amazing websites would be if that were at all true. Have you seen the user experience of Amazon? or eBay? or Facebook? or 'new' GMail or new Reddit or non-websites like Teams or WebEx?

Oh no. But that’s how all desktop programs without telemetry work.

> and it is bad for everybody

Hang on, it's not bad for me. If I visit your site, it's not because I want to participate in some kind of A/B testing (whose results you'll never tell me about). And if your site only works if I happen to have hardware X installed, you don't need analytics to tell you that your site is broken.

> European companies need to bug all users and beg for cookies.

That's nonsense. If cookies are needed for the correct operation of the site, then there's no need to beg or bug.

So your banner should be saying: "Can I please set tracking cookies that make no difference at all to the correct operation of the site? [Accept] [Reject]". Then count the number that say Accept, note that the number is approximately zero, and then scrap the banner.

> If I visit your site, it's not because I want to participate in some kind of A/B testing

Nobody wants to be part of the A/B testing; everyone wants the polished product that's the result of A/B testing.

> And if your site only works if I happen to have hardware X installed, you don't need analytics to tell you that your site is broken.

What? Yes you do, or something similar.

> If cookies are needed for the correct operation of the site, then there's no need to beg or bug.

Strictly needed. Suppose I have a dark theme slider, or a language selector, for users who don't want me to follow their OS's settings (or can't or don't know how to change their OS's settings). That's a nonessential cookie which requires the banner.

And if you disagree, your opinion is not worth a lawsuit; the only way to be sure that your use of cookies is limited to those strictly necessary is hire a very expensive european consultant.

> everyone wants the polished product that's the result of A/B testing.

I'd prefer a polished product that's the result of communicating with users.

A/B testing is the reason why websites get “optimized” for the wrong target values, typically “engagement”. It’s what drives the enshittification of the web. It’s not what users want.

If you paid for said site, sure.

If you didn't pay, I'm not sure why you feel entitled to anything.

Why would you need to know anything but hits/month and ad-revenue/month?

Why don't you just hash the IP-address and count unique users that way?

He gave several valid examples in the comment you're replying to.

that's easy: to improve the service.

This is the generic reason that is always given, but I don't think I've ever seen exactly how the service will be improved and why cookies (or any uniquely-identifying data) are the only way to achieve the desired outcomes.

I'm sorry, but have you tried making and hosting a website? Product Analytics is a very popular category for a reason.

Many, many times :)

My point is that it's never described. Even in the detailed options for those cookie banners that permit you to tweak things. Sure, there are (some) details about "our partners" and advertising etc, but exactly how the service will be improved is never explained.

The reason is simple and obvious: nobody knows. Companies collect data in the belief it will be useful in improving the service, but generally chuck it into a data-swamp and occasionally rake it over to extract basic info like navigation routes.

I'm also rarely convinced that improvements can't be better determined by focus groups and other similar methods.

"Improving our service" is a glib catch-all that rarely stands up to scrutiny.

There may be specific examples where a cookie is genuinely the best method to improve a feature -- in which case: name the feature, list the metrics, declare success/fail criteria, and stop collecting the cookie after the decision has been made.

Edit: typos.

Okay, how? Provide details, thanks.

>hash the IP-address

How would that work? I can't think of any approach where getting the original IP back from the hash isn't trivial.

you dont need to get the original ip back, just need to know how many unique ips are there, so sha(ip) is good enough

With little over 4 billion IPv4 addresses.

From a stackoverflow post from 12 years ago:

> I know I do 622 million SHA-256's per sec on a Radeon HD5830.

Which would take around 6 seconds to brute force a 32bit address space.

you can just salt it with some stable random thing

In which case you can just take 6 seconds to generate all the new hashes and build a new lookup table.

You can further add bucketing, and eventually move closer to FLoC.

But this is aside the point, as the spirit of the law only allows "processing for legitimate interests". The use of technology, cookies or on the server is irrelevant. If thread OP has evaluated their collection[0] as legitimate, they can use whatever technology within guidelines. Otherwise, even a cookie less data collection would require consent.

[0]: https://ico.org.uk/for-organisations/guide-to-data-protectio...

I apologize for the double negative. What I meant is that hashing doesn't improve privacy because if you know the hash and the hashing function it's easy to build a hashmap of all the possible IPv4s (around 3.5B). Unless the hash uses some sort of expensive key derivation function, but that doesn't scale.

You could simply salt the hash, though you'd need to treat the salt as a secret.

Alternatively, you could use a new salt every day, which would only allow you to track an individual for a 24 hour period (likely enough for many).

?? sha256 the string and you are not going to be able to get back to the original from that output.

Edit: The small amount of IP addresses makes it easy to brute force through all of them.

The hashing doesn’t matter when IPv4 has such a limited dataset. IPv4 has a little under 4.3B addresses, and a cheaper GPU such as the 1080TI has a hash rate of around 4300MH/s, so it crushes that in a few seconds at most.

From there, you have a direct correlation between the IP and its resulting hash. Meaning you can easily see what the original input was.

You don’t need to break a hash to know what the original input was.

ipv4 space is very limited and you can easily compute all the hashes. There is salting and combined with rotating salts it could work but no one guarantees that you’re not storing them

UIs were developed in the pre-web world without tracking or analytics. If you clearly explain that the cookie is for improving the UI and the user doesn't want it, it's because they don't care about it as much as you do.

Not really cookie-less and it can be considered a regression compared to cookies because of JS, but something that can work is doing stats + anonymization client-side.

You can store information client-side, without sending them over network, but randomly send digests back to your server.

For instance you can store a counter of the times the user went to visit the website, and randomly with a 1% probability send that counter to your server. (It's better to make it random, because if you send every +=1 you would end up being able to track users).

At my work, I do a lot of statistics of user usage, but I always work to do my best not to leak PII. I'm not a security or privacy researcher, so my work is probably not great but still, the way I do it I believe is largely private:

- No unique ID sent, but a daily digest (some people send every single event to their statistics server, and thus need a unique ID to know how many time one person did one action. With a digest that already counts the actions there is no need)

- bucketized persistent data: for example the available storage size of the device the app is running on. Sending precise value would make it easier to track digests from one day to the other and track users

- For booleans, add some white noise (because 20 booleans is enough to identify someone)

- For open-ended information (for instance the list of countries contacted by your SMS app), booleanize it (one boolean per country, cf previous line), and maybe keep a counter to know how many you didn't take into account to know whether you're still missing a lot.

Yes overall doing it with no PII requires much more work, but then Big Tech (and smaller techs like Clearview) clearly showed that any PII can and WILL be used against their users. The best way to never leak user's data remains to never have them in the first place.

IP address + User-Agent gives you enough data to track "user journeys" whatever this is. I'm curious, what is your favorite example of improving your website based on cookies that you couldn't do with basic IP+UA tracking?

How about tracking the users pulse while he reads your website via his smartwatch. Wouldn't that give you even more insights into the emotional journey?

My point is: You have to draw the line somewhere (and I think the GDPR line is very reasonable). If you are a business relying on having a perfect website, you can use other means like UX labs.

Sure you can. Just stop crying because you can't share data about your users with 3rd parties without asking them first and get over it.

For some reason my empathy-o-meter isn't really registering your plight. Look, the needle isn't budging, not even a bit.

> I cannot test new features and see if it makes users happy so they come back more often.

Users coming back more often does not imply that they are happy. There’s no substitute for actual user studies.

> I can't even say how many users this site has now.

> I cannot test new features and see if it makes users happy so they come back more often.

> I cannot see if the site has issues on some hardware, software, language. Because I wouldn't see if users affected come back less often.

> I can't test if an introduction text at the beginning helps users discover important features. Because I can't make the connection between showing the text early on in the user journey and usage of features later on. Because I can't see a user journey.

Why don't you just ask your users about these things ?

But it's not a question of PII, it's about processing for legitimate interests[0]. If you follow the guidelines then collecting and processing specific anonamised stories is not a problem, cookies or not.

[0]: https://ico.org.uk/for-organisations/guide-to-data-protectio...

Local Storage?

Cookies are just a Dumb Implementation and only persist due to advertising.

GDPR is about tracking, not about cookies.

GDPR isn't "about tracking", it isn't even about computers, it's General data protection regulation. If your local grocer keeps a regular customer's record on paper to give you discounts, that's covered by the GDPR.

See the UK Gov summary site here: https://www.gov.uk/data-protection

Or detailed law here: https://www.legislation.gov.uk/ukpga/2018/12/contents/enacte...

And note Part 1 section 3 points 4 and 7: "'Processing', in relation to information, means an operation or set of operations which is performed on information, or on sets of information, such as— (a)collection, recording, organisation, structuring or storage" and "“Filing system” means any structured set of personal data which is accessible according to specific criteria, whether held by automated means or manually".

You can't use LS or anything like that to get around GDPR.