Tech-savvy audiences block Google Analytics | Hacker News Reader

Tech-savvy audiences block Google Analytics

plausible.io

1,214 pointsrobin_reala681 comments

There is a German movie about the system that is used to gather TV ratings. It's a special box that some users get which reports what they are watching. Small sample size goes into a big statistic (not sure how accurate the portrayal of the system in the movie is). These boxes are given to the people who pay the German public TV fee, which excludes i.e. students (they don't have to pay) and some other groups. This group of critical people figured that out and started to hack into these machines to fake ratings. They faked the ratings away from stupid trash TV towards some higher quality stuff, documentaries, culture, ... Obviously in the movie then the country saw a renaissance, everyone got smarter, yadda yadda, you get it.

I feel like this is similar. All tech savvy people block ads and analytics and at least the known tricks they use against us. So the internet only tracks the defenseless people and is then built to serve them (and or exploit them).

Maybe we should engage in large scale AdWords fraud. Send come fake traffic away from Facebook and over to Wikipedia.

You mean something like AdNauseam?

https://adnauseam.io/

Edit: it's an adblocker that is supposed to click on EVERY ad that it blocks.

What GP describes would be stronger than AdNauseum. Instead of sending clicks indiscriminately for every ad, you would send clicks for high quality content.

That'd just end up costing the high quality content providers money. Ads wouldn't just get cheaper for them.

On the contrary, clicking only on trash, and having the high quality content have a much higher signal-to-noise ratio would have the desired effect.

Maybe.

The idea is not so much of clicking on high quality content ads, but sending visits to high quality content pages and click on the ads of those pages, to get them more money. Would obviously stop working if the sales after the ad click are not made.

But, hey this isn't a gameplan, this was just a "what if" :D

That could work, or indeed it could get them dropped quicker because they have a high fraud rate :)

I'm fully on board with the AdNauseam model which fucks about indiscriminately. The ad industry can burn.

Cannot agree more. Since the the advertisers have that much to spend, let's make them to spend more for nothing. Also by doing so, it will literally put enough noise into the data those trackers collect and renders the user profiling useless and effectively protecting us from being tracked.

If we only click ads on high quality contents, those content owner might benefit from it in short term but in long run it most likely going to back fire and make them penalized as for sure there is going to be counter measures. If we simply click every ad indiscriminately, there is no way to tell or they have to penalize everyone which almost equals to do nothing.

> to get them more money

Wouldn’t they just lose money? Ad clicks are not valuable to the businesses themselves. Only to the ad companies.

>That'd just end up costing the high quality content providers money.

I always click on Taboola bitcoin scam links for that exact reason. It's like scambaiting an algorithm.

Plus, low quality content providers would have a high click-through ratio but very low conversion rate.

Might be interesting.

Pretty much all pay per click ads are trash… and I guess most other ads too. No complicated logic needed here!

Last time I checked you could configured click frequency in AdNauseum.

Maybe that's where all the invalid clicks came from last year...

I think it wouldn't be about the ads, but the visit tracking. Like, block GA from seeing your visits on trash sites, but allow when it's a high quality post/content/source, so we skew the numbers for high quality content

That's actually very interesting.

Is there any extension that blocks analytics selectively?

AFAIK, no. You would need to manually add exceptions by domain

Untill your last paragraph I was thinking you were going to say "so let's lead from the front and let the trackers see the real internet."

What's the name of the movie? I'll add it to my watchlist

Free Rainer

This movie sounds great and I would love to watch it. It's a shame there's no legal way to watch many international movies like this if you are in the US.

I make a point of watching a non-US film on every international flight I take - I find it to be a unique opportunity to watch interesting non-US movies with English subs. I have discovered two excellent oddball comedies from in flight movies (Die Goldfische from Germany and Le Grand Partage from France), but when I tried to rewatch these films with others in the US, I discovered it was basically impossible. There must be so many other great movies from around the world we are missing out on.

It's called "Reclaim Your Brain" in the US and you can buy/rent it via Apple TV.

> which excludes i.e. students (they don't have to pay)

That is incorrect, they do have to pay (once per household). But if you get BAFÖG (student loan/social benefit mix that require you and your parents to be below a certain income bracket), you don’t.

I know that very well, I had to pay it because there is no BAFÖG for foreign students. Just felt it might not be an important enough detail to explain BAFÖG to the international audience.

You could have called them and they would usually give you an exception. Not sure if that situation has changed now.

I can't quite understand this. By "exploiting" you simply mean targeting ads?

The real harm seems to be from the tech giants censoring speech and policing payments, but what's the harm that someone targets a pair of shorts that I might like or show an ad for a conference I might be interested in?

I.e. ads, facebooks feed, endless scrollers like 9gag -> they like to use dark patterns and exploit tricks against the human mind to keep and guide your attention.

The harm if targeted ads depends on your viewpoint.

A targeted ad might serve you something you were looking for anyway, or it might manipulate you into spending on something you don't actually need. I.e. look at Instagram influencers, showing off their fake perfect live, making the viewer feel small and then try to buy the same happiness by buying the same product.

At best, ads are information that you need, at worst, they use psychology to manipulate you.

Seriously, you're worried about someone spending more time on their phone or spending more money on goods and services? Spending their own money — not the public money? That's what freedom is to you? Not the freedom to talk to whoever you want, say whatever you want or pay whoever you want?

I have no clue where you derive any of these things from. I certainly never said any of it. I never even made a claim a out freedom.

I was talking about how ads, tv programming, trackers and such have a tendency to create a positive feedback loop, which leads people towards less quality and mindless consumption. And about a fun idea from a movie, to break this feedback loop and replace it with another one, that promotes higher quality content.

You then started talking about a different topic and are now accusing me of not being interested in free speech.

hkai's mind is like a hammer: whenever it sees an opinion relating to human choice, it sees a nail.

I think you might have the meaning of 'i.e.' (that is, namely) confused with 'e.g.' (for example).

Thank you very much, I am proud to say that I did this wrong for at least 10 years :D

Thanks!

There is a shortcut to remember this: 'e.g.' for 'example given' (and therefore, 'i.e.' for 'that is').

('E.g.' doesn't really mean 'example given' in English - it means 'exempli gratia' in Latin. But it's a useful mnemonic.)

I also use "in essence" for "i.e." even though it's actually 'id est' in Latin.

In my mind, when I see i.e., i think "in other words", and when I see e.g., I think "for example"

It's a question of control; no should mean no.

Some people don't want to be tracked or monitored by advertising companies and it should be enough to just say so without companies like Facebook always trying to sneak tracking back in via dark pattens, shadow profiles, etc, etc.

For example once you've seen a website offer you the same product for different prices based on arbitrary tracking it leaves a bad taste in your mouth.

From your answer and other people's answers I feel like everyone has been brainwashed to focus on minor things like this, and totally ignore the elephant in the room.

So Facebook and Google know your religion, politics and food preferences. So what? How can they harm you with this?

They harm you when they choose what you can say, who you can talk to and who you can do business with. But literally what's the harm of ads? Making your computer a bit slower? That's such a tiny issue it's not even worth talking about.

"large scale AdWords fraud."

Be carefull - fraud is a crime. But I am under no obligation to provide AdWords any data, let alone true and reliable data.

Considering the tracking and spying, and my legitimate interest in privacy, there is no room for fraud argument here.

PS: this would be probably a Hoax

Fraud would involve financially benefiting from it - if you don't benefit I don't think it would count as fraud in most countries.

Where did you get this notion from? That’s absolutely not true, and there are many acts that can be illegal without benefitting from them.

I don’t think AdNauseum users would be prosecuted, but you’re still wrong.

source - https://en.wikipedia.org/wiki/Fraud

lots of acts can be illegal, but that does not make them fraud.

In England and Wales, all three types of fraud covered by the 2006 Act have as substantive elements of the offence a requirement that a defendant intends to either "make a gain for himself or another" or "cause loss to another or to expose another to a risk of loss", where gain and loss are defined as only referring to "money or other property".

Note also that the offence is framed in terms of intention to gain or cause loss. Even if no material gain or loss happens, the intention is what matters. (The equivalent mental element for theft in England is that one dishonestly intends to permanently deprive someone of the property being taken.)

https://www.legislation.gov.uk/ukpga/2006/35

This applies to online ad 'click fraud' in both ways—if a publisher fraudulently clicks on adverts to make money, that'd potentially be fraud to make a gain for themselves. If a competitor clicks on adverts to get their competitor to lose money on pay-per-click, that'd potentially be fraud to cause a loss to another.

I can't speak for jurisdictions other than England and Wales but I'd be surprised if a fair number of other jurisdictions didn't also define fraud in a way that covers both gain and loss scenarios.

You may want to read the CFAA and report back.

Spoiler alert, it doesn’t involve your intentions or money.

https://en.m.wikipedia.org/wiki/Computer_Fraud_and_Abuse_Act

We are talking about general fraud, not CFAA, which is a 'hacking' law.

The companies spying on me should be the first in line to be tried under CFAA

It's not actual fraud. You're not bound by the terms of service of ad providers. Your device is actually sending requests to the ads, which they are counting themselves to determine payment rates. If it's fraud to send ad impressions to sites I like without visiting them, it could be fraud to deny ad impressions to sites that I do visit (with my ad blocker on.)

If the sites set it up themselves, it's fraud. If you conspire with the sites to set it up, it's fraud and conspiracy to commit fraud. But if I'm prosecuted for the crime of not actually looking at ads I request, that's just a judge with an agenda. Whether it went one way or another at every layer of appeal would probably be a coin flip, though.

Agree. I'm genuinely want to click every ad I see to protect myself from being tracked. If that could be fraud, will there be anything not a fraud?

Correct, fraud isn’t the correct term here. The op grabbed the first word that probably presented itself, and others nearly ad nauseam (lowercase) have discussed its merits as the term.

What’s needed is a better term.

I propose either AdTurfing (hat tip AstroTurfing) or AdLighting (hat tip GasLighting). My personal preference is the second.

Isn't it just vandalism?

Vandalism is, again a crime against property that is legitimately placed on the street or somewhere.

Ad biz is a business model, not property, and a predatory one as that. You are not obligated to enable someone else's business for free.

They are claiming their clicks and impressions mean something - its their problem to ensure they are accurate.

Imagine someone is doing a survey of sex habbits, and selling the results. Is it a fraud to lie? Ofcourse not, why should you be responsible for their profits.

The fact that people think they are legally obligated to enable this is really fucked up.

I think it is just a matter of semantics. Let's call it "large scale AdWords statistics improvement". Look, sounds much better now!

While legally we are probably far away from this being a fraud, nothing stops google from adding something to their TOS and banning your account on that basis... This is the only reason I'm not using those noise generators even though all ad-tech should burn in a trashcan fire in my opinion.

Is it the Simpsons or Seinfeld where in one episode one of the characters gets a Nielsen box that is used for measuring tv viewership and they can’t leave the house for fear of shows being canceled if they’re not home to watch it.

Happens in Family Guy too.

They did something similar in “Roseanne.” The Connors were selected to be a Nielsen family and Roseanne made the family watch nothing but PBS, documentaries, etc the whole time. She wanted to hack the ratings so that over time, regular folks like her family would get exposed to a better class of information.

And ALF in 1987 S02E05

I want to believe that you just remembered that, like if the top of your head.

The idea that some die hard Alf fan just made they way past my post on hacker news really puts a smile on my face.

The movie is "Who Am I (2014)".

I meant Free Rainer, didn't know that who am I had similar themes. Is it good?

Oh. I must have remembered wrong then. Maybe I mixed those up. I think it is worth watching.

Edit: Note: Free Rainer also goes under the name "Reclaim your brain"

Is this the fee you have to pay even if you are blind or deaf, or do not have a Radio or TV? Gives you the most boring TV News in the world, plus 600 movies per year nobody wants to watch, because they are horrible? But they collected the tax and they have to spend it, as the German actors guild is worst than a cartel? :-)

https://www.german-way.com/germanys-tv-tax-the-debate-over-t...

Not a big fan of the GIS either, but my stance is not as extreme. I see benefits, but there is problems as well. It could be a good system if there was a proper reform and if you get politicians out of the system.

> Is this the fee you have to pay even if you are blind or deaf [...]

That's not correct. If you are deaf, you get a reduction - if you are blind, you get a reduction. If you are deaf-blind, you don't pay at all. If you are already receiving financial help because of your blindness, you don't pay at all.

You are right about it, did that change recently? According to the reference below it depends on the degree of deafness or blindness...if you can still hear or see something still have to pay...

https://www.rundfunkbeitrag.de/buergerinnen_und_buerger/info...

Deutschlandfunk alone is worth it imho

I personally prefer boring news backed by quality journalism and funded by people rather than sensational, outrageous and superficially controversial news! If I want the excitement I watch a drama or action movie. I think the news articles/pieces work like clickbait; the more head-turning they get, the more viewers they acquire which results in higher profits.

I am not talking about sensational or misleading like in FoxNews or CNN :-) I am talking about boring as in....very serious persons, on a serious background, reading very serious the Reuters or Ap news of the day. As the service was paid and it has to provided ;-) A better model would be instead having to provide good analysis or quality content otherwise your audience will go somewhere. As in for example, the FT or the Economist.

Actually thinking about it, the issue is wider than just the news and I think the financing mode of the mandatory tax is a big part of it. What is really a shame, as Germany has a rich culture of hundreds of years so great content in all forms should not be a problem.

"Why is TV in Germany so bad?" https://www.reddit.com/r/germany/comments/3d4vxz/why_is_tv_i...

"Why is German TV so crap?" https://www.exberliner.com/blogs/the-berlinale-blog/berlinal...

You might comment the BBC financing model is similar and I would agree. I think the difference is that the BBC also embraced a highly commercial model of selling content like Top Gear and other stuff worldwide. In this case the English language content with its planetary audience, pushed for a more competitive/commercial model and the German TV stayed too insular in my view.

Audience engagement chases entertainment (and thus reproduces CNN/MSNBC/Fox etc). You won't get what you want by tweaking that metric alone.

What even gives you the idea I am saying that? Did I say it? Nope.

Do you remember the name of this movie? Might it have English subtitles?

Free Rainer is the one I'm referring to.

> Maybe we should engage in large scale AdWords fraud.

You may like the "Google will eat itself" idea.

https://www.gwei.org/index.php

"202.345.117 Years until GWEI fully owns Google."

Nice cosmological time scale idea

Isn't this just investing the results of ad fraud into Google? Probably a profitable idea, but I'm not seeing the "Google eating itself" aspect.

According to their numbers they netted $400,000 worth of Google, essentially for free. Mind, that hasn't been updated in close to a decade; nowadays their 819 shares would be worth about $2.4 million.

> Google Shares owned by GWEI: 819

Those 819 shares are worth $2,371,005 now

That's art!

Don't modern internet-connected TV boxes snitch on what people are watching? Surely we don't need a sampling anymore?

Just thinking that the elderly population is probably the least likely to use those boxes (though I am not even sure of that), whereas they constitute the (dying) core of traditional TV viewership.

You know a connected TV / set-top box is on a channel but you don't know how many / which persons in the house is in front of it.

How long until they start putting cameras in them and doing facial recognition?

Put tape over it?

They could put an IR camera right next to the remote sensor. Since it's behind a glass that is opaque to visible-light you wouldn't even know the camera's there, and if you put tape on it your remote wouldn't work anymore.

>Since it's behind a glass that is opaque to visible-light you wouldn't even know the camera's there

If it's opaque to visible light how does the camera produce an image?

>if you put tape on it your remote wouldn't work anymore.

Not really? You just have to be more careful placing the tape. Besides, nowadays many smart TV have app remote controls, or RF-based remote controls so the IR sensor being blocked is a non-issue

> If it's opaque to visible light how does the camera produce an image?

It isn't opaque to infrared light though. The image is still high enough quality to make out people, it looks funny but it is good enough for their purposes.

Thank you for giving such great ideas for free to CIA, I just can't help to wait until it's implemented in my next smart tv /s

Just use the microphone in the smart tv. That's what the CIA does: https://www-washingtonpost-com.cdn.ampproject.org/v/s/www.wa...

Speaking of tracking, non-amp link:

https://www.washingtonpost.com/news/the-switch/wp/2017/03/07...

Some sort of localised motion sensor would probably be sufficient to tell you how many people are watching without invading too much on people's privacy. You wouldn't know the age, but you already have ton of information by knowing the address and the subscriber already.

> Some sort of localised motion sensor would probably be sufficient to tell you how many people are watching

You'd also have pets captured, curtains flapping by an open window captured, and any toys kids are playing with (like balls) potentially captured as another viewer too.

Meanwhile pap who likes to sit motionlessly while the kids round around him, isn't detected.

> without invading too much on people's privacy

Motion detection feels like a pretty major privacy violation to me.

> You wouldn't know the age, but you already have ton of information by knowing the address and the subscriber already.

Except when you have friends or family over.

So I found out my neighbors were proud to be participating in some new Neilson study. Apparently they pay you to wear these boxes in your pocket that pick up sound. At the end of the day it syncs what it heard to their servers and you get points, which eventually translates to money.

Essentially the stated purpose is so that it can pick up what music you listen to throughout the day, but in reality it's picking up everything.

Oh so like a cell/mobile phone, but only with the purpose stated out right?

Nah, they get paid to have the box in their pocket, you pay to have your box in your pocket.

Cameras are here and we have no idea what they do with them.

Most "smart" TVs have cameras these days...

> Surely we don't need a sampling anymore?

Depends, are the TV networks buying that information? Or just advertisers?

Modern TVs do snitch but there are two issues with relying on that:

1. There is a disparity between age groups and other demographic dividers who have newer TVs. This could significantly skew the results for some advertisers.

2. The data is going to the TV manufacturer, and they will not share that freely between themselves or with anyone else. This will complicate collating the data as there are several entities to negotiate with in order to get an overall picture.

And the tech savvy people never connect their "smart" tvs to the internet.

My "smart" TV has one job - show the signal fed to it via HDMI from my linux box.

So does your cable provider's set top box. So does your Roku/Fire/AppleTV/etc, except it is at the app level.

> Don't modern internet-connected TV boxes snitch on what people are watching?

Yes, which is why I will never own one.

Just wait until 5g is everywhere and companies will just add its functionality to your tv and connect regardless of what you do.

They can't add functionality to a device that I don't own.

Ironically, modern streaming apps have far more accurate numbers than anything Nielsen ever cooked up sampling people, but Netflix made the precedence that they should be "secret sauce" and not shared publicly and most of the "Streaming Wars" diaspora today are following that policy/precedent.

We're in something of a worst of both worlds situation where Nielsen has an increasingly small number of viewers where traditional TV boxes work to get decent samples, has to rely more than ever on surveys, and distrusts all streaming viewership numbers because they are cloak and dagger white lies between competitors, despite in theory being way more accurate than all the previous tools (the surveys and the TV boxes).

It's almost wild. The most forthcoming to shareholders/the general public over the years has been Hulu and Hulu's numbers at times have suggested Nielsen's data is very, very wrong right now, but Nielsen doesn't trust Hulu's data at all because it smells like lies because Netflix does nothing but lie or ghost them.

I have been exploring ways to avoid sending this data to them. To start: using an instance of Pi-Hole and running all your apps will remove some tracking from them.

Following this up with paying for your subscription but pirating all the shows might help remove you from the cycle completely.

I understand the second point is not realistic for the majority of people but I wonder if in the future we might have an easy to use version of Pi-Hole that most people can just flip on and strip a large chunk of tracking from the apps.

That just gets us back to the topic way above: if we aren't giving that tracking data to TV producers, it's tough to complain when our favorite nerdy/intelligent shows get cancelled.

That's the deep weird irony that we live in a world where we could have the best possible numbers (directly tracked statistics), and yet TV Producers are still relying (for the most part) on Phone and Snail Mail Survey Results because they don't think they can trust streaming provider numbers. Pi-Holing those numbers just gives those Producers even more reason to feel that they are lies or wrong. At what point do you Pi-Hole too many telemetrics to oblivion and aren't allowed to complain when your favorite TV shows get canceled because "no one" was watching it?

I guess this contributes to me watching less and less shows because I am tired of stuff being canceled midseason. It might be a self fulfilling prophecy. Less people watch these shows/appear to watch these shows, the networks stop producing it and these people don't come back to whatever the network does end up producing.

From what I am hearing, traditional forms of media are in decline because people are spending their time on Youtube/Twitch/Social media instead.

> They faked the ratings away from stupid trash TV towards some higher quality stuff, documentaries, culture, ... Obviously in the movie then the country saw a renaissance, everyone got smarter, yadda yadda, you get it.

> I feel like this is similar. All tech savvy people block ads and analytics and at least the known tricks they use against us. So the internet only tracks the defenseless people and is then built to serve them (and or exploit them).

I think this is right - ad-traffic is manipulative and actually I don't think it is a societal 'good' at all.

A few personal examples:

* On Youtube almost all my adverts are encouraging me to start Forex / Stock / Property investment and trading, and sign up for courses on these. These courses are scams (or at best, 'half-scams' and poor/generic advice repackaged and sold for thousands), and in general provide poor financial advice (either through extortionate courses, recommending you become too heavily leveraged or advising you to day-trade high-volatility stocks by just looking at charts). Presumably it does this because I am 32 and male, so I am considered 'prime' for this marketing.

* One of the friends I know is a girl, and she has never seen the above adverts. We were talking and she says every single advert is just about pregnancy and fertility. I wonder how many of these adverts are just reinforcing gender-stereotypes in a wider sense, i.e. while google claims to be progressive and care about 'equality' really is their business model at it's core really just targeting women and telling them that they should be getting pregnant, while telling guys that they should be the bread-winners and earn money via stocks/shares?

* While my adverts are for forex, and my apparently fertile friend is getting adverts for pregnancy tests, my older parents just get targeted adverts for pre-paid funerals. One or two are probably be fine, but they are just on constant repeat - and I can't help but think that I wouldn't the constant reminder of death before every youtube video.

* My laptop is convinced that I want to go camping. It's only my laptop, every advert is camping related. Sleeping bags, tents... and the strange thing is that when it started I didn't want to go camping, but it's been so consistent across the last few months now that I kinda wanna go camping. Like it's sold me this romantic vision which I know wasn't there before, so even though I would usually like to say I can't be manipulated through marketing, it's really made me realise I can be.

Is the above really making society better? And if it's not, why should we put up with it? IMO the biggest lie we have been told by Google is that 'personalised ads' are a good thing.

Your female friend should probably take a pregnancy test. During my wife's first pregnancy, my Kindle started displaying diaper ads within days.

On the other side of the coin, my ex-wife continued to get diaper/formula/etc ads for years after our miscarriage.

Just change your gender to trans/alien and your age to 5...

"trans" isn't a gender. But, I am trans, and advertisers are more than prepared to advertise to me. So I'm not sure what the goal would be there.

Irrelevant ads are less likely to work.

Probably, but they are much more annoying.

>why should we put up with it?

You, your friends, and your family personally don't have to put up with it. Ublock Origin can block ads on Youtube with ease.

It isn't available on all plateform utube is unfortunately.

I agree on a personal level, although at a societal level I believe regulation is required.

As per the parent comment to my original one, I just fundamentally do not believe that most advertising contributes anything positive to society and mostly generates negative externalities.

The argument for advertising is that it's so difficult for a new entrant to a market to challenge an entrenched incumbent, unless they can advertise.

I honestly don't know if that argument holds water.

Intuitively, I'd say it doesn't hold water: Entrenched incumbents would generally tend to have more money than new competitors, so the incumbent can afford more advertising to drown out that of the challenger.

The only net gain I can see is for the advertising industry, which extracts money from both challenger and incumbent.

> The advertiser has a tracker that it places on multiple sites and tracks me around. So it doesn't know what I bought, but it does know what I looked at, probably over a long period of time, across many sites. Using this information, its painstakingly trained AI makes conclusions about which other things I might want to look at, based on...

> ...well, based on what? …Probably what it does is infer my gender, age, income level, and marital status. After that, it sells me cars and gadgets if I'm a guy, and fashion if I'm a woman. Not because all guys like cars and gadgets, but because some very uncreative human got into the loop and said "please sell my car mostly to men" and "please sell my fashion items mostly to women."… You know this is how it works, right? It has to be. You can infer it from how bad the ads are. Anyone can, in a few seconds, think of some stuff they really want to buy which The Algorithm has failed to offer them, all while Outbrain makes zillions of dollars sending links about car insurance to non-car-owning Manhattanites. It might as well be a 1990s late-night TV infomercial, where all they knew for sure about my demographic profile is that I was still awake.

> You tracked me everywhere I go, logging it forever, begging for someone to steal your database, desperately fearing that some new EU privacy regulation might destroy your business... for this? [1]

[1] https://apenwarr.ca/log/20190201

I think very few actual people click on ads. Ad exchange platforms get majority of their revenue from impressions anyway.

As someone who has been in one or another meeting with German TV stations I can assure you this is not completely far fetched. The people deciding what is running at these stations are of the mindset (a translated quote):

> Our average viewer is 65 years old and watches TV while doing the dishes, we must not show things that cannot be understood by them

and then they will serenade about how they would "love to have a bit more sophisticated things", but as they are the only ones who really understand their audience, they cannot allow this, although they support the values of the 68 generation etc. pp.

From my standpoint the German television landscape is completely doomed, because the people at the levers are in the illusion they do the good thing for "the small man" while in fact they just think the small man is incomprehensible stupid and must not ever be confronted with content that shows them that there is still stuff to learn and understand in the world.

Well, you say that, but isn't that exactly why Netflix got rid of ratings?

All the documentaries were getting really high ratings, so would display highly in searches, but not many people actually watched them.

It's the same for most content, I read a huge amount. I do read some intellectual books, but only occasionally. The rest of the time I read utter, thoroughly entertaining, trash.

I don't want to read about an existential crisis after programming all day, I want someone to hit something with a big sword and get the girl.

> It's the same for most content, I read a huge amount. I do read some intellectual books, but only occasionally. The rest of the time I read utter, thoroughly entertaining, trash.

I feel the same way. However I force myself to read things that will better me once in a while anyway. I too want to hit things with a big sword (without the pain of getting hit), and get the girl (without cheating on my wife), but the world including me is better if I do something else anyway. Which is why I do sometimes read a complex math book.

> Well, you say that, but isn't that exactly why Netflix got rid of ratings?

> All the documentaries were getting really high ratings, so would display highly in searches, but not many people actually watched them.

This rather shows that ratings do work, but are used wrongly for giving recommendations:

If you want to create suggestions for a user, in many case the wrong answer is "suggest what has really high ratings", but rather "given the ratings that this user gave and the films that he watched, what will he also like."

The fact that these documentaries get high ratings (the same might hold for art house films) shows that there is some (niche?) audience which really loves this kind of films, but not that "John Doe" will love it, too.

Predicting exactly this was the premise of the famous Netflix Prize, one of the first open machine learning challenges https://en.wikipedia.org/wiki/Netflix_Prize

I remember my professor talking about the ensemble approach the competitors were taking in a data mining class I took. I was saddened when Netflix ended up not using it because it was too slow and expensive or something.

It didn't promote Netflix content above third party content. Not enough control for the algorithm is unacceptable for the marketing guys.

Thanks! I'd totally forgotten the real reason.

I remember when that competition was released, I was working (as an undergrad) in GroupLens, which had MovieLens still around but it was getting a bit of bitrot.

It was always weird to me that GroupLens didn't spin up a team for it, but it seemed like everyone in the research group had moved on to other things and didn't want to context switch back. Someone mentioned something like "shame they didn't do this 10 years ago, a million dollars would have been nice". I think someone was doing tagging on movielens, but I don't remember the details.

I got the sense that neither Riedl nor Konstan (or any of the current grad students it seemed) wanted to pursue it (Terveen, I think, wasn't in to recommender systems at all in the first place).

I don't think the lab had any funding problems haha, so maybe that was what it came down to.

Isn’t this why it now says something like “your match: n%” now?

> This rather shows that ratings do work, but are used wrongly for giving recommendations

And I know I'm (possibly) a minority on hacker news, but I prefer the new system. I was giving everything I wanted to watch more 4 or 5, even when it was clearly not the case, but because I want recommendations of things I'm going to like AND actually watch

That is how it is with documentaries. Just because they are getting lower viewing numbers doesn't mean you shouldn't keep promoting them. Certainly there should be a mix of entertainment and public interest stuff, but following audience preferences for entertainment creates a feedback loop that damages society.

Things like documentaries are mostly watched by already well-off people (mostly middle class and up).

In Germany, public TV is paid for by (nearly?) every household. [0]

Forcing everyone, including poor people, to subsidize rich people's taste for documentaries seems a bit.. off?

Similar arguments apply to public libraries and opera houses, though at least there the financing is done mostly via progressive taxation.

Of course, you can argue that we sophisticated people know what's good for those unwashed masses, and if only they watched their documentaries like they are supposed to, they would soon see the light. Colour me skeptical.

[0] As far as I am concerned, private broadcasters can and should do what they feel like.

Well, when people complain about a growing wealth divide, and those who are doing okay financially say that childhood access to public libraries and documentaries made them who they are, shouldn't those things receive funding?

That's a bit like asking healthy people. You'll get answers ranging from things like exercise to homeopathy.

Answers from people can be used to suggest avenues for investigation, but shouldn't make you fund expensive stuff outright. (Assuming here that homeopathy is obvious nonsense, that people still swear by.)

And, of course, public libraries and documentaries don't have to just produce good effects. In order to justify public funding, you'd need to do a whole cost-benefit analysis and look at opportunity costs.

So look at what you get from public libraries vs libraries financed from private charity _plus_ whatever other good things the public money saved could do (including just outright giving it to poor people).

I deliberately picked public libraries here as a provocative examples. I suspect they might actually pass the cost-benefit test without too much contortion of metrics and data.

I am much less sanguine about the bang-for-buck of publicly financed opera houses, theatres and symphony orchestras, which Germany is quite fond of. And of course, publicly financed radio and TV broadcasters.

Without US public broadcasting, I and countless others would have grown up on GI Joe instead of Mr. Rogers.

Further, I generally strongly oppose charity as a component of future plans (leaving aside whether charity is good or necessary in the present). We should never be building society such that it depends on the funding whims of rich philanthropists.

Sure, there should be some kind of analysis of benefits, but some things simply have to exist for a society to be a society, because without them, the loss of their intangible and second-order benefits will cause a society to implode Idiocracy style, and nobody will know why.

And I am suggesting we should do careful cost benefit analysis, instead of just assuming that our favourite pet causes will surely come out ahead.

On top of cost-benefit analysis, you will have to explain to everyone whose life trajectory was meaningfully improved by a resource what alternative path you are providing so that future people can also find their way to a better life.

“You wasted $150,000 on an education you coulda got for $1.50 in late fees at the public library.”

- Good Will Hunting

edit: btw, speaking of libraries, copyrighteousness is destroying our society

Of course, Good Will Hunting is wrong.

Receiving education isn't about learning. At least not primary. It's about getting a piece of paper at the end.

Public libraries and free internet resources are one argument in this direction. Another: professors are usually more than happy for you to sit in on lectures, even if you don't pay any tuition.

> Things like documentaries are mostly watched by already well-off people (mostly middle class and up).

assuming a normal distribution, wouldn't this then be the majority of people?

This question would benefit from some empirical data.

Arguing from definitions won't help us. (Keep in mind that 'middle class' doesn't mean 'median income' or something statistically simple like that.)

People choose from what is presented to them. That's consumerism. It's not like people get to pick what gets produced. If more public interest material is available and advertised, it'll get watched more. The alternative is to watch less TV and engage with society directly more. Both of those outcomes would be preferable to the excessive production and consumption of entertainment.

Private broadcasters do not pick material based on public interest or even their judgment of what is good. It is far more mechanical and influenced entirely by market forces. Herman and Chomsky discuss this in Chapter 1 of Manufacturing Consent.

https://ia802700.us.archive.org/31/items/pdfy-NekqfnoWIEuYgd...

> The alternative is to watch less TV and engage with society directly more. Both of those outcomes would be preferable to the excessive production and consumption of entertainment.

I agree. And economically, if you want less of a good to be consumed, ceasing to subsidize its production with tax payer money is a good first step. If you want to go further, perhaps even tax its production.

> Private broadcasters do not pick material based on public interest or even their judgment of what is good. It is far more mechanical and influenced entirely by market forces.

How do market forces differ from public interest?

Or rather, what do you mean by 'public interest'? It's what the general public is interested in?

Market forces come from people.

Market forces come from people.

Market forces tend toward baseness. It takes active intent to elevate society above the lowest possible level. Just as one doesn't want to hire at the median skill level of their company lest the average continue to drop over time, a society has to aim higher than what the impulsive market optimizes for, or it will decay.

Revealed preference is just code for exploitation of vice.

'Market forces' are just what people want and are willing to pay for.

Of course, there are always enlightened and sophisticated people like yourself that know better what's good for the unwashed masses.

When I was first living on my own barely making rent, PBS documentaries were the most interesting thing on broadcast TV. Infinitely more entertaining than drivel like the Bachelor. Other than available time and offered free content I doubt preference of documentaries is different among income classes.

It's crazy how many people believe being poor means you're stupid. Not true! Thank you for pushing back!

Who said anything to that effect?

Are you implying that people who have better things to do than watch documentaries are stupid?

Social class is correlated with income, but it's not the same.

Similarly, entertainment preferences are correlated with social class (and with income), but again, they are not perfect predictors of each other.

So a few anecdata wouldn't undermine anything here. Though in fact, your example actually strengthens the argument I am making: you are the kind of person that prefers watching documentaries over other drivel, and you are the kind of person who managed to get themselves out of poverty. That's likely because you have the preferences, habits and skills of someone who is at least middle class in a social sense, even if your income took a while to catch up.

(Keep in mind that we are talking about social class in a rather abstract fashion here. German middle class mores are different from American middle class mores.)

Wow, that is some overt classism right there. The world is a much better place is you just see people as people. Everybody is trying to achieve the same thing regardless of their "class". Security and safety for self and loved ones by whatever means are available to them with their skills and knowledge.

How many of these documentaries are actually public interest though? Versus propaganda by someone with an agenda to push under the guise of intellectualism?

Off course the so called "social issue" documentaries are nothing more than propaganda often created by think tanks or publicly funded "opinionators", these are very rarely not a waste of money and attention. It doesn't take away the fact that nature/science documentaries (if produced well, with sufficient funds) can capture imagination of a general knowledge of the layman public.

That is always the problem with data: it is reactive. Sure the average watcher is 65, and wants easy to understand stuff: that is what the data shows (I'll assume for discussion that is what the data shows, but I have no insight into if it is true or not). What the data doesn't show is if content would draw in day 25 year olds, they need several years of trying those other shows to see if it makes a difference - a very risky best that could run them out of business even if true (that is the older crowd stops watching faster than the younger crowd figured out it is worth watching meaning advertisers don't pay enough to keep producing content).

Of course TV in the US has figured out that the 65+ crowd is very valuable to customers (the advertisers, not viewers!), so even though they could get more viewers by not showing the nightly news, the nightly news is what they show.

This the same problem as interviewing people for a job nd collecting data about which “features” from interviews correlate with job performance… Without tracking the performance of any of the candidates who weren’t offered jobs—or turned down an offer.

Something, something, a diagram of a plane showing where the damage was on those that returned from missions.

https://en.m.wikipedia.org/wiki/Survivorship_bias

> Something, something, a diagram of a plane showing where the damage was on those that returned from missions.

Ahahahah I love this dogwhistle. You always know you're in good company when someone waves their hands around and says "put the armor where there AREN'T bullet holes" and gives you a significant look. XD

Reminds me of a thread a while ago pointing out that a single show dominates the MTV programming lineup almost every day [1]. It's as if there is some algorithm running without human intervention that feeds on itself by reacting only to current viewing habits:

    foreach (show s in lineup)
        if (s.viewers > THRESHOLD)
            lineup.replace_with_more (s)

Obviously resulting in this weird local maxima where no other shows get broadcast.

1: https://twitter.com/MTVSchedule/status/1422934028253081603

I don't have MTV so I had to look up what "Ridiculousness" is. Apparently they play Youtube/Tiktok/etc... clips? Sounds like it must be incredibly cheap to produce. This is what it looks like when you let a race to the bottom continue on indefinitely.

And the cable industry can't understand why people keep cutting the cord. Can you imagine shelling out $120/month to have some producers pick out Youtube clips for you?

I went to a big box bar/restaurant a couple years ago, and they had a big screen TV playing basically what you describe. I wonder if that is the audience, subscribers using is as essentially something in the background? It's hard to believe there are enough that want this to make it economically viable...

In the US there's actually an FCC requirement that television channels air news.

From https://www.fcc.gov/media/radio/public-and-broadcasting: "virtually every station has an obligation to provide news, public affairs, and other programming that specifically treats the important issues facing its community." The details are specific to the license, but almost every station is required to air at least an hour of news a day.

There's also requirements to air a certain number of hours of educational material for children (https://www.fcc.gov/consumers/guides/childrens-educational-t...).

Does the requirement mean air news during prime time though, or can they do it at times when most people are not even able to watch TV?

It's unclear, because the exact details are specific to the license and therefore may vary from station to station. My local television stations usually air the news at 11, which is after primetime.

This is true, but it's worth noting that the FCC revoked the fairness doctrine in 1987.

> In the US there's actually an FCC requirement that television channels air news.

Except, I thought, if they're explicitly registered as entertainment channels? (Like... Fox "News".)

The regulation is concerning over the air broadcasters. Cable stations like Fox don't fall under the regulation as they don't use public airways.

"you don't build bridges by measuring swimmers" or whatever that quote is.

This is precisely why the internet wins. It can both show the nightly news and a million other things at the same time, catering to a 60-year-old housewife, a 40-year-old car enthusiast, a 30-year old gardener, a 20-year-old fan of obscure Linux distributions and a Taiwanese 10-year-old kid living in Ireland, all at the same time. Youtube regularly recommends videos that I personally like, but that onlyhave a couple thousand views.

In France there is the 50 yo housewife: https://translate.google.com/translate?sl=auto&tl=en&u=https...

> Our average viewer is 65 years old and watches TV while doing the dishes, we must not show things that cannot be understood by them

So like, sticking only to the first and second derivative in soap opera plots about retired civil engineers?

No late-period Beethoven sonatas as background music?

That must be difficult for you as a German.

Meanwhile, here in America we're perhaps a few years away from something like the movie "Ass" from Idiocracy winning an Oscar.

Are you joking, or do you have perhaps too high an opinion of German TV?

On the one hand: yes, I am.

On the other-- we have a network that bills itself as educational and spends over a month marketing a full week of programming dedicated to propagandizing its audience to be maximally afraid of sharks.

It broadcast a wildly popular movie where a tornado full of sharks attacks a city.

German TV could be an order of magnitude worse than my parody and it still wouldn't even register on the American scale of stupidity.

> Meanwhile, here in America we're perhaps a few years away from something like the movie "Ass" from Idiocracy winning an Oscar.

Very true, this is the first thing I thought of when I head of the popular show "Naked and Afraid".

It reminds me of the (possibly apocryphal) reasoning behind the cancellation of "Police Squad" back in the day: that people would have to pay attention to get the jokes.

https://boingboing.net/2014/07/04/police-squad-was-cancelled...

The '68 generation' is pretty much the same generation that's now 65 years old and washing dishes..

> From my standpoint the German television landscape is completely doomed, because the people at the levers are in the illusion they do the good thing for "the small man" while in fact they just think the small man is incomprehensible stupid and must not ever be confronted with content that shows them that there is still stuff to learn and understand in the world.

Well, that would be more bearable, if half the TV market wouldn't be allowed to essentially tax everyone to finance their drivel.

I stopped watching TV more than 10 years ago because I was worried to turn stupid from it. Not so far fetched after all.

There are some high quality shows and TV stations though. Namely Phoenix (similar to PBS in the U.S.) and some of the news magazines that run in the late evenings. Of course there are also all the other public stations with higher quality programs but I find the program most of the time quite random and sometimes even a bit elitarian.

> > Our average viewer is 65 years old and watches TV while doing the dishes, we must not show things that cannot be understood by them

I haven't watched german TV in ages, but I distinctly remember science shows degrading from science to thinly veiled ads - things like literally running a companies marketing video or making a "scientific comparison" where they hand out random style points at the end to make a specific product win. I think they even got into trouble over it since ads and science/education shows are taxed differently. Anyone pretending that they are doing that for their viewers is living in denial at best, but probably just outright lying.

The biggest problem on television is lying, and that problem is prevalent not just on German TV. https://www.youtube.com/watch?v=gMlMH1RfnF0

> Our average viewer is 65 years old and watches TV while doing the dishes, we must not show things that cannot be understood by them

> because the people at the levers are in the illusion they do the good thing for "the small man" while in fact they just think the small man is incomprehensible stupid

So basically they have a Hacker News mindset!

In every thread about a dumbed down GUI/website it is argued that granny wouldn't understand it otherwise. No power user allowed, because data shows user is monkey.

Do you have any links or references to that? It's fascinating.

https://www.imdb.com/title/tt0810868/

There is the Nielsen system in the US and I wonder if it is the other way around. That is, in reality, nobody has watched MTV since 1994 but Viacom bribed a Nielsen family to tune a TV to it and keep it there.

So you want more targeted ads for tech savvy people? I wouldn’t give ad tech that much credit. Many tech savvy people have cut the cord and watch TV through streaming. Anecdotally, I watch Hulu. Hulu knows so so so much about me yet zero of the ads are targeted. They very much have this capability (engineering resources) but due to a number of reasons I can only assume (network contracts, ad bids) it just probably isn’t going to happen. I would love if I could have an ad blocklist cause one more Progressive Ad will drive me bonkers.

I don't live in the ad tech world, so I only vague know what is interconnected. Do the tech giants just sell info back and forth to each other?

For example, When I watch youtube on my Roku, if im not signed in, does roku still aggregate what I watched on youtube, could that be sold to Hulu, for ads when I watch that on the same tv?

I work in the CTV ad-tech space. Viewership data is highly guarded by every company. It's how every company is trying to differentiate themselves from each other. So, no, YouTube and Hulu etc wouldn't share information.

Ad data is known by the DSP that serves the ad (if the company doesn't serve their own ads), but viewership data is secret as a competitive advantage.

What an awesome idea! :-)

I don't block ads. I remember what the internet was about to become before ads stepped in. Everything of value was going to be pull behind paywalls.

Let's say everyone get their wish and ads go away. Everything will require a purchase. Those purchases are logged to a real name/address. You end up with bigger privacy leaks.

People will still be tracking you the way they are now. And at the credit card level.

As an adult in the first world I can afford to pay for adfree solutions. Most people can't. Ads level the playing field.

I don't think the playing field is disleveled without ads. Wikipedia has no ads.

If you're willing to handle ads, they should at the least be untargetted.

Agreed.

Your personal prosperity depends on a prosperous online ad industry; doesn't it?

The last time I worked in adtech was 2012.

While that speculation is plausible, another view would be that companies would offer quality content without tracking users and invading their privacy (how many paid services still flood you with ads?), and possibly free alternatives would start to come up.

From your point of view, free open source is something that wouldn't exist

The hobbiest websites will not go away. Open source existed long before the internet.

What you are left with is the hobbiest websites or the mega brands that want to funnel you into their ecosystem. To offer anything that cost resources that you are willing to spend you must be a megabrand using this as a loss leader opportunity.

I don't think we want that world. We may think we do but look at what happened in Mr. Robot.

> I don't think we want that world.

That's the world we have right now, though, just with the addition of the spying that advertising brings.

The spying still happens regardless. We just don't have advertising (benefits and drawbacks)

> Let's say everyone get their wish and ads go away. Everything will require a purchase.

No, it won't. There was plenty of high quality stuff on the internet before ads or payment was even possible, and there's plenty of high quality stuff that don't track you or require payment right now. There's no reason to think that would all evaporate.

We will see plenty of high quality stuff still I agree. But much of the free stuff will be about converting you to the paid stuff.

With ads or not you are still the product. You will still be tracked because people want you to spend money on their service. People will sell that information. Companies will use it.

Sure, much of it will. But certainly not all of it. The amount of actually free content will certainly not decrease, and it would probably increase, even if by only a little.

> With ads or not you are still the product.

It depends on the site. There are lots of sites where the site operator has no interest in it generating an income, let alone a profit. You are not the product there.

Ads have bloated the useful internet to the point that it is more expensive and less functional. We have 8,000 websites trying to show me a recipe for chicken parm, most of them with pages of family history and backstory, because they are all trying to get me to see their ads. A lightweight Wikipedia for recipes and a few high value added websites charging a trivial amount for access to their recipe catalog would be highly value added for me, and run on a fraction of the infrastructure.

Ads obscure solutions, and add redundancy and complexity with zero value added, because solving a problem means you no longer are on the page seeing ads. Simplifying or automating a process means you are clicking less pages less often and not seeing ads. If you automate something to directly connect users with what they need, then they don’t need to come back and see your ads. So we have automations they bring us to some middle man that can show us some ads before we can get to what we need.

Ads mean that maximizing the time your attention is held is the core value. High quality content that leaves you informed/satisfied/fulfilled is worthless compared to low quality content that is just good enough to keep you from leaving, without being having enough substance to actual fulfill you, because then you might leave and not see the ads we have to show you.

Podcasts show us that a tiny minority of users able to pay for content subsidize an incredible amount of added value content for everyone, whether they can pay or not.

Ads don’t level the playing field. Ads are an ever growing tumor, sapping resources and weighing everything down in a mindless effort to replicate.

What’s the name of the movie?

> Maybe we should engage in large scale AdWords fraud.

Data poisoning for something the scale of Google I fear would be ineffective to the point of laughability, sadly.

Exactly like https://syntheticmessenger.labr.io/

Though, I would say this can actually hurt publishers in the long run

Hacking democracy (ratings voting system, whatever) to indirectly effect a supposedly benevolent unilateral outcome? Sounds familiar.

It's not exactly hacking democracy. Democracy implies every gets a vote. This is P hacking small sample sizes.

So it's engineering falsified scientific conclusions then. Still sounds familiar.

It's not hacking, it's statistics.

> the internet only tracks the defenseless people and is then built to serve them (and or exploit them)

Believe it’s Scott Galloway who said advertising is a tax on the poor and technologically illiterate.

There's a Family Guy episode where Peter does the opposite.

digital idiocracy

This is funny to me, because there's an American movie where Danny DeVito manipulates TV ratings to get his own (worse) shows on the air: https://www.imdb.com/title/tt0087979/

I expected these numbers to be higher. However, an even more interesting metric is the 88% block in Firefox.

Firefox may not have a great market share, but based on these numbers it's market share may very well be eight times higher than your analytics report. This can change the argument of "it's only 3% of our users so we don't need to test on FF" to "it's a quarter of our user base, we should at least test it", depending on your target audience. I've seen tons of people claim general Firefox usage is negligible based on public data from websites such as statcounter, but these metrics prove that those statistics are unreliable and should not be used.

The best you can do is use server side UA inspection, though you can't really distinguish bots from real users that way.

I feel this heavily depends on your goal.

IMHO, this points at Firefox being used mostly by ad-averse, tech-savvy users, while the less-adverse, less-savvy users prefer Safari and/or Chrome.

If your objective is to maximize ad revenue, the most obvious approach would now be to ignore Firefox completely and focus on non-FF browsers.

Of course, following web standards would be the Golden Way, and more selfless actors follow that rule, but that song has been sung ever since the old Netscape/MSIE wars.

> If your objective is to maximize ad revenue, the most obvious approach would now be to ignore Firefox completely and focus on non-FF browsers.

No, your best approach is to test firefox carefully to ensure it is broken. That way you encourage people to use something more friendly to you.

I hope I didn't give anyone ideas.

Dont give Google Maps and YouTube ideas. Wait...

Also Google Meet. I've noticed my webcam always comes through crystal clear in Meet on Chrome, but is reliably blurry in Firefox.

Wouldnt that be a feature ? I hate broadcasting hires images of myself.

It's less of a joke than it sounds.

I have to keep Chromium around not just for testing in it, but also to make certain purchases, reservations, etc, because some sites just fail to work in regular Firefox, even with enhanced protection off. Not many, maybe 0.1%, but in a pinch there's no other way than to fire up MISE^W Chromium.

Is there a good blocking solution for Safari? From what I understand uBlock Origin cannot operate on Safari due to the way that it disallows some allows access to the underlying source of the webpage.

Also it seems like extensions on Safari require you to install them via the App Store which just seems so dumb and unintuitive compared to Chrome/Firefox.

If they fixed these two issues, I think Safari usage would be much greater. That browser is so incredibly fast and snappy especially on the new M1 macs but not having proper ad-blocking is a complete deal breaker.

Add these to hosts:

0.0.0.0 www.google.com

0.0.0.0 www.google-analytics.com

0.0.0.0 www.googletagmanager.com

0.0.0.0 www.googletagservices.com

0.0.0.0 pagead2.googlesyndication.com

0.0.0.0 fonts.googleapis.com

0.0.0.0 www.facebook.com

0.0.0.0 twitter.com

0.0.0.0 platform.twitter.com

Doesn't Firefox still include google analytics into its Preferences dialog boxes or something? Im finding these stats ironic.

I don't have any proof or study on it, but I suspect they don't do it. My anecdotal evidence is that I use an application firewall and Firefox by itself pings only telemetry.mozila.org or accounts.mozilla.org and stuff like that. It uses domains that explicitly say what they mean to be used for. At least in my experience

Mozilla has a special agreement with Google about how their analytics data is stored and handled. I thought they only used it on their websites but it is not impossible that they use it for software telemetry.

If you're referring to Firefox using Google Analytics for the Firefox Add-ons frontend, as of July 2017, Mozilla has disabled Google Analytics for any browser that has Do Not Track enabled.

https://github.com/mozilla/addons-frontend/issues/2785#issue...

This change was made in response to pressure from HN readers, so thanks to everyone for that.

It's odd a company of that size cannot roll their own solution.

Maybe it's a condition of their funding deals with Google?

Ah I had no idea, well that explains what happened I guess.

It's odd a company of the size of GM cannot roll their own gas station and toad construction network.

What's the purpose of the Mozilla organisation? Is it in creating a analytics platform? Mind that value of GA comes from all the info they have and thus can estimate age, gender, social things, ... of users etc. Building an comparable service is a notable effort with little synergies to what Mozilla does. (Whereas Google can combine with information from other businesses)

I’m a Firefox user but let’s face it, Mozilla doesn’t have to make their own — there are plenty of open-source, self-hostable analytics solutions.

(Also the “toad construction network” made me laugh)

The question is: Why does Mozilla need analytics at all? Sure, counting downloads is good, but what other useful information are they pulling out of analytics?

I think he meant roll as in created their own solution whether it was from open source alternatives or not. The point being - why link what most would expect to be private stats to google, from a web browser, the one thing that perhaps should be a stand alone entity. I get the impression maybe you work for firefox as I find it hard to understand someone who would think otherwise on hackernews - but hey all sorts here am I right?

What is super weird is everything I said was correct but I somehow got -4 downvotes. Makes me think something is up with firefox and google still ;-_-

> The best you can do is use server side UA inspection

No, the best you can do is to stop caring. A distant second is this "server side UA inspection." Whatever that means exactly.

Server side useragent inspection

Yes. I clearly know the literal meaning. I assume this means trusting client-supplied strings. That's not an inspection.

If you inspect in on the server it is an inspection.

As opposed to client supplied string but with Javascript?

The reason Firefox is higher is probably because it's the easiest one to block ads on mobile. Most people I know who use Firefox on mobile do so specifically to have ublock origin. I personally use chrome on desktop but Firefox on mobile.

With so many options for Chromium-based browsers, is there a reason why you still use Chrome?

I personally use Edge on desktop and iPhone because I give so much data to Google by using a gmail account that it lightens the load a little bit to use something other than Chrome. It functions the same as far as I see and it runs all the same plugins.

Not op.

I also use Edge on my Android, it has a built-in adblock plus. I prefer unlock origin, but it's better than nothing.

Doesn't firefox on android allow ublock origin?

yes.

and privacy badger, https everywhere, decentraleyes and a number of privacy addons.

My reason for using chrome is I don't mind it. I think I'm less suspicious of Google than most of HN. They don't allow extensions on mobile though

I use Brave and am very happy with it

The reason Firefox is higher is probably because users who choose it are more educated about the internet and are probably the ones who know how to use an adblocker. So it is unlikely that the proportion of users who use Firefox globally is higher than what stats give us.

I thought Firefox desktop and mobile blocked Google Analytics by default these days, as part of the general anti-tracking protections. Maybe I’m wrong, though?

Ad Block Plus browser on Android is easier. I use both but don't bother blocking ads in FF since I use reader mode 99% of the time.

You have to be on Android to use addons with mobile Firefox. Since Android=Google, doesn't that mean you are giving Google all your data including browsing data and the fact that you are blocking ads?

I don't know, but it definitely blocks ads which is why I use it.

Yes, but at least I don't have to see as many ads, and my phone's CPU doesn't spend as much time rendering ads, and my limited mobile bandwidth isn't all sucked up by ads.

Only if you are running Google Play services. You can certainly opt out of this if you like. I run LineageOS and do just that, but I also don't need my phone for much more than the basics, and F-Droid can fill those in for my use case.

Why would you assume they are getting all browsing data?

Motive + ability. They want all the data and they own the OS.

Why would you assume they aren't?

Motive + ability aren't the only things. That'd be a clear invasion of privacy that you could expect to win a court case over.

there's never been any evidence that Google is capturing all internet traffic on android devices.

that being said...

The other option is even less anti-consumer than privacy hating Google.

There's no Apple phone with an sdcard slot. No Apple phone with a headphone jack. Repair your iPhone ? Well Apple just slows down your device under the guise of prolonging its life.

No Apple phone where you can install apps through a non-Apple owned store.

Not a single iPhone has a built in FM tuner.

And for all the privacy flag waving from Apple over the years, they're now scanning local files for child porn. This is a feature set that will just grow.

Funnily enough, Privacy Badger is an excellent ad blocker. Turns out 99% of ads come from tracking domains. You'd think the ad people would split up the tracking from actually showing the ad, but apparently it's not worth showing an impression without also tracking the user.

Exactly right. Imagine you have a list of 10,000 people who have visited your website whom you want to target with ads. You don't want to show anybody your ad more than ten times. You're using an ad network that reaches ten million people. Your ad will only appear when the trackers tell you that someone is on your target list.

Firefox is likely higher because FF users are more likely to have Ublock or some other content blocker installed. I'd also guess FF users are older, on average. Some will remember a time when FF was the clear choice over IE6. I used Chrome exclusively for a long time, then went back to FF once Chrome started to automatically log me into Chrome with my Google account.

There's a whole generation of users who first experienced the web via mobile browsers that don't block anything. When they become old enough to start using laptops and desktops, I imagine that it won't occur to them that an analytics-free and ad-free world is possible.

>I expected these numbers to be higher.

Home vs office.

On my company laptop I am often not allowed to install software (but I'm allowed to develop the software that companies trust to handle billions of dollars in transactions) so my usage would look 60% chrome with no add blocking and 40% completely locked down firefox.

> I expected these numbers to be higher.

The actual numbers might be higher, the article notes that these numbers still don't include anyone who blocks first-party JS completely.

I don't know if any of the sampled websites actually work without first-party JS.

I expected these numbers to be higher.

These numbers are iffy or at least, very poorly described. They're not a percentage of HN or Reddit users - in the HN case, the sample is HN users who clicked on a front page link to a post about switching to Linux from MacOS. It's a fairly small sample biased in ways that are unknowable when all you have is that one sample. As a methodology, this is flaky enough to not warrant the headline and the significant digits in these numbers.

"it's only 3% of our users so we don't need to test on FF" to "it's a quarter of our user base, we should at least test it"

To nitpick. Starting with 3/100 FF, times 8 unaccounted for, you get 24/121, 20%.

Why 121? I suppose visitors are counted by more reliable web server logs.

If you're counting 3% Firefox usage in your analytics, that's 3 users in every 100 using FF, and 97 using other browsers. If we assume actual FF usage is 8x what's being reported, you actually have 24 FF users and 97 other users, for a total of 121.

I am amazed at how non-techy people use the internet. I teach college and I will sometimes have students go to some web site. I am amazed at how few use adblocking and just accept all the ads and popups and overlays and crap. Even more than that, I will see them using something like google docs and google will put an overlay for some new feature and they don't click the X to close it. They just type away with that overlay in the corner. That drives me crazy. I don't know how they do it.

there are too many popups, especially now with how many cookie pop-ups work. So the easiest way is try to ignore it if possible. Ok, yeah, the best way would be to block it completely but as this shows again: humans are lazy creatures. Every click is a click too much.

Yes, and it goes further than that: "Should I add an ad-blocker to your browser for an ad-free internet experience.. it'll only take 2 sec." and they respond: "Nah, not needed".

And also "You are using the worst browser available to you (Samsung), shall I install Firefox?" and then "Nah, don't care".

The Samsung browser isn't actually that bad at all. Has some neat features that can't be found in Firefox or even Chrome, like the enhanced video playback tools. Plus it supports adblocking plug ins (though only FF has the much better ublock origin). Keep in mind that tons of website just outright won't work with Firefox mobile too, and it has a very annoying bug that has been known for years where all your tabs will very oftem reload whenever you switch to another app and come back. And that's regardless of how much free ram you have. Samsung browser just... Works.

> Keep in mind that tons of website just outright won't work with Firefox mobile

I know this anecdotal but I'm using FF on Android (Lineage OS without google apps/micro g etc. - only F-Droid) and the only site that causes issues is google.com. For some <sarcasm>unknown</sarcasm> reason it servers images in much lower quality. Of course it's not a technical limitation. If you change your UA to chrome everything goes back to normal. That was the biggest reason I moved to DDG.

> it has a very annoying bug that has been known for years where all your tabs will very often reload whenever you switch to another app and come back

I definitely don't have this issue. I have over a thousand tabs opened, it definitely doesn't reload all of them while I'm switching between apps.

I've had some issues with layout mostly and I never use google products on Firefox, but that may be due to ublock being aggressive with it's filters. And yeah the tab reloading is weird because it only affects some people and when it does it's very constant. Honestly I didn't mind since I'm not affected by the bug anymore since I've upgraded my phone.

The real problem was the add-on removal. I know you can still get them through the collections work around on Firefox nightly but... It's a pain and I'm still honestly baffled that Mozilla would just remove one of the only "selling" points for their mobile browser. I'm not averse to change and I get that it is sometimes needed to cut complexity, so I got why they needed to depreciate stuff likd XAML. But in this case afaik nothing was communicated, the new engine already supports the add ons... but only on nightly? Very puzzling

Ah, could be. But I was not really referring to how good the browser is, but the extent to which I trust Samsung to protect my privacy. They are particularly greedy for your PII on their mobile phones. For instance, after I minimised permissions for my apps, the Samsung Gallery app suddenly popped an "Allow location data" dialog coming from Foursquare.

Edit:

Here's a video showing the Foursquare dialog https://www.youtube.com/watch?v=7_e-P0hy5QY and also:

"For Korean phone giants Samsung and LG, Foursquare's API will be used in some of their default apps. If you take a picture using a Samsung Galaxy S8 or S8+, the phone will tag your location based on Foursquare's Places database."

https://mashable.com/article/foursquare-asia-tencent-samsung

I don't trust any Samsung software to respect my privacy.

Even with an ad-blocker, there are still huge cookie and "join our email list" banners. Banner blindness is just the way things go for most people. Fun little anecdote, in high school I had to work at a pizza joint but the screens where they showed the orders had a gui that looked just like a website. Thanks to banner blindness, I missed important info all the time because it was in places that brain had internalized as looking like ads. I had to really focus on it.

Last year I was building a "Cookie Consent" banner for one of my sites. I wanted to be transparent as possible with it so I had this box come up in the corner to ask for permission, big blue button for OK, big red button (of the same size) to decline, zero dark patterns.

Looking at the analytics after the fact made me realize why these cookie consent boxes use so many dark patterns. My acceptance rate for the box was like 4%, I tossed out cookies (and the banner) in the next update.

I never click on cookie consent banners, period. It has nothing to do with me not noticing them. I just leave them where they are unless they are too intrusive. If they're too intrusive, I leave the page.

The reason that I don't click on them is because I've had bad experiences clicking on them in the past.

Zapping them with uBlock is also a good option.

True.

I've been using an old version of Waterfox until recently, when I finally had to change to a modern browser. Looking at the alternatives, I chose Firefox as the least bad of the available options. On the plus side, this means that uBlock/uMatrix is an option for me now.

Big game changer that, right-click what is bothering you and choose "Block element", then drag sliders to get it perfect. Done.

Nice, hadn't seen those sliders before. Must be a new feature.

It's been around for some time. Not sure how long, but I don't remember it ever not being there.

I still don't see those sliders. Is it something you have to enable in the preferences? Is it mobile-only or something (I don't use a browser on my phone at all, so I wouldn't know...)

I see them now with Firefox on desktop, didn't enable anything specific. They definitely weren't there when I last used the blocking feature, which was probably some time last year.

The best cookie banners have a small neutral button labeled OK to accept only required cookies and then, on the right, a happy big blue button to Accept All. Transparency!

https://how-i-experience-web-today.com/

The best way to prevent that is by disabling JavaScript. Generally, JavaScript is what enables such banners to popup and obstruct your vision with newsletter requests.

The best way is to install uBlock (or your preferred ad blocker), not to disable javascript.

Javascript is like a knife. Do I want people who know how to use a knife in my house? Yes of course. Do I want people that stab other people in my house? No. Nowadays there are tools to leave those people outside your door.

It's time to end the association between javascript and bad web practices.

I use uMatrix. It allows me to block all third party scripts by default, easily unblock them one by one if I need to, and save my changes for sites I visit often.

And well upwards of 95% of all Web users (and I'm willing to bet closer to 99%) ... will not or can not use such tools.

(I support several. Tuning sites where uMatrix or even uBlock breaks stuff is a constant chore.)

Yes, many people can't or won't use uMatrix. And yes, it's a chore to tune sites. But we live in a digital jungle, it's program or be programmed.

An answer to a systematic and institutional abuse relying on individual actions which the overwhelming majority of the population will not or can not undertake ... is inherently unjust.

I use uMatrix. I appreciate the hell out of uMatrix. uMatrix is not the solution to the present level of abusive dark Web patterns.

NoScript goes a long way to fixing this. Yes, it's a chore, but it's less of a chore than the modern web.

There are cosmetic filter lists that filter out most of that stuff, e.g. Fanboy’s Annoyances.

I don’t use adblocking because if a site is so revolting that it requires adblocking to make it usable, I just don’t visit the site at all.

I used to use adblocking but the tiny site Distrowatch made an impression on me. The owner rigged something so that those using adblock also would not see non-ad images. He said it’s really not fair to visit a free site and then block the thing that allows it to be free.

Also, I then read that one of the adblockers would take payments from advertisers to unblock their ads. What a racket.

So I block no ads. This does mean that I don’t look at the vast bulk of news sites because they have obnoxious ads. However, most decent news sites are now pay walled, and those that aren’t are junk anyway. I get my news from some sites I pay for, and from non-profit email newsletters, and from sites my employer pays for. These have few ads.

So I have found no loss from not using adblock. Adblock is like putting on a bulletproof vest to walk through a warzone. Better to just keep out of warzones.

Adblock is like having a bodyguard running ahead of you, making sure the neighbourhood is still safe to walk, just in case someone shady moved in. Added bonus is he drags you along on a rope so you can move faster and bring more groceries from the store.

Adblock makes my slow computer and phone bareable to use on "modern" homepages. Without them the load times multiplies.

My phone isn't even slow and ad blockers are required for most sites to be usable.

You all but certainly overestimate typical technological literacy.

About 5% of computer users have "advanced" literacy, defined as "Some navigation across pages and applications is required to solve the problem. The use of tools (e.g. a sort function) is required to make progress towards the solution. The task may involve multiple steps and operators. The goal of the problem may have to be defined by the respondent, and the criteria to be met may or may not be explicit"

https://toot.cat/@dredmorbius/106841164116074208

Even just general literacy and numeracy are ... far lower than you'd expect: https://nces.ed.gov/pubs2019/2019179/index.asp

An OECD 20-country survey gives the 5% "advanced" technological literacy statistic: http://dx.doi.org/10.1787/9789264258051-en

Jacob Nielsen discussed that at the time: https://www.nngroup.com/articles/computer-skill-levels/

This is a substantial aspect of my "Tyranny of the Minimum Viable User": https://old.reddit.com/r/dredmorbius/comments/69wk8y/the_tyr...

This data is incredible. I cannot believe how the most powerful/richest/most influential country in the world has these kinds of numbers. I notice that it tends to follow a standard distribution but even then, I was expecting the window to be more towards highly educated.

This is so depressing. I have been wondering for a while as to why ~40 percent of the country rarely if ever votes in elections. This might help to explain it.

This also shows that technology people are amassing an unbelievable amount of power in their knowledge of how these systems work and operate given that the masses don't know how to weld that knowledge.

Note that most countries report only basic literacy rates, not functional literacy levels.[1] The apparent poor performance of the US here is largely a function of its own investigation and reporting of of the full extent of literacy.

Note too that lack of English literacy is pronounced in regions with a high immigrant or migrant workforce and population. Illiteracy can exceed 30% amongst Texas counties bordering on Mexico in particular. It's possible that many of those testing with no English proficiency are at least somewhat literate in Spanish or other languages.

But you're mostly confirming my earlier statement: you very likely overestimate typical technological literacy.

________________________________

Notes:

1. That I'm aware of. If anyone has references on comprehensive functional literacy assessments elsewhere than the US, I'd appreciate it. I've not found any on a somewhat cursory search.

>https://toot.cat/@dredmorbius/106841164116074208

Fediverse is usually hardcore web 2.0 "runs only in latest Chrome" "I remember entire caniuse.org" sort of stuff, not much different from facebook. So its first adopters can be only non-technical users, but they can't be first adopters of new technology.

Sorry? I'm not understanding the connection here.

Bear in mind that many younger people grew up with mobile computing first. So they're not as familiar with the "power user" desktop computing behaviors us older people have developed.

Not dismissing that Google Docs overlay may be explained by any of the following: (a) learned behavior from mobile computing that very few things are configurable (mobile apps tend to have far fewer preferences on how things are displayed; (b) pop ups on mobile often don't close when clicked; and (c) they are used to operating with very tight screen real-estate. I think (c) is most likely.

As someone brought up on desktop computing, I fight to eliminate anything that needlessly wastes screen real-estate such as bloated window chrome. But I think someone brought up on small screens might actually feel agitated by "too much" space.

I think there might be a 4th option, which is that users are trained that clicking on anything other than links will result in pain. Clicking a dialog you didn't read may take you to another page and ask you to enter information. Or it may opt you in to something you don't want or need. It's just a needless distraction. This gets back to what I was saying about "cookie consent" dialogs a few weeks ago. I never click on them. I usually read in Reader view to not even see them. If I can't get the content without clicking something additional, I simply press the back button.

I know a CTO with plenty of wealth and space that sits at a kitchen table with a laptop instead of setting up a home work station.

I know information workers who have workplaces that would gladly pay for nicer monitors or keyboards but don’t bother to even request them.

I know a couple with a brand new house that has a miserably squeaky front door that could be silenced a half dozen ways in under a minute.

People of all kinds contortion themselves into knots, giving little notice to the daily, near constant twinge of their circumstances but don’t improve them.

Tolerating ads is just one example of this.

A kitchen table? That's way more fancy than I bother with. I do most of my programming work from a couch or a hammock. It's a feature, not a bug. I have a workstation with a fancy dock, keyboard and trackpad, but I never really feel like using it. I will, however, close every unnecessary popup that comes my way.

I'd love to work from a hammock, but I get crazy neck and wrist pain when I use a laptop for several hours.

One contracting job I had, on my first day one of my tasks was to do some grunt work right in front of the bathrooms. The men's door was squeaky as all hell.

"Oh don't worry, it's always done that."

I lasted as long as I could but I finally walked over to the shop crew and asked if I could borrow some WD-40. "I'll bring it back in 10 minutes."

10 minutes later it's all "Thanks guys!", "No problem", and "Wow you really did mean 10 minutes!". [Note: I'm a man so there was no problem me stepping into the men's bathroom to make the repair myself].

And the door never squeaked again. My sanity was saved; it turns out I would be doing a decent amount of grunt work in that same area over my contract.

If anyone else ever noticed, they never said a word. The door had always squeaked and literally everyone either didn't care or did nothing about it; it just was how it was.

As an aside, next time, see if there's a silicone lubricant or other lube spray available. WD is good for what it was designed to do, and is barely acceptable as a lubricant. Better than nothing though!

I hate those tutorial popups and refuse to engage them.

Why are you directing your students to ad-laden sites that are burning CPU?

I don't use an ad-blocker, and nor do I accept sites with unreasonable ads. I block the whole site by closing the tab or hitting the back button.

I'm a very techy person and I interact with modals as little as possible, even if that means ignoring a chunk of my screen. I have a probably-unreasonable sense that it's only going to trigger a bunch of JS or something unsafe [1].

[1]: https://archive.is/TZ7oe was the best, but is kinda dead now. https://blog.malwarebytes.com/threat-analysis/2016/01/clickj...

Not only that but some of them actually think there's a moral problem with blocking ads.

Perhaps if the advertising industry didn't already prove for 20+ years that they are entirely made up of scum, I would agree. People use ads to make money. I feel bad that they aren't making money passively that way.

But I grew up with ad networks, including Google themselves, turning a blind eye to deceptive ads, bait and switch, fake window popups, and outright scams. The online advertising industry has done everything they could to fuck us over and big tech companies were complicit in the crime.

So that's why I don't care about running an ad blocker. Maybe the advertising industry will have a code of ethics after I'm dead, but until then I'll keep using uBlock Origin and NoScript (and other extensions) to screw with the ad business as much as possible.

Putting aside the ethics, advertising turns the internet into a race to the bottom. Ad blocking is good for the internet because it means that your ads had better be good, minimally intrusive, and also be integrated with the content you are providing. In other words, do things like preroll and midroll sponsorships and referrals rather than slap a bunch of banners on your site that peddle crap du juor.

how are ads passive? Sites have to actively generate new content to get more views.

Technical literacy among young people is so depressing. I'm 36 so I grew up in the last of the non-digital native cohort - I vaguely remember life before the internet. But growing up surrounded by computers and the internet, but seeing many of my peers miss out, I was certain that the next generation would be so much more tech savvy. As full digital natives they will learn to code, understand the protocols on which their primary communications are built and just be steeped in this stuff from birth.

How naive I was. No one actually cares. It was like assuming everyone born after 1908 would fully understand how cars work.

Been blocking pretty much all advertising on mobile and desktop for years and I couldn't go back. At work I see how the internet is SUPPOSED to look like and it is horrible.

Why don't you use adblocking at work too?

I was doing some pair programming with a colleague not too long ago, and after several minutes he was said something to the effect of "Could you close that [expletive] dialogue?" that had popped up to tell me that VS Code couldn't deal with some file extension or other. It had apparently been there the entire time.

I hadn't even noticed it (at least not consciously). I've apparently been trained somewhere to ignore dialogues, and I would hazard a guess that it has something to do with the prevalence of hostile UX patterns.

We had to make an adjustment on our site to account for that second one. I had just assumed if you have some kind of notice that can be closed, people would close it after they'd had a chance to digest the info (or decided they didn't care, or whatever). But no, turns out the vast majority never close anything unless it's physically preventing them from using the site. (Of course the vast majority also never bother to read anything that doesn't prevent what they're trying to do either, but we already knew that.)

Oh it's not just non-techy, I think you/we overestimate the techiness of techy people.

I have yet to meet a single person, and yes they are all in the Bay Area tech industry, who knew about pi-hole before I told them.

In terms of product tours, I'm totally on-board with your students. I don't know how to access that product tip later on, often times revisiting a product tip later on is not even implemented so I'd rather have it stay in place until I have some time to study it. Hopefully I'll be off of the website before I need that.

As a developer of an ad blocker[1] our stats would seem to back up what Plausible has found:

- Despite having equivalent desktop (macOS) and mobile (iOS) apps, most of our users (> 75%) use the app primarily on desktop

- Most users say the key reason for use is for privacy protection; even with Safari providing some inbuilt privacy tracking protection

- Our app is focused on providing a simple 'set and forget' ad blocking approach; so you would think the key audience would be less tech savvy users. However a large proportion of users are tech-inclined and knowledgeable users.

- A lot of tech heavy websites are some of the worst offenders in terms of tracker usage and advertising. For example, The Verge can load 3.5x faster simply by using an ad blocker that blocks the on-page trackers and ads[2]

[1] https://www.magiclasso.co/

[2] https://www.magiclasso.co/insights/difference-adblocking/

Can you make this for Android?

Google has no incentive to support effective ad blocking on platforms they control (Android and Chrome). This has made us reluctant to develop for or invest time on these platforms – the platform owner would be working against what we would be trying to achieve.

What? Apple is even worse. There was a big cry when Google wanted to nuter adblockers API but Apple have done exactly the same thing. You fell into a pr stunt. Apple doesn't care for privacy. Just look at how they scan you pictures..

Apple literally built an API for ad-blocking. How is that worse?

Are you referring to the "content blocker" API? Because that basically forces plug-ins to supply a list of blocked domains. Just like Google's upcoming changes. Neither allow you to run your own engine for blocking like what uBlock Origin does.

No, other cloud providers scan all the pictures you upload. Apple does a careful private set intersection (partially on the client, partially on the server) on all the pictures you upload to collect strictly less data than other cloud providers.

All prominent smartphone OS come with an advertising id. It wouldn't have to be this way. I think Android is open source done wrong to be honest.

But yes, in the end they are both crappy vendors.

On Android you can use Firefox + uBlock Origin.

Add Blokada + Newpipe to the mix as well.

I don't recall ever having heard about Blokada before, but looking it up now it doesn't seem recommended: https://gitlab.com/fdroid/fdroiddata/-/merge_requests/8536

(I created that merge request)

Blokada's UI is without peer and so it makes for a very good "just works" for the majority (in fact, from what I know, it is the most downloaded DNS-based content blocker on Android by far).

However, it is disappointing that some of their decision-making is found wanting: https://www.reddit.com/r/privacytoolsIO/comments/papgeq/any_...

Forgive the questions, but you seem to be a good person to ask.. How does Blokada actually work? Its FAQ claims it:

> prevents apps and browsers installed on your device from sending your private data (known as tracking fingerprints) to the Internet.

Is it doing some kind of packet inspection?

As a secondary layer of blocking I use DNS66 which intercepts DNS requests and fails them for blacklisted domains, by installing itself as a virtual VPN - essentially a cooked /etc/hosts for Android.

Would Blokada work alongside that?

> prevents apps and browsers installed on your device from sending your private data (known as tracking fingerprints) to the Internet.

For now, Blokada's utility is limited to DNS-based content blocking. It cannot and does not prevent most forms of fingerprinting.

> Is it doing some kind of packet inspection?

Yes, only DNS packet inspection, but even for the only thing it does, it is clumsy: It leaks DNS requests; that is, Blokada does not trap all DNS traffic on port 53, and it does not handle DNS queries sent over TCP. DNS66 has these same issues, too.

> As a secondary layer of blocking I use DNS66 which intercepts DNS requests and fails them for blacklisted domains, by installing itself as a virtual VPN - essentially a cooked /etc/hosts for Android.

Blokada uses the same trick (I mean, core parts of Blokada 4 code-base does bear similarities with DNS66 which preceded it... Blokada 5 however was re-written in Rust).

> Would Blokada work alongside that?

No, it cannot. But: Apps that support "DNS proxying" (like Nebulo [0]) can. It is quite an involved setup. I'd simply use Nebulo over DNS66, as it is not only more capable but also encrypts DNS traffic unlike Blokada 4 or DNS66.

> ...but you seem to be a good person to ask..

A disclosure, rather something to keep in mind: I have been accused of spreading fud by the Blokada lead developer and using it to "market" a "competitor" app I co-develop. In my defense, it wasn't / isn't fud what I spread, unless fud === uncomfortable truth.

[0] https://github.com/ch4t4r/Nebulo (fixed link, thanks u/NoGravitas)

Would you mind to suggest any alternative to Blokada? Thanks!

https://github.com/offa/android-foss#-ad-blocker and https://github.com/offa/android-foss#-firewall

Github link for Nebulo appears wrong: is this the correct one?

https://github.com/Ch4t4r/Nebulo

Thanks for the info!

This is the way. Although the built-in blocker in "strict" mode does a darn good job as well with a rare site breakage

And uMatirx, but Firefox put stop to that one.

Pretty sure Firefox did no such thing, instead IIRC the developer (gorhill) of both addons (uMatrix and uBO) realized that uMatrix was pretty much redundant given the options available in uBO and thus archived uMatrix.

We had a HN discussion where users brought up situations where uB could not replace uM.

IMO what really killed uM was major API changes by chrome and Firefox.

In addition to uBO, I also use TrackerControl.

https://f-droid.org/en/packages/net.kollnig.missioncontrol.f...

How do you collect your analytics?

On our website we have no analytics or trackers installed at all. The app usage statistics come via the Apple App Store. App Store users can opt out of these stats via an OS-level setting.

Opt out? It should be an opt in system in a for-privacy app.

> Opt out? It should be an opt in system in a for-privacy app.

I think it's pretty clear that they are discussing an App Store policy, not analytics collected by their app.

Apple does ask you when you set your phone up if you want to share analytics or not. I consider that opt-in, since you're given the choice up front.

Great, now it's equally as privacy-friendly as Windows 10 /s

I'm curious how does this compare/align with 1Blocker ?

Why is it important to have "privacy" from someone selling me a new video streaming service, but not important to fight against censorship and policing of content by tech giants?

My experience running a web server also backs this up. The number of GET requests to the back end is around twice as much as I see in Google Analytics.

It appears that Magiclasso is not open source. That being the case, how can anyone trust your claim that "Magic Lasso Adblock doesn't see or have access to any of your web pages or browsing history?"

The Safari content blocking API makes sure of that. Content blockers can only provide a list of rules for content to block (based on URL regexes, CSS selectors, etc) but can't actually access the content itself.

Are there any ad blockers for iOS that can block ads in google search results? Or is that kind of thing essentially impossible?

I use 1Blocker which blocks those (checked just now), I don’t see why other apps could not do the same.

AdGuard is the best option, plus it’s free.

One of the reasons for mobile apps is that adblocking is disabled. Explains why Reddit promotes it so much.

Similarly, wrapping websites like Discord or Slack in Electron also gives the website owners full telemetry and tracking that they can't get in a tech savvy browser.

Would an always on VPN, a remote pihole be the only way for privacy?

Honestly, if they'd abuse their power given through electron - surely, we the users would start boycotting one way or another. Ultimately, this purpusefully regressing UX for revenue - nobody likes it except the finance department.

A DNS with adblock blacklist is a simpler solution. Also, it's the only way for system-wide adblock for rootless Android.

Not sure if VPN would help you much against telemetry in a mobile app. A native foothold in your phone's system gives them access to much better data than they could infer on the server side.

E.g. if I wanted to know where you're hailing from, I'd browbeat you into granting me Location access privileges. If that's too difficult, I'd get you to grant me Files/Photos privileges (this one won't raise too many alarm bells with apps like Discord or Reddit), and then try to read EXIF geotags off your recent photos.

Wouldn’t it be possible to fingerprint users just based on the images they have installed?

It doesn't need to be an actual VPN to somewhere else. Lockdown for iOS can be run without using the VPN server.

> Explains why Reddit promotes it so much

It must be very disheartening to work on reddit's mobile site. Not only is it deliberately made a miserable experience by forcing you through AMP via Google and insisting you "continue in browser" every time, but then you're greeted with a banner that outright says "this page is better in the app".

> It must be very disheartening to work on reddit's mobile site.

You can probably cross "mobile" out of this entirely.

Reddit have spent three years now building sites that are worse in every way than the decade old junker it's trying to replace.

Say what you want about the implementation but the new design is definitely better for watching memes. And I wouldn’t be surprised if the design doc simply said “focus on memes”

It’s definitely not better then old reddit + reddit enhancement suit.

So, maybe, they should have improved the old design instead of creating that insanity that is newreddit.

Yes but if you use old.reddit.com and specify just the subreddits you want (eg http://old.reddit.com/r/truereddit+technology+science) then you can still get something that's not too crappy.

I wonder how long it'll be before they phase out the old site.

For now I still use it because it's significantly faster and easier to use, but I strongly suspect they want me on their new site.

As long as reddit's API still exists, someone will make a site that recreates the old reddit experience.

I wonder how long they'll keep the API live with so few restrictions and fees. Twitter makes huge amounts of money from selling the kind of data that PushShift gives away for free from Reddit.

Reddit’s mobile site looks like it was made by a group of 15 year olds and doing a my first website tutorial, falling into all the traps there are.

The amount of times it crashes on mobile platforms is insane.

Agreed. I don't mind the new layout but it is still amazing that after years of this new layout that at least once a week I go to reddit and it can't load comments. I'm not a developer so maybe there is a valid reason but as a user it just seems ridiculous that the site can't do its main function reliably.

It's ironic that I find old.reddit.com to provide a vastly superior mobile experience than the mobile-focused replacement is supposed to be.

> It must be very disheartening to work on reddit's mobile site.

If they cared that much about whether or not what they were working on made the world better or worse they'd never have taken a job at Reddit.

Literally this, it's hard for me to even consider a job at Reddit without the companys' reputation coming to mind.

Reddit literally does not care about the usability of their mobile site: https://old.reddit.com/r/mobileweb/comments/o7wo1s/this_subr...

Pretty much - and there's a nice app developed by an Oxford student that does this for Android: https://trackercontrol.org

It works very well, I highly recommend it.

> One of the reasons for mobile apps is that adblocking is disabled.

That and revenue is much higher on apps than on the web. I made an android apps for a website. And just alone the android apps made more money than the web version. With less ads.

Maybe it having less ads made it a better experience, thus bringing in more ad revenue?

DNSFilter[1] does the trick even for android apps though, but yes it's even less mainstream than in-browser ad blockers.

[1] https://www.zenz-solutions.de/personaldnsfilter-wp/

I use a third party Reddit app! No ads and better experience than web

Blokada and others work by using the VPN functionality in Android, to implement DNS blacklists. Alternatively, AdGuard and NextDNS run DNS servers where you can customize the block list, a remote Pi-Hole as you said.

I'm using NextDNS as the system-wide private DNS on my Android phone, it works great and eats less battery than Blokada.

I use Little Snitch on my Mac desktop. There is about a week burn-in where you are constantly clicking to accept things. After that it’s great.

I found NextDNS to be relatively convenient and easy to set up even for a lay audience. Definitely easier than a PiHole or a custom `dnsmasq` setup, and it offers mobile configuration client apps.

What I do not know is if it will work also when apps begin using DNS over HTTPS… I suppose not?

I have been using NextDNS for couple of days, but since I don't have a static IP, it's obviously not so convenient, I have to reset my IP every time it changes But otherwise, absolutely great, awesome statistics about blocked/requested domains, countries, etc...

They have ways to automatically detect your IP so you don't have to update it manually.

Unless they provide a VPN it is only blocking the not-so-bad-actors. Everyone else use hardcoded DNS IPs. If you look at traffic from an Android phone you will get lots of DNS requests to Google DNS no matter if you use NextDNS or not. If you only provide one (primary) DNS IP in android 8.8.8 8 (Google DNS) will even be used by default together with your DNS provider. Same is going on in iOS. If they do provide a VPN then it isn't really for a lay audience IMO but it is the only thing that isn't like pissing in the wind.

IOS have the NextDnS app as a vpn setup. So I guess yes?

NextDNS offers a DoH endpoint and is a selectable TRR in Firefox. Unfortunately that doesn't help with apps doing DoH to bypass DNS blocking. The current state of the Internet / computing is a bit problematic, but there are ways forward.

What I do and recommend everyone to do is:

1. Run an edge network device using network access controls and filter which devices on your network get outbound network access (in my case just the gateway device). Block all inbound traffic except what you choose to pinhole, block all outbound traffic except ports you choose to add to the allow list.

2. On every client device run a local application firewall (I like Vallum and Little Snitch on MacOS as examples) and filter applications by domain + port on outbound requests, block all inbound requests.

3. On every client device force it through a VPN to a gateway device internal to your network to get internet access, anything that falls off the VPN is then blocked from the internet. The gateway device can forcibly route traffic and perform additional filtering

4. On every client device, configure it to use an internal DNS on your network with a fallback to a trustworthy external provider, have the internal DNS use a trustworthy external provider over DoH. Block outbound DNS at the edge device (blocks all non-encrypted lookups).

It's kind of a pain, and a mess, but it does greatly restrict the damage that rogue IoT / Smart devices can do.

I personally use doh_blacklist with around 170ips, where I block outgoing traffic for known (publicly and not so) internet reachable doh resolvers. There is no problem(+) - everything works perfectly.

ipset create blist_doh hash:ip hashsize 1024

for ip in `cat /etc/bin/blist_doh.txt`; do ipset add blist_doh "$ip"; done

iptables -A <insert some iptables placement specific to your outgoing/forwarded traffic> -m set --match-set blist_doh dst -j DROP -m comment --comment 'SPY:all ext DoH BLOCKED'

Still, as of 2021, doh rule is around 2% traffic logged compared to my other rule, where I simply block outgoing 53/udp (except my resolver). a LOT of your devices ignore your dhcp dns settings and try to circumvent it going directly to shady 8.8.8.8 etc.

(*) you shall every few months check and update it.

> Would an always on VPN, a remote pihole be the only way for privacy?

Maybe for now, but it's just a matter of time until use of DoH to circumvent your attempts at redirecting DNS becomes more widespread as well.

Time to ban/firewall DoH then.

Good luck with that. I strongly object to DoH, but it exists and we have to deal with it.

The only approach that I could come up with to do so was to install a proxy to MITM all HTTPS connections to allow me to filter out DoH requests.

As it intentionally looks like HTTPS traffic, that's a different game; you kind of have to whitelist rather than blacklist.

Here's an idea that supposes you have some server available: Force all traffic over your network, even when roaming (VPN). Squid+Privoxy as HTTP proxy (your choice if you want to DPI and add own TLS CA as root on devices, in which case you can actually block it, or just do HTTP CONNECT). Optionally also set up a SOCKS5-proxy (microsocks). Configure all applications to go over proxy. Applications that don't support it can usually be "proxified" with tsocks or proxychains4.

Now you have more fine-grained control over your traffic. You can do fun stuff like setting up several incoming ports, several outgoing proxies (including Tor/I2P) and allow/block based on destination, domain, protocol, etc.

Finally disable outgoing traffic on your devices/route to a black hole. Or route it via your VPN at least, where you can do some rudimentary stuff with nftables/iptables. Oh, and something not everyone knows: you can route traffic based on uid/gid on Linux, so on your workstation you can leverage that too selectively control traffic on an application-basis.

On non-rooted Android and iOS, I suspect this is not perfectly achievable but if you have the time you can get pretty far otherwise. It's an arms race and I feel like we can keep up.

I use nextdns to block ads on my phone

> Similarly, wrapping websites like Discord or Slack in Electron also gives the website owners full telemetry and tracking that they can't get in a tech savvy browser.

Discord, Slack and other similar webapps can (and maybe do) send telemetry in the same connections used for the app's features. You can't reliably block that.

Why is this even at the front page of HN? I give them credit for the brilliant marketing.

This is an ad disguised as an article targeting the "tech savvy" by bundling HN and Reddit (a truth + a lie makes the statement more true), a common clickbait tactic

> This makes sense especially considering how difficult it is to install an adblocker on Chrome, the most popular browser on mobile devices.

Nope, it is really easy: is just an extension https://chrome.google.com/webstore/detail/adblock-%E2%80%94-...

It makes me think Reddit users are inflating/manipulating this article w/ votes and comments

Correction: "difficult to install adblocker on mobile devices"

The headline is the perfect anti-Google HN-clickbait, and it is quite misleading. These people just run any ad-blocker, they mostly all block Google Analytics as a side-effect. The majority of those 58% probably don't care about GA specifically.

To be honest, I've used Brave for a long time and I didn't realize it blocked GA.

It says it's difficult to install on mobile

My bad, but the original premise still stands

No it doesn't.

Care to elaborate? Not sure why the HN title changed

I see where the disconnect was: the article had a subtitle of "68% of laptop and desktop users block Google Analytics"

Yet, their stated cause ("how difficult it is to install an adblocker on Chrome") refers to mobile users

A logic shift there - a bait and switch? Not sure if this is an error, or outright manipulation

This puts the entire article's credibility in question

Proof: in case they modify it like they did in HN's title

1. Title: https://ibb.co/Q87pDPr

2. Subtitle: https://ibb.co/nDshDCT

One sentence says that mobile users block GA less than desktop users. The following sentence states it makes sense since installing add-ons on most popular mobile browser is difficult. Context matters:

>At the device level, laptop and desktop users (68.2%) block Google Analytics more frequently than mobile and tablet users (49.9%).

>This makes sense especially considering how difficult it is to install an adblocker on Chrome, the most popular browser on mobile devices.

Thank you for the explanation.

How do you install extension on mobile Chrome? The link shows button to install on Desktop.

Well, for extra irony, I use the Steven Black /etc/hosts content to block on the order of 70,000 domain names, and plausible.io is in there. So I can't even read the article because I'm one of the people it describes.

Isn't the primary use-case for plausible that you can run the tracking entirely off of your own domain. Which means that blocking plausible.io doesn't really give you much.

it's just one of a list of about 68,800 hosts in the blocking data. nothing specifically intended to block plausible alone.

> > This makes sense especially considering how difficult it is to install an adblocker on Chrome, the most popular browser on mobile devices.

> Nope, it is really easy: is just an extension https://chrome.google.com/webstore/detail/adblock-%E2%80%94-...

The key word here is mobile devices.

AFAIK, ad blocking in Chrome on my phone is difficult. But with Firefox, I can easily install uBlock Origin.

I use a PiHole on my phone occasionally to block ads in anything that isn't Firefox, but I found that the OpenVPN client is a significant battery drain (~7% per hour).

Wireguard may use less power, I've not noticed any loss of battery life when using it across multiple devices.

I'm trying the option to pause the VPN while the screen is off again. I had done it before, but found that it frequently didn't reconnect to the VPN when I powered back on.

Worth noting that if you're already willing to setup a first-party proxy like Plausible does in this comparison, you can do the same thing with Google Analytics using either the NYPL project [1] or send whatever you want to the Google Measurement Protocol API [2]. You can usually send whatever you want through a first-party proxy in basically any competent analytics product. Analytics is not ads.

I find it this comparison a little misleading because Plausible admits that their own script/endpoint are blocked by adblockers, just to a lesser extent than Google Analytics [3].

[1] https://github.com/NYPL/google-analytics-proxy

[2] https://developers.google.com/analytics/devguides/collection...

[3] https://plausible.io/docs/proxy/introduction

That's why apps are so popular with ad driven properties. No cookie banners, hardly any limitations. And you get to keep users nicely inside the walled garden.

Anyway, I indeed use Firefox with ublock origin, multi account containers, etc. I also use Firefox on Android. A lot of stuff people assume they need apps for works just fine in that.

Same here. I particularly enjoy the mobile YouTube and Twitch experiences much better than their respective apps (mostly because of ad blocking).

This is why it's important to block ads and trackers at a lower level, such as using a hosts blocklist.

> No cookie banners

You still need informed consent in the EU, GDPR is not specific to websites.

This is highly inaccurate. The author is using Plausible via a proxy but Google Analytics directly (a very biased way to do things).

Secondly, PiHole, uBlock origin and most other adblockers also block Plausible analytics (there are discussions on their own Github Issues regarding it). If you are proxying Plausible to get accurate readings, same should be done with GA.

I don’t read this as Plausible vs GA but server-side anything (including a GA proxy) vs client-side GA. in that case, it doesn’t seem like these figures should be wildly wrong.

They definitely did, this is not a GA vs server side comparison. From the source :

> I compared stats between Plausible Analytics and Google Analytics.

This article clearly suggests that you should use their own solution, Plausible Analytics.

They are also JavaScript based, so how long before ad-blockers start blocking them too?

Isn't it time to come back to server side analytics?

They are blocked a whole lot too, see my comment in the thread: https://news.ycombinator.com/item?id=28365505

I don't get your comment, sorry.

By server side analytics I meant using a tool like AWStas or Webalizer which parse web server logs to tell you how much visitors you had on your site.

I can't see how it can be blocked.

They meant that "Plausible Analytics" are blocked a whole lot too.

exactly (can't edit original post anymore, sorry)

You were answering the first question not the second, totally make sense now.

Sorry for my misunderstanding.

you can serve the script from your own server (what they call proxy) and it shouldn’t be noticed by adblockers

Most adblockers I know block Piwik and that's fully self hosted in most cases. I know because I gave it a spin and the entire interface was broken because of my adblocker.

Smuggling tracking through first party proxies is certainly an easier way to avoid privacy protection systems on your customers' devices, but it's definitely not the silver bullet people want it to be.

Adblockers are fully capable of blocking subdomains and individual scripts by name, as well as forbidding certain outbound connections.

You'll probably have to use inline scripts and send data back bundled with actual data necessary for your site to run if you want to sneak past the user's defenses. But if a site has that much of an adversarial with its users, maybe it's time to stop using that site.

I've noticed several plausible.js's blocked by uBlock origin. Not sure how, maybe pattern matching? I doubt it was manually in any blocklists as it was a small personal blog.

That's the main reason we promote the backend integration for Pirsch [0] so much. In the long run, JS will probably not be sufficient, depending on your target audience.

[0] https://pirsch.io/

Thanks for mentioning Pirsch, their backend integration sounds like a modern solution.

Tools like AWStats or Webalizer are really outdated nowadays.

Plenty of filter lists block several instances of Plausible.

I welcome that; if I want to give feedback I'll do so myself. Analytics should not be opt-out.

It amazes me that over the last 10-20 years an astonishing amount of money and intellectual capital have been invested in adtech and crypto but nothing in micropayments. Ads are evil, but it will remain a moot point until we have working micropayments. We have (and will have) ads with all their downsides exactly because we don't have micropayments, because ads basically work as a substitute to micropayments.

I'd be interested in working on stuff like this; I used to work in a PCI-DSS Tier 1 company storing cardholder data on-premises so if someone wants to work on this and would like my help with this please reach out; my contact information is in my bio.

I’m still surprised more apps don’t have inbuilt marketplaces and then take a cut of payments. Reddit could do this. Users sell to each other through an Etsy like interface. Reddit takes 5% of each transaction. This solves Reddits existing inability to monetize through ads as successfully as their major competitors.

What would be sold on a Reddit market?

Like craigslist 2.0? They could capitalize on the fact that there are existing sub-communities around various niches.

Karma and insults ?

Special goods catered to specific subreddits.

On /r/fishing you obviously sell fishing supplies. On /r/$political_faction you sell bumper stickers with slogans. /r/nonbinary you sell pronoun pins.

It would be like an etsy or a shopify store for each subreddit with a UI that reflects that. It could also be a big Amazon style UI for a sitewide shop.

Moderator privileges and astroturfing opportunities.

Cryptocurrencies, ironically, enable micropayments. Orchid.com did an extensive write-up on how they achieve this (https://www.orchid.com/assets/whitepaper/whitepaper.pdf Chapter 5, Nanopayments) The Brave Attention Token is another example.

Although, I am not convinced micropayments would save the Internet from this ad-winter: It is hard to beat free at scale.

"Enable micropayments". Where are they disabled? And in what world is it easier to do a micro payment with cryptocurrency instead of one of your cards?

Not if the transaction cost is $10

There's a little bit of a fundamental problem here.

We're talking about paying a cent to avoid an ad. On a blockchain, work needs to be done to register that transaction. If the fee is a fraction of a penny, who is going to want to do the work? I get that you'll make it up at scale, but each transaction must be cryptographically secure (and therefore take up some type of resource), so there's a problem.

For "crypto" transactions, on-chain is no longer seen as a stringent requirement. In fact, the entire DeFi ecosystem wouldn't exist if there wasn't a cryptographically-secure way of doing transactions off-chain on second-level chains (0x, Polygon, Compound etc) or on chain of chains (Polkadot, Cosmos, Kava etc), or on chains built for payments (Celo, Diem, Stellar etc).

Beside, Orchid.com whitepaper talks about doing nanopayments on-chain, which is quite a radical approach.

Sure, if you put a layer on top, you can do whatever you want! It just adds more centralization, which isn't better than something like a Venmo that would be based on fiat.

Not all off-chain layers are centralised.

> but nothing in micropayments

That's not true. Web Monetization is built on top of the Interledger and does precisely that - micropayments for web content. When i wrote about it[0] and posted on HN[1] the overwhelming response here was negative, presupposing greed and lack of privacy of everyone ( as in the website would still track and run ads to earn more money, etc.). The solutions exist, people just don't want them, even supposedly privacy-focused people.

[0] https://atodorov.me/2021/03/07/please-support-web-monetizati...

[1] https://news.ycombinator.com/item?id=26375857

> overwhelming response here was negative, presupposing greed and lack of privacy of everyone

Because the history of other platforms which have done this, shows that ads didn't go away.

Cable TV, full of ads.

Satellite Radio, not too bad but still has ads.

But then again Netflix et al, zero ads

Spotify premium, zero ads

I think it's just a question of coming up with the right model. Assuming that failed examples are the rule isn't all that useful

Some Netflix originals have a lot of product placement

Product placement is a slight annoyance compared to ads. That said, I could see them becoming just as toxic.

I find product placement to be horrific. It was the final straw that got me to just stop watching TV/Netflix/etc altogether.

For now, Netflix and Spotify have no ads. I have a feeling that if these companies feel the pinch during a bad quarter or two, they will 100% start introducing ads. That's how business works, they want to maximize revenue. TV publishers have traditionally charged a fee on top of the ads. That demonstrates that audiences are fine with it. 100% Netflix and Spotify will eventually have ads.

> TV publishers have traditionally charged a fee on top of the ads. That demonstrates that audiences are fine with it. 100% Netflix and Spotify will eventually have ads.

Or maybe the huge success of Netflix and to a lesser extent Spotify is in no small part because they don't have ads and they know it?

> TV publishers have traditionally charged a fee on top of the ads.

While not wanting to get into a debate about what is and isn't an ad in the context of US public broadcasting, this claim is not true of public broadcasting worldwide.

Netflix doesn't have ads, yet. Spotify Premium doesn't have ads, yet.

Hulu didn't use to have ads on its most premium tier, and from what I know, you now get ads on that too. Corporations generally don't like leaving money on the table. If you can pay, you are even better target for ads.

Hulu, in my country at least, is ad-free on premium. It's not a hard thing to investigate rather than speculate[1]

[1]https://help.hulu.com/s/article/how-much-does-hulu-cost

> Netflix doesn't have ads, yet. Spotify Premium doesn't have ads, yet.

Both of those are replacement products for models of consumption that were previously pretty much entirely ad based.

> Because the history of other platforms which have done this, shows that ads didn't go away.

This is classic negativity bias, I can think of plenty of platforms that transitioned to paid models without ads

The reality is that you can't have it all.

You live with the assumption that everyone would just accept micropayments, and that's far from the truth.

The result would be content for a small portion of those with available income, and content for those without it (with parallel markets for content distribution under paywalls - like piracy).

That's even more messed up than the current advertising model.

Don't get me wrong, some brands would love that, to pile up those with available income and serve them marketing communication through press releases, reviews and stuff like that. I can see Apple applauding this.

Here's one cryptocurrency PoC on micropayments: https://web3torrent.statechannels.org/

More directly to your point, I think lots of people have made attempts at micropayment protocols. The difficulty is psychological rather than technical - the cumulative mental burden of repeatedly deciding whether to part with a tenth of a cent "costs" much more than value being exchanged.

I feel like most on the producer side are coming at this from a totally wrong angle. A better web isn't a web in which content is monetized in a different way. Consumers don't want to pay because most content on the web is not worth paying for. Not with ads, not with micropayments, not with anything. There's the saying about five web sites consisting of screenshots of the other four, but even with news, the vast bulk of it that isn't legitimately old media like Al Jazeera or Reuters is 22 year-olds being paid with exposure to summarize Reddit threads. Much of it isn't even that at this point and is probably just programmatically generated with no writer at all. When all you have to go on is a link, you can't possibly know whether something is worth paying for until after clicking, reading it, and finding out. But if the first thing you see is some pop up begging you to subscribe to content you've not even read yet, turn off an ad blocker to a server you don't trust, or even some gate requiring a micropayment, a whole lot of people are just going to go elsewhere for what is likely to be identical information content at a less annoying source.

If content producers are willing to put in the hard work of developing a reputation for quality output slowly, disseminated through trusted sources, surely augmented by some form of paid marketing but not by just spamming the web with links in the hopes that they can get clicks and then use dark patterns to keep people on the site long enough to monetize 20 seconds of their eyeball time, then maybe they can get people to willingly pay them. This is sort of what Substack is proving. The tiny number of writers worth reading are actually being paid by willing readers. But it's a tiny number. Micropayments can never solve that most web content is worth 0 dollars and 0 cents.

> If content producers are willing to put in the hard work of developing a reputation for quality output slowly, disseminated through trusted sources, surely augmented by some form of paid marketing but not by just spamming the web with links in the hopes that they can get clicks and then use dark patterns to keep people on the site long enough to monetize 20 seconds of their eyeball time, then maybe they can get people to willingly pay them.

What do you think traditional print media organizations were trying to do for decades before they gave up and embraced the new normal with layoffs and consolidation? Substack is an extremely, small, extremely premium part of what used to be the whole.

> Consumers don't want to pay because most content on the web is not worth paying for.

Maybe much content on the web is not worth paying for, but the vast majority consumers don't want to pay regardless of the quality, not because of it. It has literally been impossible for most news organizations to survive because people would rather read the advertising-funded "22 year-old" than a quality outlet where they have to pay any amount of money.

This is a complicated phenomenon and I can't possibly do justice to the complexity in the space of a link aggregation comment. Arguably, the truth of that very statement is part of the problem here. Our attention spans have shortened. When I was 8 and wanted to learn about something, I'd gladly dive into the library and read thousands of pages uninterrupted for hours a day. Now I'm here, skimming thousands of comments to try and figure out which seem interesting enough to make the link itself worth visiting, then possibly actually visiting it or possibly just putting in a tab I later close when I realize I'll never get to it.

Without any sort of gate to publishing, we're all inundated with information overload. So yeah, print media got their lunch eaten for many reasons, including being too slow to pivot to digital delivery at all, but also with the payment model. Outsourcing content curation to Hacker News or the people you follow on Facebook is free. I used to read the LA Times for two hours every single morning when I was in middle school and high school. Do I trust Hacker News more than I trust the LA Times editorial board today? Do I trust the LA Times more but not $6 a month more or whatever they're charging now? I have no idea, but I've changed my information consumption habits anyway.

At least part of the issue is the nature of news itself. Events happen in the world. Someone out there finds out and reports it to others. Eventually, it reaches me. It used to be that people being paid by the LA Times had a level of unique access both to the sources of information and to dissemination channels I could readily access, and that was worth paying for. Today, that no longer seems to be the case. A thousand different people are going to post the same information to a thousand different sources at exactly the same time. Which of those thousands of people deserves to be compensated? If you just split whatever the salary of an LA Times reporter used to be a thousand different ways, that isn't enough to make it into a viable profession.

Maybe information about important events in the world has become a public good in a world with such a low bar to publishing. We can try to invent technical means of preventing access and then charging for it, but it can never possibly be enough to actually cover the costs of all of the different people out there trying to publish, not with micropayments, not with subscriptions, not with anything. Maybe we need to just publicly fund some small number of professionals doing this for a living and anyone else that wants to try can do it without the expectation they'll ever be compensated for it. Expecting high-quality fact-based reporting paid for by consumers may just not be possible any more.

> Consumers don't want to pay because most content on the web is not worth paying for.

This can be true at the same as another truth: consumers would pay for some content on the web if it was convenient to pay small amounts for it on an ad-hoc basis. Nobody is suggesting that most of the web is worth paying for, but that doesn't mean that none of it is, and it also doesn't mean that subscription models (which mostly avoid the micropayment "problem") should be the only way to address this.

>Micropayments can never solve that most web content is worth 0 dollars and 0 cents.

The irony is that most of the actually valuable content is user generated. A site like Reddit could be actively curating all of the great content that people give away for free and people would pay for it. Then Reddit could actually pay their content creators. And -poof- we have high quality content that is worth paying for and people getting paid to generate it - all without the scourge of ads.

Micropayment are a non-starter, and even if they weren't, you'd still have ads. There are companies lined up willing to pay money to show you something they want to sell you, and even if your favorite news site was micropayment enabled, even the most righteous media companies would be fools not to keep taking that money to expand their ability to give you more and better news, as an example.

But to my first point, they are a non-starter because too much of the world has trouble managing their money, and mentally don't want to pay for something until they've seen it. Why should I pay a dollar to read an article that might be poorly written, full of inaccuracies, etc.?

The closest model I've seen to working is twitch. You have a central content platform, with silly cosmetic awards for subscription, and a sense of "credibility" among others there in interactions because you've been subscribing for x amount of time. This would require a more interactive news service, and hell, that I would pay for. If I read an article by an expert who answered (reasonable) follow up questions and owned their journalism, that would be incredible.

This is good to hear. I didn't think the percentages would be that high.

"Data driven design" is kinda overrated anyway. It's better to actually listen to your users. Stats don't show what people think, and they're often tweaked to show what people want to see.

Collecting telemetry is essentially doing an impromptu social science study, and it suffers from the same problems - collecting good data and making correct inferences is a difficult task, and it requires specialized skills. Social scientists are trained to do this, and yet they still fail more often than not[0]. A random business trying to run product development off automated metrics? They don't stand a chance.

This is not to say the data is useless - it's just worth remembering that, unless you have strong statistical background, you're probably reading your data wrong, so your metrics should be treated as weak evidence at best.

[0] - It's not snark, it's a corollary of the replication crisis. This is a hard job.

"Listening to your users" is just a more manual form of data gathering though. Also, it's not like "hearing what you want to hear" has been invented by statisticians, that is super easy to do when talking to users as well.

Some of the problems in talking to users:

- people will often tell you what they think you want to hear rather than what they really think. (Very few users would ever tell an interviewing dev that they think the product is unsalvageable, the dev team wasted several months and that the user would never ever use it)

- Users often don't know what they want because they don't understand the full range of possibilities. Sometimes they dream of tech that would violate several laws of physics, at other times they fail to realize that for-loops exist and just ask for a faster way to manually click a button a thousand times.

- Fads affect users just as much as anyone else. If there has been a surge of news articles about how Google is doing fancy AI stuff, you can bet the bakery on the corner of the street will mention machine learning when asked what they want to improve their business.

"Listening to your users" is a voluntary exchange with your users, not compelling data from them. The difference between the two is critical.

It’s amazing how far the goalposts have moved on analytical data.

I worked retail in The Time Before The Web and we collected and analyzed data from our customers. We had a door clicker to give us a daily record of traffic. We did detailed analytics on what items customers bought when, to inform inventory and staffing decisions. We did regular people-counting studies to evaluate our merchandising in different parts of the store. We did coupons in local papers for which we could directly track redemption. This was all in the service of making the store work better for our customers.

Now, you might say that those are all anonymous aggregate piles of data. Well, so are most web analytics packages, including many client-side JavaScript.

No goalposts have moved.

The difference between the two is that there's no question that counting heads walking through a door and watching inventory patterns are not personal to the customer.

With telemetry, including web analytics, there is always a question because there is no visibility as to their actual practices. It all amounts to "trust us". And we've had more than enough experience with these things to know that such trust is frequently misplaced -- so none of it can be trusted.

And GA especially cannot be trusted, and also happens to be pretty much ubiquitous.

As a side note, it's become just as dangerous to be in a physical store as online now. People who prefer not to be spied on must pay in cash, be sure to put their phones into airplane mode before entering the store, and so forth. And with the increasing adoption of face recognition in stores, it becomes risky to even show your face. In other words, they have to be as "on guard" in the store as online.

The percentage is misleading. it's not people intentionally blocking Google Analytics, it's anyone that runs any adblocker, which almost all block Google Analytics. I installed uBlock for my parents and they don't have the faintest clue what GA is. A significant portion of those 58% probably don't care either.

That's why I prefer goaccess(1) over any other tool that uses client side Javascript.

It's basically a real time website log analyser which gives you enough information to know whats happening on your website but doesn't require any pesky Javascript etc to do it.

Also since it is works by analysing your log files it can never be blocked.

(1) https://rt.goaccess.io/?20210826211303

By nature it won't filter out bots, however.

One of my clients' website traffic is composed of over 75% bot traffic. Server-side logs are unusable for anything other than site performance.

> By nature it won't filter out bots, however.

I have two questions about this:

- Since you know they are bots, why couldn't you filter them?

- On the other hand, couldn't there be bots that run JavaScript which would be tracked client-side?

> Since you know they are bots, why couldn't you filter them?

A decent amount can be filtered by UA. This is inconvenient because UA is a very large piece to log and index on, so you need to do UA processing to do anything useful with it and … well by that point it just becomes a chore and I suspect there's good logging services that do this better than you'd spend your time doing yourself.

> On the other hand, couldn't there be bots that run JavaScript which would be tracked client-side?

They exist but they're more rare by nature, because running JS at bot scale is expensive.

> well by that point it just becomes a chore and I suspect there's good logging services that do this better than you'd spend your time doing yourself.

Which is why the OP suggested using tools like goaccess.

Can I evade tracking by disgusing my traffic as bot traffic? Sounds like a startup idea.

You're likely going to end up implementing a poor man's Tor.

Oh I suspect you'll have a decent amount of sites not serving you ads or analytics if you put "googlebot" or "chrome lighthouse" in your UA.

If you use googlebot you’ll likely just get blocked altogether.

Using Lighthouse might be an interesting experiment though.

Goaccess does have a separate panel for Crawlers/Bots(1) but I think it's based on the UA.

There is also an option to ignore IPs/Referrers too which can work very well for such cases.

(1) https://goaccess.io/features#:~:text=Spot%20aggressive%20hos...

> One of my clients' website traffic is composed of over 75% bot traffic. Server-side logs are unusable for anything other than site performance.

I'm unclear how broad you intend that second sentence to be, but there's still a ton of info you can glean just from server-side logs:

- Referrer info, and, by extension, popular search terms being used to find your site;

- Paths on your site causing 5xx errors (so pages which might be triggering an error in a server-side script)

- Paths on your site causing 4xx errors, and associated referrers (might be broken links on your own site; might be stale search engine indexing)

- Mobile vs desktop access statistics

Finding this data among a bunch of bot-induced noise might be annoying, but if they're good bots and sending proper UA headers specifying their botness, it's easy enough to filter out. Even otherwise there might be typical bot-like behavior you can find and account for such as not sending a referrer header or trying to access known exploitable PHP scripts (in which case you should block that IP address for a few hours or days - there are programs which can do this sort of thing automatically but frustratingly I can't recall the name of one off the top of my head right now).

Granted, a lot of this can be spoofed, but I'm pretty sure the number of people sending spoofed referral or UA headers is dwarfed by the number of those (like me) who block Google Analytics and similar cruft entirely.

No, you're right, I shouldn't have written something so dismissive. (I do include error tracing as part of "performance" for what it's worth but those have their own system from within the app itself)

Frankly I would love to see some serious low-config solutions to analyzing server-side logs. Oh, especially Fastly. Client in question uses Fastly and it blew my mind to find out that there was nothing in place to answer simple questions such as "what are the slowest paths to respond", "which paths are a cache hit most often", "which paths are most hit overall", etc. And being able to look at various dimensions such as browser, bot traffic, country of origin, etc. If you have suggestions…

Any log analyzer will tell you which path is the most hit. For slowest paths, I think a server daemon could theoretically log how long it took to serve the page from request in to last byte out, but I don't know if any of them do that - you might have to set up a custom format for logging, and then from there you'd need to tell your analyzer how to interpret that field. For cache hits, I guess it'd depend on what sort of cache you have in mind, but that might be something you could only effectively log at the application level.

| frustratingly I can't recall the name of one off the top of my head right now

fail2ban

The middle way is using something like Pirsch's API [0] from your backend.

[0] https://pirsch.io/

Of course they do. Every invasive thing I'm made aware of, will be blocked.

My home network runs Pi-Hole, and every browser has uBlock Origin. The Internet has been a bearable place. And then I happened to show my 2 year old a cartoon from Youtube on a non-adblocked device.

My. Freaking. God.

It's full of ads.

The cartoon is 10 minutes. Every minute, an 15-second, at the very least, ad rolls in. Like fucking gnats by the river in the summer: you get one, two, and then you are swarmed in them. There weren't the yellow markers anywhere on the progress slider, too.

Not all of those ads were good for kids, too. One was, I kid you not, an ad for Jira. An ad for the fucking Jira, in the middle of a cartoon. Won't anybody think of the children!?

My daughter was very much not impressed. So was I.

I promptly got uBlock Origin running, and youtube-dl'd the whole channel those cartoons were in, for good measure.

I think it's more like 75% for HN alone.

I built a game - termsandconditions.game that got to #1 on HN a few months back and for a while I couldn't understand why I had such high CDN use and 4X less registered hits. This is why!

It would be 100% if I could do it on mobile.

Use Firefox mobile then.

Seriously, this.

There may be an argument that Chrome is marginally faster than Firefox on desktop, and while this may also technically be true on mobile, it's completely dwarfed by the fact that Chrome is being asked render 5x as much crap (with animations and sound and video and nagbars and ...) compared to Firefox mobile with uBlock Origin.

You will literally regain hours of battery life. Those Joules belong to you -- don't just shrug and hand them to Google.

> Those Joules belong to you

My Joules, my rules.

You can: use Firefox Mobile and turn on Enhanced Tracking Protection.

You can install ublock on mobile firefox too.

On Android, yes. On iOS you can’t, but at least you’ve still got ETP and Firefox Focus’s ad blocking.

You can use adblockers on ios aswell, I use adguard. It works for youtube ads which is nice.

I wish you could install umatrix, and it had a decent mobile UI. Umatrix has just transformed the way I browse the web and if anything it has educated me at the same time. I like seeing that this random site has about 2^5 different domain names contacted, ranging from ad-junk to CDNs. It tells me quite a lot about its developer. Sites with one hostname, minimally awful JS and few-to-none XHR requests get a thumbs up.

On the other hand, it probably means that I am a "false negative" in TFA's report. I'd love to know the correlation between the server logs and what Plausible shows for a connection. I'd also like to know how they infer OS -- e.g. for privacy reasons, my reported user agent is not accurate...

> I like seeing that this random site has about 2^5 different domain names contacted

You can see that in uBlock Origin already. The summary info has the number of domains connected and if you click on "more" you have the details, just like on desktop. You can also block JavaScript altogether for a particular site.

You can install noscript in Firefox mobile without problems, that's what I did

Just use the private/anonymous browsing feature - problem solved

For Android, I'm using fennec with ublock origin and it's great. Also YouTube Vanced is very useful. And Infinity for a tolerable reddit experience on mobile.

I use Blokada and it works well.

ublock origin is available on firefox mobile is you use android.

Android offers Private DNS, just point that at nextdns.io and block things even within apps that aren't browsers.

The biggest benefit is definitely the sheer speed at which news websites load on my phone now, even when I'm on a crap connection.

Another benefit: This isn't a VPN, so you can still use a VPN whilst using Private DNS.

Edit: Also... lol, I can't access TFA as plausible.io is also blocked on my network and phone. I'm going to assume that the article is actually an analytics competitor whose pitch is "GA is blocked, use us instead!".

Edit 2: https://news.ycombinator.com/from?site=plausible.io Yes, assumption was correct.

Edit 3: NextDNS stats from my home network: 11.45% of DNS requests blocked, but from my phone 23.7% of DNS requests blocked. You need this more on your mobile than in a web browser!

Thanks for sharing nextdns. I just registered and I'm very impressed.

Nextdns is giving me an ID. Why would I want to be tracked by unique ID? That's even worse than GA

You don't have to do that... NextDNS ultimately uses Cloudflare DNS under the hood, so just point yourself at 1.1.1.1 and you're done but it will block nothing.

If you want a DNS server to block things, know that the definition of what to block is subjective and some people may disagree with what you want to block.

For that reason you get an account, which allows you to have a configure, and that needs to be resolve to you... so at some level an identifier of the configuration is needed. This is the id.

On Android I use personaldnsfilter (https://f-droid.org/en/packages/dnsfilter.android/), which creates a local VPN, there are also actual VPN apps that allow blocking ad hosts, and you can also use AdGuard DNS directly.

Out of curiosity, what does this bring you over hosts file blocking, and how much does it affect battery life? I use dead-simple modifications to /etc/hosts and it seems to be remarkably good. Do you MITM yourself for further adblocking, or just use it for the modified DNS?

My phone is not rooted, and I just switched to nextdns.io on suggestion of another commenter here. I can't say about the phone battery, but I don't think it makes a big difference.

https://f-droid.org/en/packages/org.mozilla.fennec_fdroid/

Also on F-Droid:

https://libreplanet.org/wiki/Group:IceCat/icecat-help-help

https://f-droid.org/en/packages/com.stoutner.privacybrowser....

Use Brave.

so brave!

iOS has content blocking extensions.

NextDNS does comprehensive blackholing of tracking and ads, very limited battery impact.

Works on ios and android.

I think their free tier is very generous at 300k lookups/month; otherwise it's something cheap like 2€/month if you want unlimited lookups (i use this to block tracking on all my devices + home network)

> very limited battery impact.

Why would it use any extra battery? The phone has to make DNS lookups whether its with the one from nextDns or from the DHCP server.

I've had bad experiences, battery-wise, using on-device adblocking solutions (that inspect DOM). An alternative i tried was tunneling to my raspberry pi that ran pihole (dns blocker), and bouncing all my traffic off the home connection, but that was also battery intensive.

You're right, this dns solution is very unobtrusive when it comes to energy use.

Are you working for them?

When I want to "try for free" I'm assigned a unique ID which is then used in all DNS resolution requests. How is that even remotely tracking-safe?

No, not working for them.

The unique id is to retain settings across different devices.

Of course you have to trust them not to sell you out.

If on iPhone, install NextDNS and enable a blocklist which blocks GA, flip the switch, now NextDNS handles your DNS and will answer requests to GA with a "sorry cannot find this".

Use Bromite, Vivaldi or Brave on Android. They have built-in ad tracker protection. The first two even support custom filtering lists.

On iOS the options are a bit worse but Brave has some kind of adblocking that works. Safari also supports blocking lists via AdGuard but it's more limited.

Or just use Firefox. Android of course, your screwed on Apple.

AdGuard (local app) works in Safari. I don't like safari, but the AdGuard blocking does not work in FireFox! I use DDG browser mostly and it works well. I do see the occasional ad.

The whole browser situation in iOS is really user unfriendly.

Firefox is another nice alternative if you have a good Android phone I guess.

Sadly, I have a quite old Android device (but its battery life it's still good) where Chromium-based browsers are already quite slow and Firefox feels even slower and drains the battery faster (the old Fennec was worse than the new one, it hanged for 30 seconds).

Get yourself a PiHole&PiVPN or NextDNS and never look at ads on any of your devices again

Oh yeah because sending all your DNS requests to a US company that's probably under duress by the NSA to hand over logs sounds like a great idea!

Newsflash: most of your data is reaching a US based company anyway

AdAway (root) from F-Droid helps me out

The title seems to have been truncated. Presumably it meant to end with “, according to a Google Analytics competitor”

Its not that i care about google analytics per se, its that i really want to kill more aggressive ads, and the easiest option is an extension that does both.

I certainly dont really like GA, but i wouldn't take specific action to block it. Definitely not taking specific action to unblock.

Nice that Marko did a follow up by writing this blog post. A few months ago (in May), I also asked myself the question of "How plausible are our web analytics?", and I was able to see the same phenomenon [1]. Many people in my audience block client-side web trackers.

One option would now be to host a plausible proxy on-premise. But I didn't have time to try that out yet.

PS: We're paying plausible customers and our stats are publicly accessible [2].

-1: https://rugpullindex.com/blog#HowPlausibleareOurWebsiteAnaly...

-2: https://plausible.io/rugpullindex.com

The entire methodology of this 'study' is comparing analytics from Google to analytics from Plausible? Who is Plausible, you might ask? Just take a look at which blog is hosting this very study.

This is a clever ad for a google analytics competitor.

Does anyone have recommendation for static site hosting with simple server-side analytics? The only thing I can find is Netlify Analytics, but $9/mo is too much for hosting a few pages of OSS project documentations.

If there's no complex interactivity achieveable only through JS, I prefer to ship sites with plain HTML/CSS, so I don't really like to use any client-side analytics. And yes – I can live without all the fancy features to track and profile my visitors.

I don't want to host by myself either, since I enjoy using a simple git push to deploy my site.

I use AWS Cloudfront on top of my static blog to get basic analytics (and some edge caching). It is almost free.

Netlify analytics is very inaccurate to begin with-- it was showing me I was getting 20,000 visits a month on a 5 year old blog I never updated!

Moving to a normal analytics, the number is closer to a dozen, maybe a few hundred.

I suspect Netlify doesnt remove bot traffic

https://plausible.io/blog/google-analytics-alternatives

If you are willing to go a bit more hands on:

* Use a CDN which provides access logs (AWS Cloudfront, logs are stored in S3, Azure supports this too i think) * Feed those logs into a self-hosted analytics solution, like goatcounter [1]

[1] https://github.com/zgoat/goatcounter

CloudFlare Pages does this.

How about Pirsch [0]? It starts at $4/month and you can set up as many sites as you want for up to 10k page views/month. That should be sufficient for most smaller sites.

[0] https://pirsch.io/

The remaining 42% of users work in adtech and need to monitor their own product, are stuck on some platform where an adblocker is not possible, or are they just masochistic ?

A lot of people in adtech block ads.

forgot the ignorant or not-tech-savvy ?

Or they put their money where their mouth is and stop going to websites that are ad ridden to the point of unusability.

Can confirm, part of the 58% who block all the ads. Using a raspberry pi as a pi-hole, works great.

Getting 100% blockage [0] on this adblock test [1].

[0] https://i.imgur.com/7hdaHmh.png

[1] https://d3ward.github.io/toolz/adblock.html

Haven't seen an ad in months. (kidding)

Same here. I don't even open the browser when my phone is not connected to my safe heaven WiFi.

Just wireguard to your “safe haven” and you can open your browser anywhere.

Or, if you’re lazy like me, just use NextDNS when you’re outside. There’s even an option to only activate NextDNS when not connected to the Wi-Fi of your choice.

To be honest. I enjoy being semi offline when not home. But yeah, would be easy to VPN home.

> Plausible proxy runs as a first-party connection and is only blocked by those visitors who block JavaScript entirely.

Which means it still underestimates; those who selectively disable Javascript, through things like uBlock Origin's advanced mode or uMatrix, will not be counted. So the real percentage is probably higher than that 58%, and we don't know how much higher.

And 90+ % of those likely have plausible.io on their block list too. At least it is on all tracking/analytics blocklists I use.

Are there relevant studies on the effectiveness of advertising?

I remember spotting such study.

I still wonder if advertising is always worth spending on, beacuse google and facebook make a big amount of money with it, but I'm still skeptical when I see the actual amount of people "engaging" in ads, when you see how much it actually costs.

Yes, pretty much every company first data person runs Cost-per-action (CPA) analysis to estimate how many sales can be attributable to ads vs. other ways of raising your brand. It’s an imperfect science (just use the word “attribution” to trigger shivers in any data person) but it’s very clear that targeted platforms offer dramatically cheaper leads for almost every product.

That’s why the hostility to ads feels misplaced: it works, it helps company to whom you want to give money to find you and people like you. It’s just that condescension from _some_ people in Ad platform and a press that insistently favours the worst people mean that people like me who argued for more user-friendly interfaces (ban certain content, prevent repeated ads) were routinely overruled.

> it helps company to whom you want to give money to find you

You mean, the companies who want me to give them money. I'm sure there are a few companies out there whose product I actually want and don't mind paying for, but their number is absolutely dwarfed by the number of companies that seek to induce FOMO/status anxiety/etc in order to get me to buy things I don't need and would not want if not for advertising.

every advertising platform wants to pretend they are delivering useful products that people would otherwise be unaware of, but almost exclusively deliver what you have identified.

Yup. I find it amazing the quoted myth lives on. It doesn't even make sense.

How many problems do most people have that can be solved with an existing product they are unaware of? The numbers can't be very high.

How do you explain well-known brands advertising the same product for decades on end? Surely Coke and Pepsi aren't suddenly enlightening many people to the existence of their drinks.

How about ads that get shown to the known-same individual time and time again after mere minutes? (See Hulu, at least back in the day, not sure what it's like now.)

The whole line about ads being mere consumer education is ridiculous and doesn't even stand up to a cursory thought.

> That’s why the hostility to ads feels misplaced

I have no hostility to ads themselves. I have a huge amount of hostility to the spying that comes with them.

Every business with a competent marketing department will continuously track and measure the effect of the ads they run, like how many users who click the ads turn into paying users.

My startup has tried a variety of marketing strategies from in-person campaigns on the street, video ads on YouTube, "free" PR through newspapers etc. In order to measure the effect of each approach we only did one at a time.

For us paid marketing on Facebook/Instagram was, unexpectedly, the most efficient form of marketing by far. But I would not assume that applies to all, or even most, businesses. So you should experiment with different strategies for your business.

It's the tracking part here that's hard. How do you know that the FB/IG ad was the first time the converting user heard of your product, or that it was the deciding factor? If you literally have no other way of discovering your product than this works, but it's easy for FB/IG to show your ad to users who were already going to convert and claim the conversion...

I see this so many times. Someone Googles for <product name> and then clicks on the ad for said product instead of their website which is the first organic result. Google claims it’s an ad conversion and gets the money, marketing monkey will happily take this as credit for their work and justification for further ad spend & their own salary, while the truth is that this user already made their decision to use this product (as they’ve searched for it) and didn’t need the ad.

Ad platforms, like Google Search ads, will give you metrics on clicks/conversion on a keyword basis (obviously).

No marketing dept is dumb enough to equate brand-keyword traffic with organic traffic.

I dislike ads in general. Specially Youtube ads. They are hysterical and for some god knows reason advertisers think its a good idea to repeat ad nauseum the same ad multiple times even on the same video. I end up hating the brand more than having some interest in the product.

(Paid) reviews on the other hand like unboxing, configuring and testing a product that I'm interested in are totally another thing. This applies to furnitures, house appliances, computers and so on. A good example is that I did not knew how much I wanted to build a fully silent computer before watching so many build videos of a certain fanless case that looks like a metal cube.

The old saw is that 50% of advertising works, it's just that nobody knows which 50%

I assumed it was more since posts I've had on the front page have more like 70-80% of traffic missing in GA. It could be because I only enable it when DoNotTrack is off though which I don't believe is the default.

Blog post for a competing platform criticizing the effectiveness of the market leader in the space.

It's a widely known trend that more tech-savvy consumer audiences tend to employ more tracking countermeasures.

The sales message in the post claims GA is cookie-based, which isn't entirely true. Cookies are one complementary component of tracking within the Google stack, but not the only one. Depending on your account type, GA deployment, and reporting method, you can achieve reporting results that are not solely cookie-based.

The over-simplification is for sales purposes.

Really curious how this compares to Cloudflare's analytics. I run a site for developers with decent traffic, and according to Cloudflare only about 25% of our readers block Google Analytics. I had always thought it would be a higher percentage, so this seems to make sense.

It's also odd that the clicks from Google Search Console line up very closely with what we see from Google Analytics. I had always thought this data would be more accurate since Google SERPs uses (used to use?) forwarding URLs to track this stuff.

One solution is to use its own Google Tag Manager server. In this case, the events are sent to the server container via a custom URL and the data is then transmitted to Google analytics.

That's something I often think about. Are blockers used for fingerprinting? Especially by the likes of Google who controls both the ads/analytics platform and the website (aka first party).

There is a specific demographic who use blockers, generally of the tech-savvy kind. It should even be possible to detect the blocker being used and some of its settings. It would be ironic to detect that a user is blocking almost everything and show them ads for privacy-oriented products and services.

I think this is why most sites—especially those targeting technical audiences—should rely on server-side analytics instead. Add some middleware to your web framework of choice which logs request data and parse that, or use something like https://www.tabbydata.com (disclaimer: I built that) to pipe it into a data warehouse. Voila! No JS tracking, retain useful metrics.

I’ve used server log analysis, awstats, for maybe 20-25 years. It’s really interesting the difference between awstats and Google analytics (or Adobe analytics, etc).

The reason I keep using Google stats on my backend is convenience and “Google magic” for tracking session length, bounce, behavior, etc etc. I can get most of that out of awstats, but that requires more work.

> Plausible proxy runs as a first-party connection and is only blocked by those visitors who block JavaScript entirely. It is not blocked by any browser or adblocker

Would be interested to also include that into the statistics. How many nerds block javascript? I wouldn't be counted by the plausible proxy since I disable javascript per default via ublock and if I allow javascript I still filter it using uMatrix.

I'm curious as to why Reddit users were considered a tech-savvy audience. While there are some tech-centered subreddits, many others are not.

the subreddit that drove the traffic to the post that was analyzed was /r/linux which should be more technical than the average subreddit out there

Thank you, I missed that part.

>I looked at analytics of a site that had a post trending on Hacker News and Reddit with more than a thousand upvotes and more than a thousand comments.

Reddit is a very diverse site and not a tech savvy site overall. This just means the subset of users who liked this, likely very technical link, blocked GA. To use that to claim 58% of all reddit users block GA is very disingenuous imho.

doesn't ublock black GA by default? That might help explain it

I used to have Google Analytics blocked on my home networks (through Pihole/Adguard Home) with the exception of one network which my wife uses. She works with SoMe so not having access kinda interferes with her job.

I've since moved to NextDNS.io to have network agnostic ad/malware filtering. Still keep a "special" configuration for my wife :)

what is SoMe?

Social Media manager.

I was as confused when i first saw job postings for "SoMe experience", and figured it was a MeToo thing :)

Social Media

So proud of the plausible guys for getting to the top of HN. I remember reading about them in 2019 on indie hackers! All the best

For the tech-savvy audiences and blog writers: Google provides a dedicated opt-out plugin [0] for various browsers. (So it's not just browsers interfering, ad-blockers and disabled JS.)

[0] https://tools.google.com/dlpage/gaoptout

"All in all, the difference in stats would mostly come from people blocking the Google Analytics script. Google Analytics is listed on many blocklists while the Plausible proxy runs as a first-party connection and is not."

So what exactly does Plausible do to avoid being blocked in the same manner? What prevents GA from doing the same?

> So what exactly does Plausible do to avoid being blocked in the same manner?

It seems they serve plausible from the same server as the origin website (first party), probably under an unique name, which means you need to block the script manually on every domain instead of simply blacklisting `plausible.io` (for example).

> What prevents GA from doing the same?

That's a good question. Probably ease-of-use (no need to explain how to host it, easier updates) and the lack of need to do so. YouTube, for example, could also make it _far_ harder to block ads by simply sending a single video stream, but they don't for some reason.

IIRC, It is possible to serve Google Analytics script on your server if you really insisted in doing it: https://www.freecodecamp.org/news/save-your-analytics-from-c...

This is pretty much in line with what I observed a few days ago when my post[1] reached the top of the front page here.

Google Analytics reports 10k uniques vs. 35k in Netlify's server-side analytics.

[1] https://news.ycombinator.com/item?id=28288760

Google Analytics is only the tip of the iceberg; moves to pixels, premium analytics, saas tools reporting, partners, to CDPs and on and on. Found easy tool to show extent takes only few minutes to set up: https://truetraffic.io/

I have been using professionally Google Analytics for over ten years. When I set up Summon The JSON store on Shopify I made decision to not use it.

It does not give any value. And moreover it takes a lot of time to study. At the end rather on quality and sales you focus on visits, and a lot of other non important metrics.

PiHole is my preferred solution. I have come with some insights here:

https://gioorgi.com/2020/pihole-lockdown/

Give it a try!

https://pi-hole.net/

is there a way to run this pi-hole on cloud so, multiple users can use it , and also while using mobile data ? also, for those who do not want to spend on / does knot know to fiddle with RPi.

This should only be done in combination with a VPN. Open DNS points are not generally recommended

Check out nextdns instead. It can likely do what you had in mind

thanks. I will check this out.

is AdGuard also a similar approach ?

https://adguard.com/en/welcome.html

https://en.wikipedia.org/wiki/AdGuard

I know adguard has a pihole like thing called adguard home. Don't know anything about it beyond that.

Aside: I thought I heard a while back that Chrome and Safari were changing their browser extension APIs such that it would make it difficult to truly block ads / 3rd party scripts / blacklist domains. Did this ever come to fruition? Was it overblown? Never heard where that went.

Google even has official add-ons (for multiple browsers nonetheless) to opt out of Google Analytics: https://tools.google.com/dlpage/gaoptout

Their open analytics page shows the live HN traffic to their website, driven by this article - quite a peak from regular: https://plausible.io/plausible.io

Ok, so I sheepishly admit that didn't know how to block analytics. Now I do.

https://www.privacyaffairs.com/block-google-analytics/

That link is a joke: The highest ranked solution is installing a google extension.

This is a bit concerning. I would like to block ad, but would prefer to keep analytics enabled, since it helps improve the products that I'm using. I'm using Brave and I don't see any obvious way to set it up like that.

How is the percentage that low? Maybe iPhone Safari is to blame?

It's not quite as bad as things were ten, fifteen years ago, where not using an ad blocker was just begging to get your machine pwned, but it's still bad.

The thing is, they also block plausible. It's the ad blockers that block.

Why so little? Honestly, I believe the percentage is much higher, we just deliberately unblock certain sites for market research and so on. Most of the web is completely unusable w/o adblockers.

Maybe browsing from work? Some companies have policies against installing third party browser extensions.

Or using default browser on your phone.

This is incorrect. Google Analytics uses a first party cookie. Only the ability to create paid media audiences using GA is blocked by non chrome browsers and ad block extensions.

> Plausible proxy runs as a first-party connection and is only blocked by those visitors who block JavaScript entirely.

Then sorry, as a NoScript user I probably manage to escape both GA and your experiments :)

PC: SimpleWall, uBlock, Decentraleyes, Privacy Badger Android: AdAway, uBlock, Decentraleyes, Privacy Badger Home network: Pi-hole

I never asked for ads and telemetry, so there is no other option for me.

Every reputable Digital Marketing Agency now switches to Google Tag Manager Server Side (first party) tracking. Only disabling Javascript would circumvent this.

Pihole (2M domains) + Firefox strict mode + uBlock Origin is what I do. Any other tips? Used to have privacy badger too but it breaks too many sites for me.

Disable JavaScript by default (try uBlock's advanced mode).

If I do this, the internet basically becomes useless for me to be honest. Many useful sites breaks for me (shopping, banking etc)

So Linux user base is kind of almost 5% instead of <1%?

Me, I don't block Google Analytics intentionally, I run Ublock Origin to get rid of ads in general and it happens to block Google Analytics.

I wonder why Google did not yet make a tool to integrate your server logs into analytics. That way you can really measure everything.

Can some HN users that don't block ads across the board in their browser explain how you can stand using the Internet at all?

How do you deal with "please disable adblocker" messages? I found them way more annoying than the ads themselves.

There are anti anti-adblockers built into uBlock. If that does not work, just ditch the site.

Strange, but I haven't seen one lately that actually blocks you. Clicking outside of the modal tends to make them disappear.

Could be that I am visiting less annoying websites these days though...

The blockers also block most of those. When that fails I find the back button works pretty well.

If it's a popup for something I don't have a choice about using (e.g. a government site), right clicking the offending element choosing inspect and then deleting it out of the dom usually solves the problem.

Any website with a fully blocking message I disable JavaScript entirely in uBO. If the site is still unusable after that I just close the tab.

uBlock Origin eye dropper, select that element, and add it to my blocklist.

>Plausible proxy runs as a first-party connection and is only blocked by those visitors who block JavaScript entirely.

That's me.

Original title (matched that of the blog post)

"58% of Hacker News, Reddit and tech-savvy audiences block Google Analytics"

I'm surprised the number is so low. I would expect it to be way higher for this specific group.

I don't. I like personised ads. I don't want bra ada at 3 am whish I won't buy.

Now I need to teach my adblocker to block Plausible, even when hosted first-party.

hmm, so this means it's about time for the newest how to hire guideline being - come to our site, if you block Google Analytics that means that you can go to the potential interview pool.

...but the same people stuff their own products with ads and analytics.

Get me some lawyer and enforce Schrems2 against Google Analytics.

Seeing as Plausible is still a script, how much of the remaining percentage of Firefox/Linux users are blocking scripts entirely? My default uMatrix settings block everything but first party CSS and images.

Considering that most of the web is unusable for most people without javascript, I'm pretty sure no-script people are a very very small portion of the internet.

A lot works quite okay, reader view also works great a lot of the time. And if necessary I'll allow first party scripts and most tracking scripts are third-party.

That remaining 42% surprises me even more...

as i browse from duckduckgo browser

And we know it doesn’t make a difference.

Can I block this on safari on the iPhone.

Healthy people don't eat junk food.

Pi-Hole on my domestic private network.

Only 58%? I expected 95% at least.

In other news, water is wet. ;-)

Shocking it's not higher.

How do I block Plausible?

Pihole

Might as well stop the whole damn thing :). For the better internet health

* at the DNS level.

I don't.

And…

100% of Google Analytics authors are tech savvy.

This is called irony.

plausible.io is also blocked

only 58% of us? I thought that number would be much closer to 99%

Only 58% :)?

Why people block google analytics let the owner of the website track you.. it will help them to earn because they open a website to serve you not to untrack you...

Only 58%?!

The fact that it's so low (yes imo 58 is pretty low) speaks volumes to the attitudes of the "tech-savvy crowd" towards tracking and data mining.

Including Reddit's audience as "tech savvy" is an outright lie. Reddit is one of the most popular websites in the world. (And even if it weren't, it doesn't take much browsing to see how non-tech-savvy its audience is. It's basically Facebook at this point.)

While I agree with you, unfortunatley that is still technically considered the tech-savvy crowd, amongst others at least.

The audience isn't even sort of comparable to Hacker News though, so putting both in the same headline is weird. There was some correlation long ago, now not so much.

The Firefox numbers above are a much better tell of what a "largely tech savvy crowd" looks like. 88%.

Depends really on the subreddit in question. The traffic to the blog post that was analyzed came from /r/linux which should be more tech savvy than the average web user and also more tech savvy than the average Reddit user (i would put /r/linux audience close to the Hacker News audience)

Adblocking an immoral tragedy of the commons type move. Psychologically, It's not unlike NIMBYism or residents of a gated community that figured out how to avoid paying their taxes.

These poor, poor ad companies are being taken advantage of by me. I have no shame. I even immorally block ads on thousands of endpoints used by other people! Quite the tragedy!

This is a hilarious take, thanks for the laugh!

You mean because the free internet is infrastructure belonging to all and ads are its maintenance?

I fundamentally disagree with this parable. I don't think common practices in advertising are a necessity, especially when it comes to analytics.

> Adblocking an immoral tragedy of the commons type move.

Adblocking is a reasonable defense against the constant and expanding abuse (of both people and the commons) that the ad industry brought on us.

For the record, I don't engage in ad blocking as such. I block scripts, which has the side effect of blocking much of the more abusive ads. However, reasonable ads aren't blocked at all -- it's just that there aren't many reasonable ads on the internet.

We detached this subthread from https://news.ycombinator.com/item?id=28365335.

Well done lads!

99% of users ignore online adds.

Most advertising is a con, on the ad buyer.

You're saying that while commenting on a HN post from a company that sells analytics solutions, with a piece of content showing that Google Analytics it's getting blocked a lot by a specific type of a audience.

Like... hey guys, you know, if you want to sell to these folks maybe you should consider an alternative to GA, right?

Why didn't you ignore this?

This is fine for you, yet an AD that's it's clearly identified as being an AD, with, literally, a defined area - it's not.

HN users think they're too smart to be susceptible to ads, but I'm skeptical! I think ads work.

I block ads not because I'm anti-ad, but because

1) it's nearly impossible to browse the web at all without an ad blocker

and

2) Malware has been delivered via ad networks, sometimes on otherwise reliable websites that were duped into running the ad.

I'd have no problem with ads if they appeared as ads used to appear in a newspaper - static content between articles that doesn't distract or interfere with my reading experience.

This is exactly why metrics like "only 10% of our users use Firefox" are complete rubbish.

Only 10% of the users that don't block GA.

That's pretty in line with "X% of users that participated in our poll say they don't min participating in polls".

I surf in private mode. A super simple solution for all your tracking protection needs people. It boggles my mind how this comes up again and again yet we have private browsing since when? The 90s? Don’t be lazy - that’s the real reason, isn’t it? Having to login again..

Yes, because of course private mode protects you from fingerprinting tech and other tracking infrastructure

/sarcasm

No, we really did not have private browsing since the nineties. It's a new thing, for sure.

What we had in the nineties was that we could configure our browsers to ask for every single cookie that a site wanted to place on our computers. That got old really fast when internet advertising became a thing, and tracking cookies were all over the place.

> No, we really did not have private browsing since the nineties. It's a new thing, for sure.

Indeed! Seems the first browser to implement some sort of incognito/private/separated profile (specifically to hide your tracks, not general like Firefox Profiles) was Safari around the release of Mac OS X Tiger (April 29, 2005).

You do know that fingerprinting is a thing, right? Private browsing is designed to keep your local device clean of your tracks, not the wider web. At all.

Telling Facebook "hello, I am visiting baddragon.com from Mozilla Firefox 63.1 on Obscure Linux in Podunk, Saskatchewan, population 32" is barely an improvement over telling them "hello I am John Q. Smith and I am visiting baddragon.com".

And of course, private browsing is generally per-instance. Log in to your Google account in a private tab? All your other "private tabs" are also logged in to Google until you close the private browser.

I use a generic iPhone without any mods

And yeah probably need to use windows not tabs on desktop. On iPhone each tab wouldn’t see that another is logged in though

(Otherwise yes good to be aware of fingerprinting I agree with you)

Chrome's incognito mode doesn't block Google Analytics.

Reply on news.ycombinator.com