Yahoo, AOL results biased in favor of parent company Verizon’s websites | Hacker News Reader

Yahoo, AOL results biased in favor of parent company Verizon’s websites

ctrl.blog

285 pointsmikro2nd77 comments

For all the worries about tech monopolies and oligopolies, I am always more worried about the ISP's and their power.

Them and the ecosystem of data brokers and marketing companies that are invisible to consumers and yet do far, far sketchier things with personal data.

Like, remember that thing where cell carriers were literally selling real-time user-specific location data to, essentially, anybody who could pay? And probably still are? I can't conceive of how people can be so angry at "Big Tech" but not at these shadowy fuckers who are doing so much worse things wrt. your data.

Data brokers are a part of big tech as much as apple and Netflix are.

Apple has a more aggressive stance than most companies but even they are pathetically far from being privacy first. I'm a big fan of the proposals to treat all used data as radioactive waste that if mishandled, even accidentally, can destroy entire companies. It is absurd that we're even having these conversations. We're one dictator away from another genocide on the soil of a world leader. China notwithstanding. These vast databases can be misused to quickly indentured entire minority populations in countries. We've gotten a taste of this under trump with the "illegals". If we're unfortunate to find ourselves under an actual dictator trump will pale in comparison. By excising all customer data from companies we prevent countless attacks, and make many other attacks far harder.

I agree with you on the "radioactive waste" aspect, but I put "big tech" in quotes to suggest the sense in which lay people tend to interpret the phrase. I'm fine if there is an angry public response to all of it, but if we've going to be selective (only angry at the sexy companies with consumer-facing products) then IMO our selection to date has been suboptimal wrt. effective protection of consumers.

We're in agreement!

Unfortunately, I think it's going to take wholesale misuse of the data of large swathes of the population that dwarfs the reach of Equifax and the sensitivity of ashley Madison while effecting at least a couple ultra wealthy public names, to get sane consumer data protection laws with teeth and enforcement.

As Apple has prioritized privacy, I have migrated my stuff towards them. If they are pathetically far from being privacy first, I would like to know how so.

Prioritizing user convenience over user privacy is driven by a profit motive, sure. But that’s different than Google’s ecosystem which prioritizes monetization of user data over user privacy.

So, can anyone expand on where Apple sacrifices user privacy (other than for user convenience)?

> Like, remember that thing where cell carriers were literally selling real-time user-specific location data to, essentially, anybody who could pay? And probably still are?

Or before that, when they caught flak for header injection[1]. Which is still a thing according to the "Subscriber ID Headers" section of [2], just gated behind a whitelist instead of sprayed out to everyone. It'd be interesting to know if the rise in HTTPS traffic has reduced the utility of that, or if they've found a mitigation for that inconvenience.

[1] https://www.eff.org/deeplinks/2014/11/verizon-x-uidh

[2] https://docs.adobe.com/content/help/en/analytics/technotes/v...

Try searching on google for nyt retractions sometime. For that matter, twitter and facebook manipulation also can have some very direct influences... After it's all said and done, they're all a huge cesspool of deceit.

An mobile, I'm using Brave (FF just didn't work right with ublock for me), and using uBlock and Privacy badger on desktop.

I really wish browsers were far more restricted on IFrames more than 1-layer deep... cross-origin on 2+ layers of IFrames is all ads.. if the networks really cared, they'd work around it if they were cutoff from cookies/session/localstorage etc... As it is, layer of huge JS and tracking, nope, no ad, pass to another layer, etc. 85% of overhead is often in layers of ad iframes, cut them off at the knees technically... no cookies or data access 2+ layers deep.

I just checked a typical day's correction by the NYT: https://www.nytimes.com/2020/03/17/pageoneplus/corrections-m...

It's two minor mistakes, transparently corrected. I can't think of a better process except "let's agree never to make any mistakes".

And the self-certified "Brave" browser is a for-profit scam. It's an answer only if the question was "Why isn't there another middleman between me and content creators, skimming 30% of ad revenue?".

You are setting a false dichotomy. Most people that care about privacy are angry at _both_ ISP stalkers _and_ BigTech stalkers.

That being said, BigTech has developed significant stronger capabilities for aggregating and exploiting the data at scale, and using it for manipulating mass behavior. It's their business model: serve ads and curate content for maximum $$$ and engagement. For better or worse, ISPs just pass bits around on behalf of their customers, without deciding which bits to show and which to hide. Given the death of net neutrality things may change, but for now ISPs have a lot of ground to cover to become as noxious as BigTech.

> For better or worse, ISPs just pass bits around on behalf of their customers, without deciding which bits to show and which to hide

Comcast does DPI and will inject "warnings" into your port 80 traffic if they see you downloading bits they disapprove of. They've also been caught blocking sites multiple times; it is always a "mistake", but somehow also always something like this:

https://www.digitalmusicnews.com/2018/03/07/comcast-xfinity-...

You're also forgetting Verizon's magic headers, various carriers' zero rating, mining DNS requests, AT&T's Room 641A, and about 50 years of other charming behavior.

No, I'm not "setting a false dichotomy" at all, actually. I could be wrong, but I'd bet some amount of money that less than 1 in 10 Americans who have an opinion about "Big Tech" (in the sense that if you say those words to them, they will provide an opinion) have any meaningful knowledge of these less-visible players outside of the services they provide directly to consumers.

("Oh, Verizon? That's my phone company! Yeah, their customer service is pretty bad but it's cheaper than T-Mobile, so...")

That's the point of my comment. It may be a dichotomy, but from my observations of "regular people" it's certainly not a false one. IMO your viewpoint here is a very tech-centric one, and doesn't accurately represent "most people".

Yes. If there had to be a group of people to have access to so much data, I trust Googlers more than I trust my own government both in terms of ethics and definitely competence.

That said, I don't really trust them much either.

Google has existential power from their information and search. If they wanted to be a hedge fund, my god, they'd be very successful. Most elections these days are narrow, if they wanted to tilt the board in one direction, they absolutely could - which is a shocking thing to consider.

Imagine if you're French, and some massive foreign company could adjust their algorithm so that most of the news about Candidate A was good and Candidate B was bad.

Sometimes it's hard for Americans, in particular, to get the concept of such foreign control because nobody is 'big and powerful' enough to control the US.

But consider the analogy: everyone in the US uses Baidu. Everyone buys a lot from Alibaba. Most people read China Daily and watch films made and produced in China. Chinese stars are beloved and often more influential than local stars.

It's almost incomprehensible, but that's what it's like being in most countries in the world.

So you can argue about Verizon/AOL etc - but to people outside the US, it's a more existential concern.

Can someone explain to me how this is problematic? If you go to a grocery store, you will see store-brand items promoted too. They will be cheaper and possibly more prominent on the shelves than name-brand items. They might even be in disguise (e.g. Archer Farms is a Target brand).

How is this any worse or wrong?

"Wrong" or "worse" is usually measured as a product of the impact to the customer, measured in value: that is, when a potential conflict of interest results in a lower cost the consumer, it's usually deemed acceptable.

What is a similar measurement for information? Grocery stores only sell physical goods; they don't, for example, determine what medical information you see, or what information about political candidates you consume... those are things that search engines do.

So you see why this could be "worse" and "wrong": there isn't a good way to measure the impact and the scope of influence is much larger.

> What is a similar measurement for information? Grocery stores only sell physical goods; they don't, for example, determine what medical information you see, or what information about political candidates you consume... those are things that search engines do.

Okay, but that's a different discussion. Google showing Google Maps first instead of Apple Maps on a search for "maps" is not related to the kind of shadowy political subterfuge you're referencing here.

I still don't see what's wrong with highlighting your own products to your customers, which is the discussion here.

The Apple Maps service doesn’t have a web presence, so they’re not even trying to compete with Google Maps on the web. A map service is objectively more useful than a promotional page trying to sell iOS devices with a built-in Maps app.

> The Apple Maps service doesn’t have a web presence

It kind of has: https://maps.apple.com/place?address=San%20Francisco

No, it doesn’t: https://maps.apple.com/robots.txt

The robot file doesn't want you scraping actual locations.

So it does have a way of accessing the maps over the web. But because they dis-allow locations they don't compete with google for location based queries.

Everyone is wrong or right, depending on if your glass is half full or empty.

> Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith. [0]

Replace "maps" with the names of any competing products from both companies with a web presence that can be linked to in search results, if you want to genuinely rebut the point.

[0] https://news.ycombinator.com/newsguidelines.html

If they are promoted results, they should be marked as such.

When performing a search you are implicitly asking to see the "best" (most on-topic) results according to an impartial computer algorithm.

That's the user expectation.

A corporation inserting its own pages into organic results means it's not a search engine anymore. It's "a paid directory service augmented by a search engine".

> "best" (most on-topic) results according to an impartial computer algorithm.

Any algorithm looking for the "best" result is inherently partial to something. Best result in terms of most recent, most local to you, most-likely-what-you-actually-want, I mean, the dimensions are endless.

The concept of "organic search results" is only possible if the algorithm were public, and that's just not a realistic expectation.

>A corporation inserting its own pages into organic results means it's not a search engine anymore. It's "a paid directory service augmented by a search engine".

I mean, Google doesn't have to pay themselves to promote their own pages. If you want to split the difference at "a search engine, part of a larger business whose results might show up preferentially in that search because why wouldn't they," that makes sense to me.

I thought it would have been obvious, but to clarify what I meant by impartial and organic — I mean "ranked using the same criteria regardless of origin".

Of course any human-written algorithm will have bias. But it's a reasonable expectation (and the current norm) that every page faces the same bias.

I'm not sure where you're going with your last comment. If you've got evidence that Google is manually promoting its own pages within the search results (not ads) then please provide sources. This has never been proven and Google have consistently maintained they don't do it.

(I'm not a Google fan but I believe them in this case because of the anti-trust implications)

Google has been doing this since the beginning. They maintain they don't manually change the rankings.

Google originally picked a bunch of sites and included there own and gave them a 10 page rank. That meant they automatically ranked higher for the same words/term because of a higher reputation score.

I'm not sure exactly what you're alleging. The discussion was about whether Google manually moves their own properties up in the SERPS but you seem to be speaking of how their PageRank algorithm was seeded some 20+ years ago.

They’re often cheaper because the store increases the price of the competitors’ wares to make their own store-product more appealing. It increases the price to consumers.

Can’t both practices be wrong?

Grocery stores shouldn’t be allowed to sell their own brands that compete with the otherwise free market. It’s an abuse of their position in the value chain. Heinz Ketchup can’t compete with the store brand on equal terms when the store sets the prices for both products.

> Can’t both practices be wrong? and an abuse of their market position?

Sure. But my point is that no one is railing Kroger for putting their cheap pasta above the DeCecco. So it's hypocritical to call out tech companies for doing what physical retailers have been doing for ages.

If we're not mad about the pasta, we shouldn't be mad about Flickr being #2 on the search for "photos". And if we are mad about Google Photos being #1, why aren't we mad about the pasta?

I’m mad about the pasta too. Price gouging is usually regulated internationally, though. E.g. groceries in India are labeled with a maximum retail price set by the manufacturer to avoid problems like this.

Your analogy is flawed.

Google's value proposition is to provide the most useful/accurate results for your query. Period. That's Google search's "raison d'etre".

Kroger makes no such (equivalent) claim.

It's a free market, shop someplace else.

Some of those shops use price dumping to kill all local competition when they move in. Try again.

> They’re often cheaper because the store increases the price of the competitors’ wares to make their own store-product more appealing

Maybe irrelevant to the point, but in my experience that is very wrong. They're cheaper because:

* they have approximately zero marketing costs other than the cost of the shelf space, which they have better insight to sizing than their competitors

* They didn't have to spend as much testing and refining to develop a quality product, because they already have a good sense of what the market is after because they're copying something people want

* They don't have to maintain a staff of salespeople working to negotiate their products into stores. They know exactly where their product is destined to go, and how much to produce to satisfy market demands.

* Because they have split-tested the price against which users see their value brand as enough of a bargain to switch from Fruit Loops to Fruity-Ohs or whatever.

Yes, there is a portion of that component that has to do with the margins applied against the name brands, but stores aren't marking up their competitors' prices beyond the normal markup, and just as an aside, if Fruity-Ohs were selling better at a price than Fruit Loops, the store would increase their pricing. TLDR, if the market agreed, it would be very possible for the generic brands to cost more than the name brands, but due to point 1 (marketing) that is extremely unlikely to happen.

Search engines aren't selling peanuts clearly marked as store-brand; they're showing information most users presume is in descending order of relevance.

Both are wrong. It gets more and more wrong as the entry barrier for a new player becomes bigger and bigger. So Google promoting their content is FAR more detrimental than Yahoo promoting Verizon content, because Google drives most of the traffic. One solution is to force the producer of any service to always be decoupled from the distributor or marketplace. So Amazon cannot promote Amazon Basics stuff knowingly, Target cannot promote its own brand, and Google cannot promote its own sites.

Why are both wrong? This is a question of ethics, and is therefore subjective. It depends on your personal values. "Free markets are good" is one of my values, so both are wrong through that lens. If you do not believe that enabling competition is good, then it is not wrong.

I'm quite sure I see Google products and services on top of Google search.

And Google gets scrutinized for that. Now it’s Verizon’s turn. It’s good to know what and when these companies are doing.

But those are easily recognised as Google properties, allowing you to easily scroll past if you want to avoid them.

That's vastly different from selling "organic" rankings, or boosting your own products.

As an analogy: Google's approach is like The New York Times printing full-page ads for "NYT Caribbean Cruises" or whatever their side businesses are, while what Verizon is doing is as if The Washington Post ran "12 Great Products Now on Amazon" articles indistinguishable from their articles.

To give an idea of how strict the latter would break with tradition: I once designed a full-page ad in the New York Times for a non-profit event. They made us change the design because ads are not allowed to use a 4(?)-column layout. Their own design is also 4-column and they want ads to stand out. The same applies to fonts, colours, etc.

> Verizon Media-owned search engines have decreased from a market share of 4,10 % in February 2019 to 3,63 % in the United States in February 2020, according to StatCounter Global Stats. 3,63 % of the internet-connected population of the US is roughly 10,6 million people.

It's pretty safe to say the only people using those search engines are people who really like aol or yahoo properties for some reason.

3% of 300 Mio is 1 Mio, not 10 Mio ....

3% of 300 million = 9 million.

That said, the latest figures I have seen predict the US internet connected population to be 293 million [1]. That figure comes out to 10.63%.

[1]: https://www.statista.com/statistics/325645/usa-number-of-int...

> It's pretty safe to say the only people using those search engines are people who really like aol or yahoo properties for some reason.

People who use Yahoo/ AOL are people who use those sites out of habit and are happier with the crap experience they get there than learning new technologies. My mom and one of my former bosses were tied to Yahoo & AOL respectively and it was 100% just the comfort of the devil they knew.

My grandmother (before she passed) used Yahoo as her homepage and main search from there... It's what I originally put on her computer long before Yahoo got bad, and it's what she stuck with as a result. It was actually painful to see...

That search traffic is probably largely DNS lookup failure spam searches.

It is truly terrifying that people still use Yahoo and AOL

I logged into my old yahoo mail account the other day and was surprised to find it had some interesting upgrades, such as views for emails containing photos and receipts. It was a fun way to time travel.

In fact, except for (or maybe due to) the entirely blank pane on the right that presumably has a bunch of ads blocked, looks cleaner and more appealing than what gmail has become.

Yahoo's email client is one of the best in terms of usability.

The story there is that Marissa Mayer wanted employees to use it internally, so it got heavily dogfooded and improved. (Yahoo employees were using Google docs and sheets for company use like password lists. lol.)

It's worked as your described above for a few years, but they claim to have made speed improvements recently (as it was unusable on low-end devices like netbooks.)

The problem with Yahoo Mail is that Mayer allowed a FISA order to install a kernel-level sniffing module on email servers, and I never heard if that was removed (interpret that as a no.) In addition, their network was pwned multiple times, so there's that too.

https://www.engadget.com/2016/10/08/reuters-yahoo-email-scan...

https://www.theregister.co.uk/2017/02/16/yahoo_forged_cookie...

Source: ex-Yahoo, offered role as technical lead of Mail.

Are these results surprising? Probably not. But does that mean we shouldn't talk about them? Because that's what (a portion of) this comment section seems to think.

Is the story that this is happening or that people are honestly surprised this is happening.

It's not happening at that other, reputable search engine.

Of course if you take pride in cynic nihilism, nothing bad will surprise you. But lowering your standards very quickly becomes a self-fulfilling prophecy.

You'll get your entirely corrupt politicians, for example. Because the honest ones leave after a few years of being constantly called "corrupt" or "stupid" with zero connection between what they are actually doing and the feedback they get.

If I have a search engine and I create some other services that I believe are better than competitors' services, shouldn't I be putting my other services at the top of search results?

That'd be a problem if anybody ever used them

This "study" for sure doesn t prove anything. And fore sure not a bias. How do you know that the other search engine are not bias either?

NOOOOO How could this be?

I’m shocked, shocked, to see favoritism going on in this establishment /s

I think the real news here is that people still use these search engines. :D

"OneSearch" is completely new to me. I know Yahoo has a decent market in Asia, Japan I believe. I'm always surprised to see AOL mentioned in any context outside of history anymore.

Note that Yahoo! Japan is separate from Yahoo. When Verizon bough Yahoo, certain patents, holdings and investments were not part of the sale and were spun off into a new business called Altaba.

The (minority) stake in Yahoo! Japan that was owned by Yahoo is now owned by Altaba. Probably the only connection between Yahoo and Yahoo! Japan is the name, which Yahoo! Japan license from Verison.

source: https://www.reuters.com/article/us-yahoo-m-a-verizon-idUSKCN...

Such an uninspired name.

It’s an old codename for Yahoo! Search’s in-house search engine. Back in the day when they still had one.

I see those on computers run buy laymen who somehow installed one of those nasty toolbars that change the default search to Yahoo or other stuff.

Lots of people tricked through browser toolbars and self-installing "configuration CDs" i guess

Some of their success can be explained by the tyranny of the default. Some browsers use these search engine by default in some regions, and people don’t tend to change the defaults. I don’t know for sure, but I’d be surprised if Verizon don’t set it as the default on at least some of the devices they sell. Didn’t they recently announce Yahoo! Mobile as a cellphone service too?

Surprised to see they're launching Yahoo Mobile in the west but there actually already is a ymobile in Japan that I personally use for data/calling. I use my own phone with it but I can't imagine them being able to set yahoo as default on any iphone or android phone they sell

And how is this different than Google?

Could it be that Google is biased? Why everything is biased when compared to Google?

They weren't just being compared to Google. They were mostly being compared to Bing. Remember, the underlying data provider for Verizon's search results is Bing. When compared to other Bing-powered search engines, the ordering of results should be nearly identical.

There's no universal guidelines for (document|query) rankings so the "bias" is always going to be very hard to find a smoking gun for. The appearance of bias in the rankings is a real issue that companies (and regulators?) should care about.

Putting this out there in case folks aren't aware:

DuckDuckGo is powered almost entirely by Yahoo!. In turn, Yahoo! search is powered almost entirely by Bing.

No we're not.

even on bing ?

epi0Bauqu is the founder of DuckDuckGo, he will know.

Ha, this is why I love HN.

Thanks for the awesomeness of DDG, and if you ever need help with your mobile site, let me know. claire at theoic dot me.

I love what you guys do, but your mobile UX leaves a ton to be desired and even some very basic tweaks would go a long way to significant improvement. :)

On average what percent do most search results served come from bing vs your ddg crawls or any of the other 400 sources?

I thought DDG was powered by Bing, like Yahoo. Not DDG by Yahoo by Bing. Color me surprised. Any evidence for this claim?

This is incorrect. Per https://help.duckduckgo.com/duckduckgo-help-pages/results/so...:

> ...DuckDuckGo gets its results from over four hundred sources. These include hundreds of vertical sources delivering niche Instant Answers, DuckDuckBot (our crawler) and crowd-sourced sites (like Wikipedia, stored in our answer indexes). We also of course have more traditional links in the search results, which we also source from multiple partners, though most commonly from Bing (and none from Google).

Reply on news.ycombinator.com