Hacker News Reader

Comment by cletus | Hacker News Reader

So a common trope in the startup world is the difference between a feature and a company, the idea being that lots of features are masquerading as companies. This inevitably leads to them being shocked--SHOCKED--when some bigger player adds that feature, eating their lunch.

I feel the same way about a lot of outbound sites on Google. There are a bunch of things I just don't want to go to another site for. Off the top of my head:

- Exchange rates. Although this one I find infuriating because Google doesn't know how to correctly round off exchange rates (as in, there's a standard). This manifests as, say, showing 2 decimal places for AUD/USD when they should be showing 4.

- Calculator

- Mortgage calculator

- Song lyrics

That sort of thing. If you go to any sites that provide these sorts of things they're typically "scummy". Lots of ads, lots of Javascript, lots of dark patterns to make you load more page views (eg a mortgage calculator that'll mysteriously take 3 steps/page loads to calculate).

I'm glad these are in search results, typically in significantly better versions. And I don't think anyone who runs a site built around a basic formula for interest calculations has any right to complain about it.

Of course this will be painted as "where does it end?" but not every surface is a slippery slope.

Just look at the likes of Yelp who complains about Google "stealing" their content. Well, Yelp is about one of the scummiest businesses out there. So I won't feel sorry for them, not now, not ever.

The one weird case here is AMP. Like I get Google's motivations here. Many companies develop terrible mobile sites that run badly or not at all and AMP IS much faster, generally speaking. Yet still it seems so heavyhanded with seemingly no opt out (on the consumer or publisher side). I don't really understand why Google wants to die on this particular hill.

View on HN

I understand your stand, but what Google did with its featured snippets is essentially a form of bait and switch. It encouraged entrepreneurs and creators to create content that could answer specific queries with the promise that Google will drive traffic to their sites.

Now Google is using the same content and depriving them of the traffic.

That's pretty scummy in my opinion.

Hm, I think there's scumminess on both sides, since (as GP noted) even simple sites (e.g. for lyrics) are bloated and unusable.

Before:

User: Hey Google, what's the monthly payment on a 20-year mortgage with 3.9% APR?

Google: Oh, MortgateSite would know, go there.

MortgageSite: Here's your calculator.

User: Okay, let me put in the numbers ... awesome, thanks! Oh, neat, they can hook me up with lenders. Let me take a look.

MortgageSite: Thanks for the referral, Google!

Everyone wins.

Today, it's more like:

User: Hey Google, what's the monthly payment on a 20-year mortgage with 3.9% APR?

Google: Oh, that would be this much: ... . Here are some sites where you can dig deeper.

User: Um, okay, might be worth a look. Let's try MortgageSite.

MortgageSite: uhhhhhhh hold on a second. Hey, see our BUY THIS PRODUCT mortgage calculator. AND THIS ONE

User: Uh, okay, I'll just type in--

MortgageSite: HEY! It looks like you're new to this site. Want to get on our mailing list?

User: You know what? Screw it.

MortgageSite: Confound you, Google, for stealing our traffic!

Perfect world:

User: Hey Gateway, what's the monthly payment on a 20-year mortgage with 3.9 APR?

Gateway: Here's your calculator, already prefilled with the data taken from your query.

Done.

The problem with all these little sites is, besides bloat, that the "Oh, neat, they can hook me up with lenders. Let me take a look." ends up with user getting malware and/or scammed. The problem with Google is that they can hardly be trusted at this point to do things like this in the interest of users. They want to be the frontend through which you access the Internet.

I used the word "Gateway" in my example as a placeholder; my imaginary perfect world recognizes that things like "currency conversion", "song lyrics" or "mortgage calculator" are data[0], which should be separate from the frontend used to access it. I dream of the Internet where things like these are API-driven and do not involve loading anything other than what's requested - neither ads, nor "value-adds", and definitely not all the rest of the webpage surrounding a mortgage calculator, which is bloat obstructing requested functionality.

[0] - yes, even mortgage calculator; it's a mathematical model, an algorithm, and code is data.

https://www.wolframalpha.com/input/?i=what%27s+the+monthly+p...

You're dreaming of what wolfram alpha is trying to build.

Edit: funnily enough, they parsed the input slightly wrong but you can quickly get to your answer.

works perfectly if you put a percent sign after 3.9

User (spoken): Hey Google, what's the monthly payment on a 20-year mortgage of $500k with 3.9% APR?

Google Assistant (aloud): $3,004

Seems pretty straightforward, no need for a web page.

Actual current results: "Sorry, I don't know how to help with that yet"

It's not even malware/scams - it's just "Person who wants a mortgage calculator" is "person who is interested in getting a mortgage", which is some seriously high-value data. There's a lot more profit in selling you on to mortgage lenders (with a lot of data, like your budget, pre-filled) than selling you to scammers.

And surprise surprise... That's what DoubleClick (= Google) does, they'll sell your cookie ID as "interested in mortgage" audience to advertisers.

While in the interest of "privacy" Google limits the amount of data the banks and others you interact with get from any competing solution, they will sell all of it in Ads Data Hub which is a Google Cloud product. So not so much in the interest of privacy but much more in the interest of Google selling more cloud products.

Perfect world:

User: I can afford a livable four walls and a roof from a small amount of savings, without needing to mortgage decades of my future. Brilliant.

Are you arguing that everyone should be able to buy a house, that they can live in forever, with cash they have with little savings?

The problem with that is that if a poor person can afford a house with cash, then a slightly richer person would just go, "wow, I can buy a really big house! Or two!", and suddenly there is no more space.

That would be nice, wouldn't it?

Maybe they wouldn't be allowed to buy two, because in "a perfect world", a place to live is a necessity, not a profit vector. Keeping housing supply limited to boost house prices, to appeal to house-owning voters and wealthy landlords, to extract six figures of money from normal people over decades, is nothing like "perfect".

Some houses are still going to be more desirable than others... how do we determine who gets which?

This sounds a lot more like inviting a trolling argument than any kind of genuine inquiry.

Here I say "landlord epxloitation, profiteering from necessities, houses left unlived in as investment vehicles for the international super rich, NIMBYism and entrenched residents voting against housing stock, AirBNB, broken zoning incentives" and then you say "FREE MARKET it's rich people's right to buy up everything and if you disagree you're dumb, free market is best market". And then we disagree to agree and both leave unhappy. Is that an approximately good summary?

We don't need to determine who gets which for this comment chain, we only need to question whether a 30 year mortgage should be the default way for a normal person to keep the rain off their bed, someone who isn't thinking of a beach condo in Malibu or a NYC penthouse, whether that is the best possible "perfect" world. I think it isn't.

And yet if you type in a mortgage-related query, the first few entries Google will show you are ads. Not the calculator you're looking for.

Even then, mortgages aren't something Google can realistically calculate because lenders don't structure loans that simply. There is a fixed rate period, and a variable period after that.

First-time searchers may get suckered in by a personal finance site that relies on ads just as Google itself does, but after that, they'll be using the calculators that are on lenders' own sites when it's time to comparison shop.

The ads in search results are a utterly, completely different character to the on-site garbage parent is talking about. They are simple text entries that are clearly marked.

The majority of US home mortgages are 30 year fixed rate with no variable period.

The majority of UK home mortgages are 25 year with fixed rate for 2-4 years followed by variable rate(which if course no one ever goes on, you simply remortgage as you're about to go on the variable rate instead).

Canada is the same. Nearly impossible to find a mortgage where the fixed term is longer than 5 years.

Funny thing that usually "BUY THIS PRODUCT" is also a Google Product (Ad)

>MortgageSite: HEY! It looks like you're new to this site. Want to get on our mailing list?

Dear every web developer in the world who makes no effort to fight your employers' marketing department on this: I hate you.

sometimes I try to mess with sites that ask for an email address for a mailing list by entering their own addresses: like "support@domain.com", "webmaster@domain.com", etc.

let them deal with their own spam, then they'll understand.

That's a great idea. I'll try that next time. My version of petty revenge is to sign up with one of my throwaway Gmail accounts, and then mark them as spam as soon as the first newsletter rolls in. Hopefully, there's others doing the same and they'll be at least partially blacklisted.

My go to has been "ceo@example.com" for years.

Or various Five Eyes addresses.

>My go to has been "ceo@example.com" for years.

I think I may make the effort to learn how to make a Firefox extension to automate this. Scrape "about us", "contact", etc. page in background for first email address, then autofill it into the email spam form. Boom, done.

This is great! I usually just do something like bs@fake.com.

I like you.

> employers' marketing department

In some places this a growth hacking/marketing tactic (that works), in others it isn't the marketing team but someone up high demanding it because they've seen it. Am still a marketer, but I've been in the latter and trying to fight the good fight.

I'm not related in any way to that scene, but what would you tell to marketing when they tell you that that while the mailing list pupup is obnoxious and annoying, it results in X users enrolling and later on Y of them actually ends being customers?

There does seem to be a sizeable number of people who take any request on a webpage as a command, and comply with the email popup. Which would then generate leads.

But, this popup harassment is just going to turn away experienced users. Many of whom will be the educated, well-compensated demographic the site most wants to cultivate. And a subset of that group will have the ability to create or edit blockers, which will then be turned against the offending site. Thus making it more difficult for that company to reach the more desirable users.

That's what I would tell marketing. That wouldn't work for all companies. Some will be thrilled to get a boatload of inexperienced or naive users. But would be nice if the others would start to get worried about what they're losing with their bounce rate.

Are you asking for a hypothetical scenario where I was the web developer in question having to stand up to some marketing drone?

Take a closer look at the key phrase in my post: "makes no effort". If a web developer who's trying to pay the rent makes even a single comment to the marketing drones about how email signup forms might not be a good idea, and asks if they're sure they want to do it, then that web developer has my sympathy and not my hatred. But if they just cheerfully say "Yes sir!" when the marketing drones make that demand, then I hate them just as much as I hate the marketing drones, because at that point they are marketing drones.

I was asking because I have been thinking if there are any convincing arguments or incentives for avoiding such tactics, besides moral and integrity. I'm afraid that if's all depend on moral and integrity it is unlikely we are going to win this battle.

Morals and integrity are foreign concepts to marketers. I used to think that marketing had a place, if properly used. After working with marketers for a decade, I just consider them and their whole wretched field a lost cause. I have no sympathy, no empathy, no compassion for any of them. Nowadays I make every effort to use only paid services whenever possible. (If any Fastmail devs are reading... I frickin' love you guys.)

This is great but it ain't gonna fix my internet

You forgot:

MortgageSite: Please wait while we take 10 seconds to calculate an interest rate AND CHECK OUT THIS SPONSOR!

MortgageSite: Five more seconds. YOUR COMPUTER MIGHT HAVE A VIRUS!! DOWNLOAD THIS FREE SCANNER NOW!

MortgageSite: Your rate has been calculated! Click HERE to reveal it using a totally pointless JavaScript animation!

User: Finally, now I’ll just move my cursor up and close this tab.

MortgageSite: OMFG YOURE GOING AWAY! Please come back and click on me more!!!

also, "your rate is ready! just enter your email address and we will send it to you!"

Your second part is wrong. According to the data most users don't dig deeper, they just stop at the answer Google already gave. MortgageSite doesn't even get the traffic in the first place.

I was using that example to establish that the sites (that are losing the clicks) are just as scummy as (if not more than) Google for "stealing" the clicks. That, in turn, helps explain why users might be reluctant to dig deeper, showing that general scummy practices are at least as much to blame as Google "stealing" their content.

Those sites or sites like them made the content initially that google now steals.

I remember a world without lyric sites. These sites provide value. The top links google shows may not because of page layout or bloated sizes but those are the links google presents there are lyric sites with no javascript. Meanwhile tons of mortgage related calculator pages exist on most bank sites and other non-spamming pages.

Clicking a google search result takes seconds to load the link while clicking a ddg link feels much faster. The additional tracking and redirections start to make things feel sluggish.

It's a chicken and egg problem. Website owners are forced to monetize more aggressively as the traffic from search slows down. And as they monetize more aggressively, the user experience suffers, causing them to tank in the SERPs, fueling even more desperation.

Google could fix that if they wanted to.

I remember when Google used to push content developers to be better: no paywalls, no overlays, faster response times, no scammy ads, no content farms.

Those days seem to be over. Google is constantly sending me to content behind paywalls and under pop up content.

I agree that paywalls seem more common. But the early web had plenty of popups and scummy ads. Popup blockers were added to browsers quite a long time ago.

Content farms were huge before Google started kicking them out in 2011. Have they made a comeback?

Google's latest attempt to keep the crap under control (AMP) is very unpopular on Hacker News, but I guess it shows they are trying to do something?

> Content farms were huge before Google started kicking them out in 2011. Have they made a comeback?

Have they not? Content marketing seems as alive as ever, if not more.

> Google's latest attempt to keep the crap under control (AMP) is very unpopular on Hacker News, but I guess it shows they are trying to do something?

Because we - myself and many of those other HNers disliking it - believe that with AMP, making web more performant and less crappy is only an excuse, and one that doesn't stand up to scrutiny.

It's hard to tell since there is little in the news about them, but it seems like once, content farms were fast-growing businesses, and now they are bottom feeders? Or maybe they're just boring compared to all the other threats nowadays?

So you blame Google for paywalls and full screen ads on the pages... you demand they send you to in lieu of just hosting some content themselves? Seems like Google just can't win in that scenario.

FWIW: I agree that for a long time, the need to rank highly in Google's searches pushed content sites to be a better experience. And the equilibrium we've reached isn't as nice as it was 10 years ago. I just don't think "because Google" really captures the complexity here.

Google's whole value proposition is sending me to the best place on the web for a given query -- so yes, when they don't do that it's frustrating.

No, I don't blame them for hosting the content themselves for some queries. I never said that.

Just as Amazon holds some responsibility when I purchase a counterfeit item on their platform, it behooves both companies to have higher quality standards -- and both have the market strength to do it.

>Now Google is using the same content and depriving them of the traffic.

I found some instructions[1] that says a website can opt out of Google's Featured Snippets with special HTML code:

  <meta name=”googlebot” content=”nosnippet”>

Are there any cases of websites deliberately opting out of snippets and therefore seeing their referral links (and ad revenue) increase?

[1] https://searchenginewatch.com/2019/03/27/google-featured-sni...

It's hard for me to shake the vague feeling that this feels like someone setting up on the sidewalk in front of your property and saying hey, you could have done X, Y, or Z, to stop me from being here. And you say, I did do X. And they say, oh, X was two years ago. Y was last year. Now you have to do Z. Until next year when it'll be AA. And weirdly, AA is the exact opposite of X, because what you thought would take care of things two years ago now penalizes you.

It's not a perfect analogy, because obviously people still hope for referrals from Google. But this idea that because something is published somewhere, it's okay, reminds me of Arthur Dent finding out that the notification about demolishing his house was in the third-subbasement of a local government building in a file cabinet marked "Beware of the Leopard." From rough memory; please forgive if I mis-remembered.

Do I get to bill google for all extra traffic I have to send by appending that to the body of my websites in order to prevent them from stealing my intellectual property?

Also going forward you have to add this to your site or I will steal anything that isn't nailed down.

I deliberately opted out of snippets for TLD List [0], a price comparison site I made for domain names.

I don't know if it's made any difference. Organic search traffic from them has slowly but steadily increased over the years.

Google now sometimes displays a snippet from my competitor's website for searches like "cheapest .io domain" [1]. The snippet seems pretty useless as it doesn't include any registrars' names/links (and my competitor's price info is quite outdated).

In these cases, since the snippet is the 1st thing users see in SERP, and doesn't provide enough info to fully answer the question, I'd wager that my competitor is ultimately receiving the majority of clicks from these snippets.

[0] https://tld-list.com

[1] https://i.imgur.com/aFoZbFw.png

>I deliberately opted out of snippets

Appreciate your datapoint. Also btw, when I "view source" the HTML of tld-list.com, I notice it has "nosnippet" inside a comment:

    <!--
    <meta name="googlebot" content="nosnippet">
    -->

Does google crawler parse and obey "nosnippet" embedded in HTML comments?

No I don't think so. Thanks for pointing that out.

IIRC, I eventually removed nosnippet because it caused google to not display microdata in SERP (see the "$25.99 to $99.80" in the above screenshot) that were desirable for my traffic. I instead replaced nosnippet with:

``` <meta name="robots" content="noarchive"> ```

And this seemed to have the same effect as nosnippet, but with the added benefit of my microdata still being displayed in SERP.

Thus allowing your competitor to take your place

wanna take bets google then deranks you heavily?

I prefer Googles snippet to visiting a shitty site. This is one of the few instances that I fully support Google. If websites want to stay relevant, they need to be more inventive.

Also the data they are serving is often incorrect or outdated. I've grown accustom to not trusting these info boxes.

Yep. And what's worse is that sometimes the site is correct and/or up to date and the Google snippet attributed to that site isn't or is poorly chosen for the question.

Google "chancellor" from the UK and it'll suggest the question "who is the UK chancellor now" which expands to name the previous chancellor (an error Google attributes to gov.uk which would have been updated in July with the new appointment). Click on the link to Google the question and this time the snippet attributed to the same source will be up to date and give you the correct answer. But if I still need to load a new page to get the right answer I'd preferred to go direct to the source page without it serving me wrong answers wrongly attributed first.

And the other questions are also a rabbithole of rubbish to fall down (the question "who was the previous chancellor?" answered by "previous chancellors have opted for whisky..." is my favourite combination) or excerpt the wrong part of the page and then encourage you to search Google again to get the same unhelpful excerpt again rather than deign to look at the source period.

Getting this stuff right across millions of queries and unstructured data of mixed quality is undoubtedly an incredibly hard technical problem but I think it's probably better to sometimes let users leave Google properties than feed them inaccurate answers inaccurately attributed...

I don't trust them either. They seem to pull info out of random places, and you can't easily tell how up-to-date the content is.

They also crop the content so even when it is correct you can't see all the steps.

Not from a user search perspective. That's an awesome search feature. And for all the website that aren't trying to monetize, like my personal blog, or wikipedia, it also isn't a problem, and a great feature to have, which might be argued serves the greater good and improves the internet by making finding free flow of information easier.

I think we need to go back a little and think. When the internet started, if you wanted to monetize your website, and attract users, what would you do? There were no search engines. Nowadays, people claim they're at the mercy of Google, but it's more that their very existence and business model was enabled and made viable by Google. Since it seems, without Google, no one would find them and visit their website. In that sense, I find it hard to say Google is taking things away, it seems more like Google is giving and giving, sometimes, it gives less, and sometimes it gives more.

Can a web site simply stop providing the markup for featured snippets?

Are you arguing that Google violates copyright?

Facebook did this with pages

A bait and switch would imply more of an implicit contract than there ever was along the way.

In the 90s, people were making websites because they wanted to make websites, and some of them were informational, and some of them were just whatever cool thing somebody decided to publish. Keeping track of your favorite sites was harder, but eventually browsers added bookmarks and it became a bit easier.

Okay, now how do you find all of that stuff? Three answers kind of sprung up: sites that are list of sites, directories (Yahoo) and search engines. They kind of work, but finding good stuff is still hard, and while the best answer here is probably search, the indexes aren’t entirely comprehensive and the algorithms aren’t good at sorting good stuff from bad stuff yet, they’re too easily gamed.

Google came along and went with search, but they did two things differently: they built a comprehensive index and they had a good algorithm for sorting good stuff from bad stuff. This didn’t stop an entire industry from springing up to try and game it, but they made gaming their algorithm an increasingly expensive proposition, and just doing the right thing easier.

So now we have a good search engine with a comprehensive index, now the web has a different problem: as much stuff as there is, it turns out there’s not enough stuff about a lot of stuff, and this is obvious once you have a good enough search engine that will till you whether there is enough stuff about your term. Well as it turns out, it is already 2004 and people are just now figuring out how to make lots and lots of money with this internet and web stuff. Google has come out ahead of the pact as the obviously superior choice for searching all this stuff, their competitors get a lot better, but Google stays ahead of them, and they’re able to maintain a lead by simply being so much better there is no reason to stop using Google so long as they’re not evil.

They also figured something else out: people are looking for information, or the tools for getting that information. Who knows more about getting information than peoples whose jobs it is to find and surface the right information? And it turns out that some things you can’t find with just a webpage, you need other kinds of tools, like calculators. So they built them in, because at the end of the day, it’s the information that people actually want, they don’t need the debris that comes along with loading a full webpage.

tl;dr Google is an information services company and always has been. Characterizing them as a search engine or advertising company is a massive understatement and was always inaccurate.

I miss having a good index site. Yahoo's gone, DMOZ shut down. We need a good human-curated index that's not bound to ad spend or SEO nonsense. I don't want to know about "long-tail keywords", I want to know if the site has good content that serves eithet a need or an interest. Now, if it so happens that websites with valuable content also have SEO valuable, that means the search engine algorithm is optimized correctly. But we need humans.

A bunch of those things aren't really searches though, are they?

I do queries like "2 tbsp in tsp" or "45 * 22" or "$25 CDN in USD" all the time and I don't think of them as a search.

Same goes for my Echo. I ask it all the time for today's weather or what time the Texas Rangers are playing or who is the starting pitcher for the Giants tonight. None of that feels like a search.

Well, simple math isn't, but think about your currency conversion, your weather lookup, or your sports schedule. Where did Google get the data for those things? Does whoever provided that data have expenses, and did Google compensate them in any way for those expenses or did Google lift the data for free with GoogleBot?

I'm pretty sure they have partnerships with the major stock and currency exchanges, pro sports leagues, and weather.com to get that data.

EDIT: added some sources (sorry, that's the best I could dig up in 5 minutes)

[1] https://www.google.com/googlefinance/disclaimer/

[2] https://searchengineland.com/google-now-with-real-time-nhl-h...

[3] https://bits.blogs.nytimes.com/2015/03/31/ibm-scores-a-weath...

US weather data is available for free from NOAA as long as the API isn't abused. Everybody else is just reselling the same data anyway.

A good comparison here is WolframAlpha, which can also answer all those queries (I was a bit surprised that the Texas Rangers example worked, but it does [1]). Most of their data comes from a curated list of primary sources, not Wikipedia or random websites.

1: https://www.wolframalpha.com/input/?i=texas+rangers+next+gam...

Curated might be better, Google has been wrong in their answers in the past.

Not only are the sources of WolframAlpha curated, they even go the extra step to make sure it's correct. From their FAQ:

"How is Wolfram|Alpha's data checked?

We use a portfolio of automated and manual methods, including statistics, visualization, source cross-checking and expert review. With trillions of pieces of data, it's inevitable that there are still errors out there.

[...]

How is real-time data curated?

Wolfram|Alpha effectively checks real-time data (such as weather, earthquakes, market prices, etc.) against built-in criteria and models. If an unexpected deviation is found, Wolfram|Alpha will normally indicate it, for example by showing lines as dashed."

https://www.wolframalpha.com/faqs/#data-in-wolfram-alpha

I think the best one I've seen was when Google's natural language processing mistook the subject of a sentence. Rather than saying that the Mariner 10 space probe used a gravity assist on Venus to reach Mercury, it instead said that Mercury received a gravity assist from Venus.

Or maybe Google was getting its astronomy from a site that believes Immanuel Velikovsky's theories... [1]

[1] https://en.wikipedia.org/wiki/Worlds_in_Collision

I think they generally pay for it. It came up with song lyrics a couple of weeks ago, where Genius accused Google of scraping lyrics from Genius, but as it turns out Google payed a third party vendor LyricFind for them, and LyricFind pays the copyright holder, so if anything it seems Genius was more in the wrong...

https://www.theverge.com/2019/6/18/18684211/google-song-lyri...

How was Genius in the wrong? They were not paid for the service, and the data was coming from them. It is possible LyricFind stole the lyrics from Genius, but then Genius is still not in the wrong to sue them for scraping their site.

So LyricFind says they licensed the lyrics from the publisher and then verifies against other lyrics websites. Somehow, they ended up with a copy of the lyrics as posted by Genius.

Genius doesn't own the copyright to the lyrics and screwing with the punctuation doesn't create a new work, so I'm not sure they have much of a case against anybody. Maybe they could come up with a ToS violation or CFAA case against LyricFind?

How did that site get the sports schedule? Did they just copy it from somewhere else?

And that's part of the question: If Google's getting all it's data from first party sources, say, scraping it from the team's website, it may not have harmed anyone there. But if it's scraping it from a site which aggregated and normalized the information into an easily parse-able format (hence, providing value, of which Google took advantage), then they must be compensated for it.

Presumably the site being scraped is getting some value from being included in Google's index. If they aren't, they can always opt out.

First you have to find a way to prove it’s your data which isn’t trivial.

Genius had to invent a watermarking system for texts: https://betanews.com/2019/06/17/google-genius-com-lyrics/

You don't have to prove anything to opt out of indexing.

And in the lyrics case, it wouldn't have helped because Google wasn't scraping the lyrics. Google was paying a third party that copied the Genius site.

So it’s a multilayered problem, and content owners are on the losing side.

If it's getting it from a site that's adding no value (e.g. just copying and pasting another site), then there's probably nobody to compensate either.

I agree here. For the median search I make, having to click on a website is a failure. If I search "John McCain age at death", why would I click on a website? This seems trivial, but it is frustrating for even more technical issues. If I search "Non-standard evaluation in left-hand side of mutate command R", it's a bit weird that I'm expected to click the first StackExchange rather than getting a canonical answer excerpted. Ditto when I search "Analytic standard error of a regularized regression". These are basically true/false questions, and it's a win for Google if they give me the answer.

I do emphathize with the companies being scooped by Google, and so I have concern that they are viable economically and that Google compensates them for getting the info from them.

But in the mean time, it's a success for me when my search gives me the answer immediately instead of two or three rounds of indirection.

> If I search "Non-standard evaluation in left-hand side of mutate command R", it's a bit weird that I'm expected to click the first StackExchange rather than getting a canonical answer excerpted.

Are you saying that there's no excerpt at all above the search results, when you expected one; or that there is an excerpt, but it's not from the most appropriate/relevant source?

It's annoying when there's a page that has all the information you might need, but to find it, you first have to scroll past a bunch of Stack Overflow posts that are more suitable for dilettantes who prioritise trying a quick fix over learning best practices. But when you do make the effort to find the most promising link and your browser allows the search engine to record your choice, you at least get a vote regarding how the ranking should be adjusted. When users just look at the results but don't interact with them any further, that feedback vanishes.

A win for Google isn't necessarily a win for Google users.

Maybe Wikipedia could do with a fact-pulling tool that answers small questions like this?

>The one weird case here is AMP.

If you have been to a local news site recently, you might agree that they are similarly "scummy," overrun with banners, popups and native ads.

Try a local news site with Firefox's Reader Mode. If it's still garbage then the local news site isn't worth any more time whatsoever. Keep a running mental list of what sites not to click on. They're typically in the top 1-10 links of search results.

I wish I could hit reader mode before the page loads so my phone doesn't melt in the time it takes for the page to finish loading its 10000 js frameworks.

If you're following a link, then copy the link and paste it into the URL bar. Before you hit enter, prepend with `about:reader?url=`.

check out the mortgage calculator i made: https://www.mortgagecalculator.io/

no dark patterns, loads quickly, uses fathom for analytics so no tracking. There are affiliate links to LendingTree, but they're just text links and i tried to be non-obtrusive to the ux.

also, have an android app that just hit 5,000 installs since being launched in mid-April: https://play.google.com/store/apps/details?id=io.mortgagecal...

currently ranking #3 for 'mortgage calculator' on google play. pretty proud of that! beating out Zillow and Quicken's mortgage calculators.

It's interesting how one of the best sources of information are sites providing the information as a side dish to their core business.

For example I was searching for a lease termination letter template. There are sites dedicated to documents, letters, forms and templates. Many of them are pretty spammy or messy. Then I found a page of a real estate portal. They had just what I needed, without superfluous ads with nice UX. It's in their interest to help people to do this, because then they will probably search for a next apartment. They, of course, are fast to offer additional services on top like searching for the move services. But nevertheless, providing this service is something on edge what they are doing - a marketplace for real estate. Only then they will get some money, so the side experience should be as smooth and painless as possible.

Another example is a tax/salary calculator on a job board. Or... most what Google is doing to some extent, but with different trade-offs.

I know it's nothing new, but it was interesting for me to notice it recently so directly.

Don't know about everything else, but Genius is unparalleled for lyrics to rap and hip hop. I honestly don't see Google having clout to get Zack Fox to do some commentary; Tidal and spotify pretty much cornered that.

Genius is a good resource for lyrics of all genres now. They used to be called RapGenius, if you remember :-)

>with seemingly no opt out (on the consumer or publisher side)

I though AMP was opt-in for publishers? As in you have to write the front-end spec and pick a CDN to cache with.

>I though AMP was opt-in for publishers?

It is but Google will only put you in the carousel at the top of the search if you use AMP. That's a significant enough driver of traffic where publishers can't afford to lose it.

it's not just the carousel. it's organic rankings too.

we had an emergency AMP project after our traffic dropped 35% due to a smaller competitor implementing AMP and google rewarding them by boosting them in the search results.

That's depressing, but I will remember that when I come across an unnecessarily AMPified URL, I shouldn't blame the publisher for making this possible in the first place.

It's not really. Google will highly reward sites with AMP (even when you're not searching on mobile.)

So unless you can survive getting bumped off of the first page of results, not implementing AMP isn't optional.

Yep, it's opt in for publishers. For users, the only way to opt out currently is to view in desktop mode.

Users can use encrypted.google.com to search on mobile, but yeah, there should be a search setting for them.

I've heard of this strategy before, but it has never worked for me. Encrypted.google.com just redirects to google.com, and I still get AMP.

I now use https://github.com/bentasker/RemoveAMP, but that's only possible because I'm lucky enough to have a Jailbroken iPhone. :/

Are you using a userscript app or extension for jailbroken iOS devices?

I'm using software that allows me to load userscripts in Mobile Safari on my iPhone.

I guess you'd call it an "extension"? It feels weird to use that term, as it implies (to me) that there's some sort of extension framework in place. There isn't, it's just code injection.

Thanks I’ll try looking it up!

Yeah extension does sound weird.

Oh, you're Jailbroken? Feel free to shoot me an email (in my profile), there's one thing that's a little tricky...

Okay cool I will!

> pick a CDN to cache with.

Every link aggregator that indexes your AMP page will cache it, including Bing and Baidu. That's what makes safe prerendering possible.

I was referring to the registered CDNs so you could get the icon in search results.

That's what I was referring to as well. They cache your page automatically to enable safe prerendering. Here's how they work: https://medium.com/@pbakaus/why-amp-caches-exist-cd7938da245...

>the difference between a feature and a company

If the right investments were made, every software company's product could be a feature of Microsoft Windows, and Microsoft Windows could be a feature of Intel CPUs.

Microsoft tried that with Internet Explorer and another time with Bing, it didn't work out so well.

Apple's taking another crack at it, and so far nothing looks likely to stop them.

It's not the same as copying the "feature", in many cases Google straight up takes your content and puts it on their homepage. The difference between a product and a feature is nothing. Every app made is just a bunch of features put together. AWS S3 is a great example, it's just like Dropbox for nerds.

A lyrics website recently put fake lyrics on their site to prove Google was copying, and sure enough the fake lyrics showed up a couple weeks later.

Google isn't just taking content from "evil" companies like Yelp, they're doing it to everybody.

Job search, shopping, song lyrics, news, and who knows what else, is all being somewhat blatantly lifted. And nobody can stop it because blocking Google is a death knell to any site

> The difference between a product and a feature is nothing.

Granted, but the comment you replied to was about the difference between a feature and a company.

So it's a case where the webmasters get greedy, and google steals their food, and everyone cheers for google. I 'm starting to think we need search engines to go back to their beginnings, when they were crawling the web , not creating or replacing the web. If they are only pointing to the websites, but rank them by speed, then webmasters would compete who will answer the question faster.

If I want exchange rates I type xe.com in the address bar

This example proves his point exactly.

I'd rather just type CAD to USD.. or BTC to USD, or any other combination in my address bar than type xe.com, load a bunch of libraries, including Facebook connect. Ghostly reports 6 trackers blocked.

Furthermore, it took 3.45 seconds to load xe.com in Incognito, while google only took 1.1

And I like Dave Ramsey's mortgage calculator: https://www.daveramsey.com/mortgage-calculator

Or retirement calculator: https://www.daveramsey.com/smartvestor/investment-calculator

For an increasing proportion of users, the "address bar" is now the "Google search" bar. You may have discovered xe.com when the distinction was much more obvious, but as time went on the browser UI underwent changes, that will change as well.

New versions of Chromium will begin hiding the full URL entirely, probably so that more and more marketing/targeting related UTM parameters can be jammed in without the user ever even knowing.

No it’s so you don’t remember it or care for urls and just search everything making google the only source of info online.

Pay to play baby if you want to show up

I just type "20 usd in eur" in the address bar and get the result without even hitting the enter key.

I don't know about Google but DuckDuckGo pulls conversions from Xe. It is much quicker to duck the conversion than to find it after hitting Xe's homepage.

Song lyrics really ought be from a rights holders site and not j random site with questionable translations - I have seen some lyrics sites that totally mangle the meaning of basic English words.

They "ought" to be, but I fear that regulation censoring these low quality sites would be worse for society in the long run.

Monopolization of cultural information is antisocial.

> Exchange rates

I set up custom search keywords in every browser I use for this, among many other custom searches. In that particular case I query xe.com from an address bar search (not affiliated with them I've just always used the site) as xe <amount> for my most common currency conversion and others like xeyen for Japanese rates, xegbp for British Pound, etc. Takes me right to the page.

Even for various Google services I set them up to save unnecessary clicks.

The automatic calculator and translator is what me switch back from DDG a few weeks ago.

> Although this one I find infuriating because Google doesn't know how to correctly round off exchange rates

I generally search for "1000 USD in AUD" as a workaround.

Both cases showed 2 decimal places for me.

You get 5 decimal places if you divide by 1000...

Thanks, I missed that. I was just looking at the number of decimal places. Is that method accurate?

(1000 * incorrectly_rounded) / 1000 == incorrectly_rounded

It's not incorrectly rounded. It's correctly rounded to two decimal places in either case. Two decimal places just isn't enough digits when you give it 1 USD, so you have to give it 1000 USD.

The great thing about the calculator and specific DSLs like the exchange conversion is that it's trivial to parse. The helpfulness of the "more than just a search index" degrades rapidly into siphoning people to top 50 sites, like scraping metadata from IMDB. I'd much rather they gave me the ability to restrict the information they show me as most of it is distracting to me and reduces my productivity.

If I want to look up an actor I'll just add 'IMDB' to the end and get a much better experience on the site itself than google could ever offer in their search page.

>If you go to any sites that provide these sorts of things they're typically "scummy". Lots of ads, lots of Javascript, lots of dark patterns to make you load more page views

Exactly why go to those sites when you can get the same experience without leaving Google? And its not like Google isn't serving the ads on both Google and those "scummy" websites anyway, and no one is going to out dark pattern Google.

For pop-culture things or very simple queries Google will do, but for everything else I rely on Wolfram Alpha.

As if Google is not the scummiest service out there. Just because it hides its tracking and doesn't quite use dark patterns but manipulates much the same doesn't mean it isn't any better.

Reply on news.ycombinator.com