Hacker News Reader

It's worth mentioning that indiehackers.com is a JavaScript-only website and doesn't work with JS turned off. Google's spider can execute JS, while others do not. This is why indiehackers.com doesn't show up, not because of the search engines themselves.

I have a similar issue with a side project of mine. JS-only websites solve this with server-side rendering or static generation.

When I think of a search engine I think of it as two things.

1. A web crawler 2. A search index

Or as a good boss used to say garbage in garbage out... the quality of the engine is as much a function of the index’s ability to rank as the quality of the input from the crawler... the fact that none of the other engines crawl with JS enabled is a huge competitive advantage for google

Well sure, but you could also just make a website with a modicum of thought and then all search engines will have no problem.

Anything that loads no resources unless I have javascript enabled (and is a general purpose website, not some 3d toy or whatever) is trash.

I mean, clearly this site has value to the OP so it is obviously not trash.

What value a service provides is what matters in the end. Implementation details are secondary.

Pulling gems out of trash does not make it not trash.

Bing also renders JavaScript [1][2]. Though there's not a lot of information out there about the details like what browser engine they use.

[1] https://blogs.bing.com/webmaster/october-2018/bingbot-Series... [2] https://searchengineland.com/bing-crawling-indexing-and-rend...

>> Google's spider can execute JS, while others do not.

So Google is enabling the excess of JS we see today? Sites wouldn't do that if it couldn't be indexed by Google.

Yes, they would. They'd just use NextJS or similar to push down a static skeleton that minimally covers the crawler. (My current company has to provide data to the Facebook crawler and that's what we do.)

Nothing prevents a site from doing that and just loading JS afterwards. Or from still being basically read-only without JS. Which might be an improvement...except within epsilon of zero people care in the first place and Google is building software for a rather larger proportion of the market than that, so it's an improvement for what's barely an audience.

Nobody is really "enabling" it except for a browser. And that's not changing. The shirt-rending is becoming tiresome.

I'd be curious to know why your company specifically targets large-market crawlers like that, instead of just making things crawlable. It feels like leaving money on the table when you overfit your stack to one market.

It's a shame you find it "tiresome" when people insist on retaining their own opinions, but your exhaustion is not particularly relevant.

We need to build shareable landing pages. We share via Instagram and Facebook (for now). We use NextJS because they do provide static rendering that covers what we care about and are likely to care about anytime soon.

And you can have whatever opinion you want; nobody’s saying otherwise. But the constant whining about nothing — and it is nothing — is exhausting.

Yet another reason to support other search engines. I haven't used Google in years, and I haven't had any real problems.

> Google's spider can execute JS, while others do not.

I’m honestly having a hard time seeing how this isn’t a bug in Google search.

1 point by jefftk 0 minutes ago | edit | delete [-]

A search engine is trying to predict which pages will be helpful to you in response to your query. This means it should take "how does the page look to users" as input, which today means executing JavaScript.

(Disclosure: I work for Google, not on search)

Contrapoint: if you keep enabling hostile behaviour, you'll get more of it.

Non-JS web indexing as a default would vastly dimnish the utility of JS-only pages.

It is a feature, not a bug, because it provides value to customers.

I separately think sites that only work with JS enabled are not great, but that's a problem with the system and not with individual actors.

On the other hand, in the same way that Google is (claiming to be) trying to make the web fast with AMP, they could make the web fast by refusing to index pages that don't provide a non-javascript experience. (Yes, you can make a slow page without Javascript, but it sure is easier with it)

> For everyone who is suggesting Google search quality has taken a hit, I would suggest a quick test to see for yourself that Google is still WAY better than the competitors when it comes to long tail queries.

I think these are both true. Google quality has certainly fallen in my opinion (perhaps as a result of having to constantly counter SEO tricks). Other search engines still have quite a bit of catching up to do as well.

Depending on your searches, YMMV. I finally switched to DDG at work because Google refused to show me relevant results for some (admittedly obscure) searches related to lensing in F#, preferring instead to give me political opinion pieces(!) that were completely irrelevant to my query, along with various results about video games. DDG had no such problems.

This is one example of many, but it was the last straw for me. I guess techies aren't Google's core audience anymore, I get that, but it feels like we've lost a valuable tool.

For the last decade, the competitors have been meeting or beating Google search relevance in test where the branding is removed.

People that can actually notice a major difference in result quality should probably go work on search optimization.

> DDG (usually) finds the article written on toomas.net, but not the indiehackers interview. Bing often fails to list toomas as the top result, and doesn't find the indiehackers interview at all.

Are you able to explain this? Bing and DuckDuckGo are the same thing. DDG uses Bing's API (for non bang searches).

I cannot explain it, hopefully someone from DDG actually chimes in :-) If they just use the Bing API, how can their results be better than Bing? I suppose they also use other sources.

I find their results are usually as crap as Bing, especially when you use a localized version. That's why I use StartPage (uses Google).

https://help.duckduckgo.com/duckduckgo-help-pages/results/so... says "a variety of partners, including Oath (formerly Yahoo) and Bing."

I'm also seeing crawls from duckduckgobot so they're working on their own system at the moment. It may already be deployed for some easy queries.

As far as I know duckduckgo used being for images and some other stuff, but not for the main text search.

> Bing and DuckDuckGo are the same thing. DDG uses Bing's API (for non bang searches).

This is not, and has never been, true. Why do people like you keep saying it?

Google being better then competitors on this one query tells nothing, especially regarding Google's performance compared to its past self.

Slight tanget, but it might be interesting to some on search results and how Google's search results can sway users.

Googles been able to use ephemeral experiences (like autocomplete and answer boxes) to influence users, especially undecided voters. The research, which was reproduced by a German team, is believed to have influenced 2+ million people in the US election alone --

take a listen on the congressional testimony http://naplay.it/1157/37:25 (recommended listen speed is 1.5X)

That testimony is highly misleading. Taking the highest estimates possible, which are based on essentially a conspiracy theory, the number of influenced voters is much lower than 2 million.

He claims they moved 2.5 million votes, not voters. This relies on an assumption of straight ticket voters and about 20 votes per voter. He doesn't correct people when they make the votes/voters mistake.

And even that new number, 100k voters swayed is not really well founded. He makes a couple of overlapping claims, one of them I looked into deeply and found that using his own papers, a reasonable way of expressing it was that 8-10k additional votes for Clinton was a reasonable upper bound, as he claimed, might be affected.

And the number was likely lower (and was only that high because more people identify as democrat than republican in the US).

(I work at Google, but take interest in this mostly because it's just terrible abuse of mathematics)

> For everyone who is suggesting Google search quality has taken a hit, I would suggest a quick test to see for yourself that Google is still WAY better than the competitors when it comes to long tail queries.

Maybe but it's pretty clear to me that Google now no longer indexes entire websites. This as far as I can tell has not been a concern in the past.

Saying other search providers are worse does not refute the statement that Google Search has been reduced in quality.

The reduction in Google Search quality is their need to be "helpful" with their desire to use my past search results, and other "big data" they gobble up about me to "improve" my specific results when in fact it makes it worse. That is just the base line, then you have the 100's of other things they have done to the search over the years for political, regulatory and other reasons.

Google search gets objectively worse every year than the previous year.

Is there a bang to disable personalized results?

I agree that personalization is often not helpful, but I think the more important reason for Google getting worse is that the web is getting worse.

That doesn't explain why DDG returns more useful results for many development-related queries.

I get the feeling that DDG specializes in dev related content a little bit.

Bing seems to simply ignore the quotes in your query after the first result (for query "This idea that you need to water down your pricing or offer a free beta period is bogus") and then just displays random results. Even SoundCloud is listed there..

Is saying you are better than Bing and DuckDuckGo saying a lot?

OK, are you suggesting there is a search engine which produces better results for this specific case?

The version of Google they had in 2010?

More like 2003-2005 for me. By 2007, the quality was sliding due to them targeting mainstream folks, and deprioriting the tech fridge stuff that I care about.

would google have returned results from a JS rendered site back then? The ability to index client side JS rendering puts google ahead of their competition, as more sites become JS rendered they support google's monopoly.

Improving one minor feature can’t fix something where the basic functionality has been broken.

That’s like Yahoo designing a fancier logo while the company falls apart.