Yahoo is making a return to search
searchengineland.com
searchengineland.com
As one just example, I searched for a unique error message in code that exists on GitHub, is in a fairly popular repo, and is not new and Google just could not find it. That seems like a very basic failure.
oh wait
Google ignores robots.txt: "Google is evil! They're trespassing on my webserver!"
Google follows robots.txt: "Google search results suck! They're not indexing GitHub!"
User-agent: *
Disallow: /*/pulse
Disallow: /*/tree/
That "/*/tree" rule means that search engine crawlers are allowed to hit the README file of a repo but effectively NONE of the other files in it.Which means that if you keep your project documentation on GitHub in a docs/ folder it won't be indexed!
You need to publish it to a separate site via GitHub Pages, or use https://readthedocs.org/
(Side note: I just noticed https://github.com/ekansa/Open-Context-Data is explicitly listed in the robots.txt for GitHub - the only repo that gets a mention like that. I'd love to know the story behind that!)
Also, very relatable to see a decade old "I'll update this shortly" comment that was never updated. We all have a few of those.
(mandatory disclaimer: I'm a GitHub employee, not speaking on behalf of the company)
Disallow: /account-login
Disallow: /Explodingstuff/
The first for obvious reasons, the second probably because they've uploaded nothing of substance besides a copy of WannaCry.Another extremely noticeable degradation is in finding part numbers, IC markings, service manuals (NOT the useless user manual), schematics, and the like. Anything that proponents of right-to-repair would be extremely interested in, to the extent that I wonder if there's been some sort of conscious effort being made by certain interests to eliminate or limit such information.
Then there's the niche-but-legal adult content. I won't go into too much detail about that, but suffice to say it used to be far easier to find.
It's been 5 years since this notorious item here, and I've only seen Google get worse: https://news.ycombinator.com/item?id=16153840
To test your hypothesis, I did a basic search for exact matches on "we do not synchronize on the update of the broker node" and Google returned 2 search results in 240ms:
- https://github.com/a0x8o/kafka/blob/master/core/src/main/sca...
- https://jar-download.com/artifacts/org.apache.kafka/kafka_2....
Which contain exactly the source code from GitHub that I was looking for. You'll notice that the first result is actually a0x80's fork of apache/kafka. Google states that some entries very similar to the 2 already displayed were omitted, and I'm able to remove that filter. With that filter removed, I can see the same document indexed from apache/kafka on GitHub.
There's nothing I can do or promise directly, but I can assure you that Google takes the quality of our search results very seriously. If you believe we're not delivering quality results, I strongly encourage you to click that "Send Feedback" link at the bottom of your results so that our teams can act upon your feedback.
Disclosure: I work on Search at Google.
Disclaimer: The words, views, and opinions expressed in this post are my own. They are not representative nor do they represent my employer in any capacity.
What I can say is that we have a feedback process in place for Google Search that we use to improve our product. You can send feedback and check the box to allow our teams to contact you if you’re interested in a follow up. Of course, given our scale, we’re not able to follow up on every bit of feedback but that doesn’t mean we don’t review or act upon that feedback in some way.
Congrats and hope Google takes advantage of HN, similar to how startups use this forum to engage with users - it is now a meme that Google Search is unusable so there must be something to learn from the audience.
I will use the send feedback button tomorrow as you suggest.
Yes, that meme is very common. I hope I can contribute positively to these discussions by offering an outlet for feedback, and humanizing our organization. Google’s Search organization is large, so it’s certainly not monolithic, but we’re staffed with a bunch of normal, hardworking, genuine human beings like most companies, that care about the impact we’re having.
I’m happy you’ve found some value in our discussion. :)
1. Most large classic forums using vbulletin. Try picking any rare word or phrase with less than 100 total matches via the forum's search tool and compare to the Google verbatim results.
2. This very site, searching for an uncommon word such as "memeplex" returns hundreds of unique results according to hn.algolia.com, but only 65 according to Google via site:news.ycombinator.com "memeplex".
3. Fanfiction sites such as fanfiction.net . Try randomly picking an obscure 'fandom' with only a few hundred stories, and search for the name of one of the main protagonists. It will only retrieves a small fraction of all the existing stories that mention the protagonist's name.
EDIT: I originally had another example involving macrumors.com but then realized there was a formatting mistake in the search query.
1. If you could link to specific examples and queries that’d be super helpful for someone like me that’s not active on the forums you’re describing.
2. Algolia is a fuzzy matching search engine. Searching for memeplex [1] returns matches like “megaplex”, “memepher”, “meeples”, etc. Unchecking typo tolerance in the settings returns < 100 results in line with Google’s results.
3. Again, if you could link to specific examples and queries that’d be helpful.
[1]: https://hn.algolia.com/?dateRange=all&page=19&prefix=true&qu...
I'm not quite sure what you mean for 2., I see every exact match highlighted in a rectangular box in a different color. Do you not see that on your end?
Just counting the exact matches, there are well over 100 unique results.
Don't they use their own tools? Is there an internal search engine that everyone uses at Google? Are they trying to gaslight us pretending there's no problem? Can't they hire a hundred people to use Google search and report what they found?
Sure, props to that person for engaging with the userbase but we're not talking about an obscure bug here. Every day there are dozens of complaints about Google search on HN alone. Surely we're talking about low hanging fruits in terms of bug reproduction.
First, we can agree that Google Search is attempting to solve an astronomically hard problem. Like mind boggling hard. Indexing the entire web and serving quality results to unstructured queries from billions of users every day in under one second is no small feat.
Second, Google is not monolithic. We employ more people than most cities have citizens. Furthermore, many more people than our current staff have come and gone over our 20+ year existence. It’s better to think of Google as an organic entity than a rigid command-and-control hierarchy. Are you able to think of a city in the world that does everything perfectly? I certainly can’t, and yet, there are cities that are better and those that are worse for any set of criteria that one may care about. As it is with large companies like Google.
Third, while there’s an objective element to search result quality, there’s also a significant amount of subjectivity. Your idea of quality results may differ from another person’s idea.
Checkout Paul Haahr’s talk on “Improving Search over the Years” [1]. He summarizes our work the best when he says things that look easy on the outside can take a lot of work to implement.
As it was with our state-of-the-art automatic synonym system that works on any written language in our corpus. (More details in his presentation.) This system is a transparent workhorse from the user’s perspective.
Here’s a simple example you can compare between Google, Bing, and DuckDuckGo: “united flight formations”.
Two of those search engines will show a bunch of things about United Airlines as top results because that’s what you would expect to get when you’re only focused on matching terms. Only one of those search engines understands the meaning behind the query and returns everything to do with formation flying as the top results.
If you use our products and you mostly enjoy our products, it’s in your best interest to give feedback when you feel we’re not serving your needs. You’ll find that most of products, Search included, have open feedback channels that we do review and act upon.
I've definitely seen that sort of thing before but there is no such link there at the moment -- at least not when searching from my iPhone, whether or not I'm in desktop mode. I just see a large error box that says "It looks like there aren't many great matches for your search" followed by the link to the a0x80 fork.
By the way, the a0x80 result highlights a serious problem with search results: the GitHub URL is strangely modified. Instead of showing the full URL or even a prefix leading up to it Google is selecting parts of the URL, showing "https://github.com > src > transaction" on mobile and "https://github.com > kafka > coordinator > transaction" when I request the desktop site. In neither case is it obvious that the content isn't the canonical source from Apache. I've noticed this middle-out truncation for GH urls before but I'm not sure when it started.
Every single returned link after the first 3 had the superstring version of the name and not the correct name. It turns out that this returns endless results for a fairly well known singer, not my friend.
So now did I not get the results I was looking for, I got tons of results that were objectively wrong.
Then suddenly, about 6 pages into those results, I started getting ones for the correct last name, but now the first name is a mess.
This happened on Google, DDG, Baidu, Sogou, Haosou, Dogpile, the current Yahoo search, Bing, and to some extent on Yandex. Naver was worse, Daum totally worthless with incorrect results.
Utterly worthless.
The thing is, my friend's name is surprisingly fairly unique, there's probably less than 20 people in the world with that specific name. It's like the search engine's desire to fill the screen with worthless garbage results has overpowered the need to supply the 2 or 3 that are actually correct, even if the quantity is a little disappointing.
I've basically decided that my friend's name is going to be my search engine quality test from now on, the results are so spectacularly terrible.
All I want is something the crawl the web, suppress SEO spam, and let me "search" on things exactly as I've quoted them. Like we used to in 1998.
Synonymization is the worst thing to ever happen to search, and it keeps getting more and more aggressive.
Generally though these days I'm trying to distance myself from Google so if anyone has any other search engine suggestions (beyond the usual DDG, Bing, Yandex, Neeva) I'm very open.
I do like this idea however.
I read this constantly here (echo chamber) and I can't help but feel it's a little biased/overdramatic.
Now only we get tons of ads back as first results, Google keeps rewriting the queries for whatever "helpful" nonsense.
I have recently almost completely stopped using Google's search engine due to the fact that I am very often offered zero search results for simple queries (usually involving quotes though) .. It's so bad I can't even believe it.
Note: I've been a Google search since it started... Gmail since Beta, etc...
At one point, I thought that maybe they started punishing ad-block users excessively.
Pure speculation, but innovative companies at first, they started over-hiring and bloating, using questionable interviewing techniques (puzzles, Leetcode), taking on thousands of employees who were just there to game the system, coast, and collect the check.
It just looks like they stopped caring.
It’s time for competitors to start wiping them out.
Otoh what Google's terrible at is finding new sites with interesting content. If you don't already know where your going, you mostly end up in spam land. I'd like to see a search that addresses that.
Browsing through directory trees is complimentary to search, and has its own power for surfacing unexpected connections, i.e. information you didn't necessarily know you were looking for. I would love to see something well maintained like that again.
Asking ChatGPT for websites about a topic does a fairly decent job, it seems. At least, it doesn't give me any spam.
The question is if it should be human/user curated, or automatically generated with e.g. some AI.
Mastodon is similar to the old decentralized discussion boards - each server run by its own admins, and specific to a set of narrow topics.
ChatGPT/LLMs can essentially crawl _anything_ they want, regardless of legality, license, consent, etc. These models are trained on anything that can be ingested. Once trained, you can release the model with plausible deniability. There's no 1:1 relation between ingested content and outputs. LLMs that "cheat" by ingesting content they shouldn't have will have an advantage over those that don't.
Google and other search engines don't have this luxury. If they serve a result, they have to make sure that they're not violating any license. If they crawl the wrong content, they have to make sure they don't serve it.
If Google removed copyrighted content from its index, the results would be Wikipedia and...not much else.
We will see. The idea that ML models contain the mere creative essence and are generative from something that cannot be copyrighted is not one that has been tested in court.
I personally am not convinced: My own experiments with prompt-stuffing GPT definitely seem to reveal corpus.
I am reminded of a story of how billg would type a command into basic computers at trade shows to "reveal" that it contained microsoft-copyrighted code (gotcha!).
I imagine if someone did that in front of a judge it would be game-over.
suppose chatGPT or whatever is taught to ignore SEO, those people will fight tooth and nail to be relevant and keep their jobs of pushing users towards the content THEY want....
will these new search engines reject/punish/hurt SEO? what will happen to that industry?
Which in my opinion is a good thing.
Like a Frankenstein monster made out of the decrepit remains of dotcom juggernauts. I love it.
They don't have action games like pool on there, though.
Now there is just so much of everything we use to enjoy with a sense of curiosity. It's all very similar to growing up. You don't and mostly can't fancy your childhood toys the same way anymore.
If they can't fix something they already have, I doubt they would be able to win my trust with search.
Looking for a product manager for this enormous undertaking by posting a job offer is quite telling already.
Let's hope that I'm wrong.
I wonder. Now is the time to pick up a whole slew of ex-googlers.
(And aren't non-competes illegal in California, too?)
It would be funny if Google is just so full of itself, that it lets its search slide to uselessness, then fires skilled devs re: search, and yahoo snags them up.
edit:
You know, something is just really dysfunctional at google. I almost think a government forced breakup would help them.
(sorry for that joke :P)
Almost every search engine on the market is either Google results or Bing results repackaged.
Fun fact: because of legacy contracts, some search engines use results from a Yahoo feed, which in turn are just Bing's results. DuckDuckGo was one of them, they were paying Yahoo, who was paying Bing. The last I checked (3 years ago) they struck a deal with Bing directly.
The company I used to work for would often work out deals where we'd resell search engine results to other search engines too... Very interesting world.
I must have missed the memo on this, but when did “people” become an unacceptable word in corporate communications?
3 use the word "folks"
This distinction is often unconscious and absorbed through immersion in social justice circles, similar to how luxury beliefs (such as "defund the police," "trans women are women," etc https://nypost.com/2019/08/17/luxury-beliefs-are-the-latest-...) are absorbed.
You have to use that kind of language to signal your belonging to the "elites", and the HR departments that usually write job ads or staff Twitter accounts are full of those type of people. From that it's just propagated everywhere else.
Edit: Congratulation HN on your brilliant policy that prevents me from responding to any of the comments under my post (looks like I got throttled, so editing the post is the only thing I can now do).
Edit: Congratulation HN on your brilliant policy that prevents me from responding to any of the comments under my post (looks like I got throttled, so editing the post is the only thing I can now do).
"So what often happens in political discussions on HN is 1) a person, such as yourself, posts dissenting views in response to several messages, 2) those messages are heavily downvoted by a brigade of users, 3) the person's account is rate-limited or shadowbanned, 4) the brigade freely posts comments against the dissenter because, being many users, each only posting one or two messages, and being of the censorious brigade (the dissenters usually oppose censorship and don't downvote for disagreement--see how even one GitHub user couldn't resist drive-by down-thumbing your comment), their comments don't get downvoted, so they don't get rate-limited.
The end result is, very predictably, a form of mob censorship. And since HN approves of downvoting for disagreement, nothing is done to stop it." https://github.com/minimaxir/hacker-news-undocumented/issues...
Though for wokeness, you also run into supporters who say it "doesn't exist", by which they mean it's good (in a "it's just common sense" way).
Also, my post wasn't a criticism of it and I don't even disagree with it (fcvo "it").
So really it’s the anti-“woke” crowd are unable to look at anything without feeling oppressed by the “woke”.
I'm from the south, in the US. Folks is just another way to say people. End of story. The people I've heard use the term "folks" are kind older people with a thick southern drawl. I also hear corporate managers use the term to refer to a group of people. I also hear young adults use it to refer to a group of people...
On second thought, I don't think someone using the term "folks" has ever been used to signal anything more than them referring to a group of people. (Also, I feel like I should mention I'm conservative, and it has never crossed my mind to think somebody using the word folks is "woke" or "antiwoke")
https://newdiscourses.com/tftw-folks/
or my other comment here: https://news.ycombinator.com/item?id=34589768
Once again, that website has nothing of substance. It pulled a couple random quotes from "woke" literature that used the word folks and then made a grand conspiracy about how there's some sort of secret meaning behind the term. The term "folks" is still alive and well everywhere and has no double meaning associated with it whatsoever. What you're doing here is equivalent to what the "woke" did by trying to convince people the OK symbol really means you're a white supremacist[0]. Both of these are delusions, in every sense of the word.
One last thing, if anyone ever tries to convince you there's a grand conspiracy at the top echelons of society, where the elites are actively doing this one thing to control the masses, they're probably deluded. Hanlon's razor typically applies in these cases: never apply to malice that which is adequately explained by stupidity.
[0]: https://www.npr.org/2019/09/26/764728163/the-ok-hand-gesture...
> ends up with males being incarcerated in such prisons
Well, no, those are women. The myth that men use inclusivity to get access to women is just that, a myth, unless you can provide evidence that this is really a common occurence. Obviously violence in prisons needs to be avoided but that's a different topic.
[0] The quote is either Bush, Jr. or Obama.
Oh... And I can't for the life of me imagine how it could be interpreted as more "woke". In fact here in Canada it's kind of a phrase used more by the "right" than the "left"; (it's actually a source of much humour how much this word is overused by Ontario's right wing hockey enforcer barely high school educated 'populist' premier)