AntiPolygraph.org appears to have been “de-listed” from Google on key searches | Hacker News Reader

AntiPolygraph.org appears to have been “de-listed” from Google on key searches

antipolygraph.org

330 pointsgiles_corey139 comments

Well, to be fair the website looks like it does the most it can possibly do to avoid good SEO and search engine visibility.

It uses `<meta keywords` and other techniques that apparently search engines like Google do not take into account.

On the other hand the HTML of the page (view-source:https://antipolygraph.org/ ) does not contain any guidance as to what's the page's focus. No header tags are present, everything is `<table>` based and contains deprecated markup and almost all of the links it contains are internal. The markup is HTML4 which might be an indication to Google the page has not been updated in a while.

The website can easily be found in Google (https://www.google.com/search?q=site%3Aantipolygraph.org&oq=...) - it just appears to not be ranked very high as it aged.

Don't forget Google tends to favour newer and updated content to older content. The website also doesn't have a sitemap as far as I know which doesn't help with the problematic markup.

Unfortunately all these changes mean that it's now tougher to find old websites with specific information that you're looking for on Google nowadays, in general. It feels like Google used to be much more useful for that ten years ago than it is now.

Google's purpose seems to have changed from helping people find the most useful information, to funelling people to information that Google wants them to find on a particular topic, whether its the most useful or not.

Are we really still pretending like Goog/FB/etc. aren't effectively mass information control mechanisms first and foremost?

What we can do in this day and age to find relevant information fast? Use another search engine? If yes which one?

As far as search engines go, DuckDuckGo is pretty neat, have been using it for 3 years now and can count with my fingers and toes how many times I've used Google since then. And half of those times it was because I typed the search term directly into another person's browser's address bar which defaulted to Google.

Second DDG. It wasn't very good a few years ago, but it's improved significantly since then and is now my daily driver.

I haven't used ddg as it used to be fairly terrible, but google may have slipped enough and ddg may have improved enough now, I'm going to give it another shot.

i've also used DDG for the last few years. it has indeed improved a lot. i don't really ever need to turn to google to get what i'm looking for these days.

> Google's purpose seems to have changed from helping people find the most useful information, to funelling people to information that Google wants them to find on a particular topic

This statement needs examples to stand.

Normally I only use Google, but I've also experienced this. In contrast with a few years ago, more recently Google gave me such bullshit that I was like, "Just give me the pages that have these keywords, this is so obvious" and I went to duckduckgo.com. There, with literally the exact same query, every one of the top links was relevant and had my search terms. On Google none of the first page of links had what I searched for. This certainly supports "they show me what they want me to see", rather than what I asked for.

On the flip side, I still find Google useful and I must say that I often am happy to read what they decide to give me: even if it's not what I searched for.

I've done searches on Google with 3-4 words and the results given back had excluded all but one word. The "not on page" results are usually pretty infuriating since that's what I am looking for

Those aren't "examples". Can you be specific?

If you haven’t run into this I submit you don’t use google much.

It’s infuriating

It is impossible for me to give you a search term that will be garunteed to show you this effect. Google tailors every search to each person too much.

It ends up feeling like you've been gas lit by Google when you report one situation and no one else can see it

Add double quotes around the words, works like the old +word syntax.

(I don't work on Search, but use it a lot :))

Quotes or pluses stopped working a while ago.

Even "Verbatim" isn't actually verbatim.

I am amused that people are only now realizing just how awful Google has got compared to 2000s.

That's simply not true. Quotes superseded pluses and they definitely still work and it's trivial to demonstrate.

"mainframe" "kubernetes" vs mainframe kubernetes

https://www.google.com/advanced_search?hl=en&fg=1

agree on ‘not on page’ then why are you showing this!? facepalm

Stratechery has a great article about how Google chooses to show their local results over more useful results from its competitors:

https://stratechery.com/2018/the-bill-gates-line/

The relevant explainer video: https://youtu.be/k9UqqmIJW4g

> Are we really still pretending like Goog/FB/etc. aren't effectively mass information control mechanisms first and foremost?

No. They are a search engine and social network first and foremost.

Well, no. They are massive databases of detailed, specific information about every individual, first and foremost.

One of Google's cofounders once said that if there is more than one search result then that is a bug. That sort of thinking inevitably leads to this sort of thing. There are always shades of grey.

I'm looking for a link, will update when I find it.

Here is an article citing a Charlie Rose interview from 2005.

https://techcrunch.com/2013/01/04/an-old-eric-schmidt-interv...

yikes, i really hope they are not so shortsighted these days.

a lot of the time people are searching for something, they do not know exactly what they are looking for enough to articulate the perfect string of search terms and need a plate of more general results to get to what they want... this seems painfully obvious.

Page and Brin are saying that they want to deliver that specific final result even when someone starts off not knowing what they’re looking for by using context and their user data to understand what the search is really about. Not that they want to deliver one general result when someone makes a general query.

I’m not necessarily agreeing; just explaining their thinking.

We have a blog on agrifood tech associated with our site and we’ve published over 1,000 articles often interviewing some of the most successful entrepreneurs, VCs, and senior execs at food and ag companies. We’re by far the most authoritative thought leader in the industry and if you go to any industry you’d hear our name a few dozen times but we get massively outranked by Forbes articles, TechCrunch, CB Insights (which is a poor authority in our space). It’s quite sad just how bad Google can still be.

What do those out-ranking publishers do differently?

higher domain name authority

But not domain expertise. Google has a rich get richer ranking that favors broad generalist publications. If you’re seen as an authority in anything then it seems like you can also rank high for something tertiary.

yea VOX ranks high for everything even though they are generalists than specialists. .

They have bigger and more general-purpose audiences and thus a higher probability that someone going to google is looking for them. Self reinforcing.

That sounds to me like Google is optimizing retrieval for deeper searches within a narrower set of "authoritative" websites. The ideal solution would involve putting in the logistical and theoretical engineering work to classify and validate a huge number of websites good for fewer, specialized answers.

But for whatever reason, the reality seems to be a relative few websites are selected to provide "good enough" answers on shallow queries for a huge number of topics or objects, Wolfram Alpha style.

Maybe this is due to developer laziness, gradual dilution of cutting edge engineering, explicit desire for simpler solutions or real practical limitations, even at Google's scale. In any case, the result appears to be that sites like yours are poorly surfaced because they're not very generalizable. Whereas CB Insights claims it knows about your query already, there's an internal API setup for exposing CB Insights already, and CB Insights passes a basic sanity check on the answers.

This is really unforunate and I sympathize with you. I think there is a lot of opportunity for a search engine that can optimize for very specialized queries across many domains, including arbitrarily historical resources. But I think such a search engine would have very different goals and incentives than Google from the outset.

I think you’re onto something. Google’s “fake news” algorithm change probably didn’t want to be seen as politically or viewpoint driven, so an easier approach was to just de-prioritise large “authoritative” websites even for deep searches where more relevant results exist.

Actually, that’s almost exactly how Google described the algorithm update...

People still seem to find us depending on the exact search terms you’re using but it’s Google’s user experience that really suffers. If someone randomly publishes “the top 25 Agtech companies” on Forbes it’ll jump to the top of the list when searching for Agtech and stay there for half a year. Google is embarrassing itself with these kind of shallow and superficial results.

If your industry doesn't have a large web community, then it's difficult for Google to know that you're authoritative. From Google's outsider perspective, all the links are going to Forbes.

You are thinking in terms of the old world, the PageRank algorithm.

Everything you say is true in that old world.

In today's world, Google is optimizing for ad revenue.

I could see how optimizing for PageRank could go out the window in such a world.

Even if PageRank still has influence, it's not clear to me the site doesn't have a large web community with a lot of links. Maybe it does, and the ad revenue considerations are overriding that.

doesn't google "rank" journalistic sources by perceived credibility? Could be that nobody at google has bothered to give your website a rank?

No, Google ranks journalistic sources by revenue and distribution, and pretends that it is ranking by credibility.

What is an "authoritative thought leader"?

Semantic markup : Internet conspiracy theory :: fluoridated water : real world conspiracy theory.

Here's your monthly reminder that "relevance" is fundamentally subjective, and therefore all search and discovery services, both on the internet and offline, give you their opinion of what you'll find useful.

How useful you actually find a given service to be is entirely a function of how closely you agree with that site operator's opinion.

I definitely think that's an incredibly legitimate concern and wouldn't want to dismiss it because it's such an important issue. It is also important to remember, though, that a lot of these changes are made in ways that (seemingly, at least) attempt to prevent sites from gaming the search algorithms. I have seen the quality of results I'd expect from a Google search go down, and I attribute that to quite a bit of factors, including Google's self interest. I also try and consider just the massive rise in general content available through the web, the further and further growth of it as a tool for manipulative information or propaganda, as well as its growth as a marketplace. I only mention these things because I think a lot of people, including myself, will tend to ignore a lot of the external factors that forced as well as allowed Google's search and results algorithms to be in the position they are in.

It's much more adversarial than it was at the beginning.

This may be a good thread to let people know about a hidden setting I found out about recently called Google Verbatim. I thought certain sites I used to search for had fallen off the Internet, but the change was how Google ranked its search results.

Try the two searches below to see the difference:

https://www.google.com/search?q=java+vm+language

https://www.google.com/search?q=java+vm+language&tbs=li:1

Enjoy!

How do we use it? Just append &tbs=li:1 ?

In any search result page, click Tools > All results > Verbatim. That is also how you restrict e.g. the temporal range of results (Tools > Any Time > ...)

Yes! It is time for someone to disrupt Google. Please, we need a better search.

We really need good, alternative search. I miss Lycos, Hotbot, Yahoo and the days where if you didn't find someone in one place, you checked another. Searching for things took real work and you could find a lot of different rabbit holes.

Today, rarely do we go past the 2nd page of results. About 1/3 of the time, I get frustrated with Duck Duck Go and add !g.

Nonsense, seo doesn't matter (as per Google) when people come or seek your site. The more likely explanation is that Google and its 3/4 quarter-of-a-trillion valuation depends of having good relations with the Feds, anti-trust and all. See At&T how it manages to screw everyone when it comes to protecting their own interests. They do what the Feds and NSA asks them to do.

Your 20 year old site doesn't age in months to go from #1 to page #3 (dead!). The fact that the site has gotten press is even better, it should help your rankings. But by now Google is not longer a fair engine. Their rankings are manipulated to boost their bottom line, directly or indirectly.

Nobody has any illusions about Google being fair but if the first thing people do to "solve" this is to bring out all the hairy conspiracy theories instead of fixing the stupid site, no wonder the ranking still sucks. facepalm

Only once that is fixed and the rankings are still bad is complaining about "feds" and what not maybe warranted.

">> instead of fixing the stupid site" Fix what? If Google wants to demote you, doesn't matter what you do. See https://www.mcclatchydc.com/news/special-reports/insider-thr...

What happened that it went from #2 to page 3? Nothing. Page 3 is a black hole, used to be called a page 3 penalty (get zero traffic, might as well be banned.) The site is accessible and readable. A lot of sites are old fashioned...see Drudge.

This tendency to characterize the recognition of basic aligning interests as "conspiracy theory" is really odd.

I personally don't think GP's theory passes Occam's Razor, but that does not refute the underlying contours of the situation.

All of what you said the website doesn't do is exactly what a decent website shouldn't need to. Google should index it according to the user-visible content, not hidden metadata.

Metadata defines what is the content, the sections and the titles. A blob of text is worthless without context.

HTML pages (written properly) don't need metadata for that. It's part of the document.

Metadata is good for things like thumbnails, authorship, or summaries, none of which are necessarily distinguishable in valid HTML.

You're conflating metadata with the HTML meta tag. Please consider that everything is metadata except the bare text.

The point of the message above us was that the content on this site is not formatted properly as HTML.

Tags aren't just meta. For example, a sentence has a different meaning when it's alone vs. part of a list vs. a caption of an image. All of those things have their own tags.

We are saying the same thing.

Google does crawl "the way you see it" and not via raw html parsing. They interpret JavaScript and CSS.

Try putting display:none on a div with some text in it, or putting the font-size to 1px. It will not rank for that and you will probably trigger a penalty of some sort if it's egregious.

Invisible or near invisible text can be discerned without rendering the page.

Google does execute JavaScript but as best people can tell it doesn't happen all the time, just periodically. It would be computationally expensive to do this for every request on every spider view for every web site.

Less than encoding millions of hours of video per day? Libchromium is quite fast and Google has lots of resources for this.

They had rendered screenshots of every result with the matching section highlighted... almost 5 years ago.

I'm not saying they don't do it - everyone knows they do. I'm saying for many, many cases there's no reason for them to do it for every page and every site.

Then again it's the third result on DuckDuckGo when you search for 'polygraph' so it's not like it's that hard for search engines to find it.

None of these are particularly red flag SEO issues. The page is clean, mobile friendly, served over HTTPS. Google doesn't need to assume content update frequency based on the style of markup used, it literally spiders sites and reads their content, likely multiple times a day.

When I do a search for Polygraph I get the results from Wikipedia, Vox, APA (http://www.apa.org/research/action/polygraph.aspx), NYTimes on the first page. They are mostly critical articles about lie detectors and they all look more like something I'd want to read than AntiPolygraph.org that looks more like a conspiracy blog than well-researched information. I'd say the problem lies somewhere else, not by google censoring your content intentionally.

I appreciate how the present blog post may strike some as conspiracy-minded. Especially those not familiar with the history of Operation Lie Busters, a U.S. criminal investigation that targeted the proprietors of websites that provided information to the public about how to pass or beat a polygraph test.

As you'll see in this recent post announcing the leak of the U.S. government's polygraph school's documentation about polygraph countermeasures, the federal polygraph community considers AntiPolygraph.org to be a major threat, and it features front and center in their training:

https://antipolygraph.org/blog/2018/06/09/ncca-polygraph-cou...

None of this necessarily entails any state influence in Google's action to delist AntiPolygraph.org on certain key search terms.

It is true that our web design is dated. But this doesn't seem to account for the site being completely delisted from Google on such searches as "polygraph" and "lie detector."

I am not familiar with any other instances of such partial delisting by Google. Are you? If so, what were the causes for that?

> But this doesn't seem to account for the site being completely delisted from Google on such searches as "polygraph" and "lie detector."

It's not delisted, it's just not on the first page as other users also reported and I just verified myself.

With every change of the ranking algorithm there's a bunch of sites going up or down and nobody can tell you the exact reason why - it's easy to just call it censorship but in reality it's probably something else. Like the top comment already explained that your site markup has a bunch of missing information that makes it hard for search engines to properly scrape you.

What search term(s) did you use, and on what page of results did AntiPolygraph.org appear?

The google search "beat a polygraph" gives antipolygraph.org as result number 10.

The google search "pass a polygraph" gives antipolygraph.org as result number 11.

The google search "polygraph" give antipolygraph.org as result number 188.

The markup rules were downgraded years ago. No h1 or title or meta tags shouldn't be the primary cause. Probably more to do with https, mobile and amp than the above factors.

Google has no problem scraping html pages and getting importance and content topics from inbound links, social media links and the actual text on the page.

> With every change of the ranking algorithm there's a bunch of sites going up or down and nobody can tell you the exact reason why

And that is a problem. No company with as much influence on the web as Google has should be able to hide behind a phrase such as "it's just the algorithm". "De-rank every web site with the word humuhumunukunukuapua’a" is an algorithm too.

Every ranking should have a very clear reason, and legitimate web sites (yes, this is a judgement call, but a site with a history like this one would qualify) should have access to that reasoning behind their ranking.

But isn't the reason that the exact ranking algorithms and factors are not public mostly to prevent people from gaming the system? If the exact formula for being the best ranked result would be public every seo-spam site would adapt to them too and everything becomes irrelevant again.

> I am not familiar with any other instances of such partial delisting by Google. Are you?

I've been experiencing partial delisting on my own website over the last 3 months. Google Search Console [1] shows they now exclude 25% of my website from the index. Search Console lists those pages as "Crawled - currently not indexed".

Mine is also an old site (nearly 20 years old), but it already used CSS & XHTML, and I'd been updating it to HTML 5, mobile friendly & responsive design & implemented HTTPS. I hadn't been adding new pages / content, though.

When did your Google traffic start declining? My Google traffic started taking a hit around March 9, and there was apparently a "Core Algorithm Update" on March 7:

https://www.seroundtable.com/google-confirms-core-update-253...

[1] https://search.google.com/search-console

It has been a few years now that AntiPolygraph.org's Google ranking on the search term "polygraph" would vary between the first, second, and third page of results. We noticed the complete delisting on "polygraph" early this year. As observed in the United States and several other countries, a Google search on "polygraph" will return no results at all from AntiPolygraph.org, whether or the 1st page or the 30th.

Hmm, I tried from Australia, and your home page did show up on a search for "polygraph" on Google, but you're result #164. There were other sites opposed to polygraphs ahead of you - Polygraph.com was on page 7, while there was a YouTube video at the top of page 2, and some NY Times & Smithsonian Magazine articles about "how to beat a polygraph test" on Page 3 and Page 8 respectively.

Supposedly the March algorithm update was about query relevance [1], and maybe your site just isn't the most relevant for a very general search like "polygraph". My site used to be on page 2 for "Photoshop Plugins", but after the March update it crashed to page 5 (according to Search Console). Since my site isn't about every Photoshop plugin, just the ones I make myself, perhaps that's fair.

I'd definitely recommend playing with the new Google Search Console Beta - you can do a search on the specific "polygraph" keyword, and it will show you what your site's Google rankings were for that keyword over the last 16 months. Click on Performance in the left menu, set Date to Last 16 Months, then add +New -> Queries -> Query Is Exactly -> polygraph. My guess is you'll see a big drop on March 7 / 9. Also worth checking to see what your Click-Through-Rate is.

[1] https://www.mariehaynes.com/march-9-2018-google-algorithm-up...

Thank you for the time you took to research this question and for sharing your findings and thoughts on this! Yours is the most helpful reply I've received.

If I interpret things correctly, the Google Search Console Beta suggests that the situation is worse than I imagined. I have taken a screen shot that you can view here:

https://antipolygraph.org/graphics/google-search-console-ap-...

As you'll note, over the past 16 months, Google reports 147,000 impressions for the search word "polygraph," with an average position of 8.4, but only 381 total clicks. And the number of impressions and clicks flatlined somewhere around March. What do you make of this?

Yikes, it does appear Google stopped showing your page in most results for that keyword. (Hey, at least they've confirmed it!)

If you now click where it says Average Position, you'll get a purple chart of your average position/ranking for that specific keyword over time. (You can actually click on each of those boxes to turn the graphs on & off.) In my case, some of my keywords have a very obvious & sharp drop in ranking beginning around March 7 / 9. My guess is you'll see something similar.

If I'm right about the March Google update, then there isn't anything wrong with your site. It's just that people aren't clicking on your site when they search for "polygraph" (0.3% CTR is rather low, as you can see compared to your other keywords) or Google feels your page isn't the most relevant to their query. You could try making a page on your site focused specifically on that keyword, but you'd be fighting against Wikipedia & Merriam-Webster for the niche.

The same happened to me, right around the end of February/beginning of March. A website of mine which was consistently in the top 2 results for certain queries for the past 4+ years and is the most relevant result for that query abruptly disappeared entirely. It's got modern HTML/CSS, is responsive, and uses HTTPS. It has frequent activity (it's a community) and so is constantly updated.

Looking at the indexed pages, it seems to be stuck in some sort of strange limbo where half the time the homepage isn't even indexed, then it shows up again, then it's gone, back and forth endlessly for the past few months. It never appears in most Google results anymore unless I get very specific (like searching for the name of the website word for word).

I'm at a loss for what to try next. Traffic has dropped to 1/6th of what it was before.

Your website is the first result for "antipolygraph" and the third for "polygraph countermeasures", so it's a little difficult to take seriously a claim of a targeted "partial delisting" because your website doesn't perform so well for a primary search term (which yields a series of results from sources which are mobile friendly and have better SEO, most of which are critical of polygraphs and several of which also purport to include information on how to defeat them)

See [0]; this is a well researched book with a significant amount of actual evidence about the (non) validity of polygraphy, not a conspiracy theory.

[0]: https://antipolygraph.org/lie-behind-the-lie-detector.pdf

> looks more like a conspiracy blog than well-researched information.

Are you basing this on how truthy the web design looks, or some aspect of the content?

This was purely my first—and very subjective—impression after opening the site. I was reading through the blog since this was posted and I'd say the content is interesting and is far from a conspiracy theory.

Once upon a time, that layout was pretty standard on the web. I feel like there is not a penalty for older sites that don't have the resources to update and move from the 90s/00s.

And something is lost there. Now all our sites are bloated with a shit-ton of Javascript. I turn Javascript off by default and are surprised by how many blank pages I get. (If it's a news site, I use archive.is instead of enabling it). Square Space sites don't load most images without Javascript.

Google has had terrible search quality as of late. I'm finding myself using DuckDuckGo more and more just to find very basic things that used to be easy with Google. This is just in the last couple months or so.

Not sure if that's the case here, but if it is I wouldn't be surprised.

Quality has been going down in the last few years, not months, if you ask me.

I've got a simple (for me) use case for which Google completely fails every time. It's mostly for programmers/sysadmins though. Paste an error message from a daemon, kernel etc in google and see what you get. With a little luck, 95% of the results on the first page won't be related to your problem. I'd say 20% won't be related to the piece of software you're dealing with at all.

This can be sort of mitigated by selecting 'Verbatim', but for some reason you can't turn that on forever, you have to select it on every search.

Add censoring of search results, and Google loses its usefulness completely.

You know you can use quotes to get verbatim per phrase, right..?

Google doesn’t always respect quotes

I dont know too much about search engines, but I think that's a side effect of how they normalize their ngrams

Why would I have to do that? Not to mention that no, I don't think quotes are respected by google any more.

Quotes are indeed still respected by Google: https://support.google.com/websearch/answer/2466433

For example, this search without quotes doesn't mention Dennis Ritchie's NYTimes obituary on the first page of results:

https://www.google.com/search?q=became+more+interested+in+co...

Add quotes and the obituary becomes the first result:

https://www.google.com/search?q=%22became+more+interested+in...

That's not what people mean when they say "not respected by Google".

They're not saying quotes never work, they're saying sometimes you use quotes and google returns a page that doesn't include the quoted word.

I always use quotes when posting error messages into Google as I always get better results that way. Why does their tool work this way? I personally don't care, I'm just trying to use their black box of magic as best I can to get things done.

If you are searching a commandline argument, such as "gcc -g", you are probably not going to find ANYTHING useful at all. Why? Because the minus sign (-) means exclusion! Same thing applies for lots of other symbols that may appear in the output of a computer program. Using quotes largely solve the problem.

This has been going downhill as well, it works maybe half of the time now.

This doesn’t actually work. There is a drop down option under search tools that’s called something like “verbatim search”. I only discovered it recently so I haven’t had the opportunity to test it yet, but presumably it works.

Once upon a time I could find blog posts of people reviewing things, travel, software, equipment etc. Now I just find pages and pages of review aggregators. The advantage of finding a blog was didn't take more than a few clicks to figure out if the blogger and me shared thinking, lifestyles, financial means etc. Now all that is gone. We just have ridiculous aggregation.

The problem is exacerbated by search quality guidelines that need to be followed if you're going to rank for any competitive search term, especially when there are clear incentives for doing so - such as site speed and deploying ssl.

Coming next is the 'insecure site' warning that's likely going to cull many more hobby sites.

> Coming next is the 'insecure site' warning that's likely going to cull many more hobby sites.

Good. Setting up https shouldn't take more than an hour now with LetsEncrypt even if you have no clue about the process.

I can totally understand outdated webdesign, can't understand not having https.

I can, because I just converted a site to https, which had a combination of relative and absolute href and src parameters on it in a confused manner. This was combined with html embedded in table records.

It wasn't a simple search and replace task since several admin functionalities were also included in the site.

Think about all of the sites that have relevant information that aren't selling anything that would love to get things like this fixed, but just can't throw money at the problem.

Before you say oh it shouldn't cost that much. Think about where some of these sites are hosted and for how little. Some people don't even have the means to pay for such a fix, but they have important information to share.

This probably should have been a rule applied to sites that sell things and not sites that just have information.

I am all for making money, but I also feel like doing some charity work for this problem as well. I heard the deadline is the 24th. Anyone else interested in doing this collectively for a couple sites?

> I can, because I just converted a site to https, which had a combination of relative and absolute href and src parameters on it in a confused manner. This was combined with html embedded in table records.

Just add an upgrade-insecure-requests header to your webserver config, boom. No search and replacing needed.

https://www.w3.org/TR/upgrade-insecure-requests/

It's been added for exactly your usecase:

> Most notably, mixed content checking [MIX] has the potential to cause real headache for administrators tasked with moving substantial amounts of legacy content onto HTTPS. In particular, going through old content and rewriting resource URLs manually is a huge undertaking.

I mean, uh sure, I'll volunteer to move your sites to https, but I don't think giving a random dude on the internet root access to fix the webserver config is a good idea ;-)

This site also had mixed content due to it using a forward proxy with ARR in IIS. Since ARR doesn't forward https requests it is truly turning into a mess. There isn't an option to just move it to another webserver as that would be it's own undertaking with the dynamic part of the site being ColdFusion.

> I mean, uh sure, I'll volunteer to move your sites to https, but I don't think giving a random dude on the internet root access to fix the webserver config is a good idea ;-)

In this point yeah I agree you don't want to let just random dudes have root access to a site. On the other hand I run my own legitimate consulting business and if you think about it. Every time I am winning over a new client for all intents and purposes I am just a random dude. :)

You can always just put an nginx in front of it to terminate TLS there. Sounds like a legacy mess nobody intends to maintain anymore anyways. Or hell, cloudflare them. Kinda pointless since you won't have TLS to the backend server, but the easiest "solution". I maintain that's it's possible to inject a header and terminate TLS however messy the system is within an hour.

The world has changed though. Now that blogger is more likely than not an affiliate con man rather than a random enthusiast on software or travel.

Not that review aggregators are the answer, but right now I don’t know that there is one.

Problem is that there's a ton of money in the "review industry," (through referrals) it's very lucrative, so the big ones are spending a lot to rank high.

If it helps, you can click Tools on the right under the search bar on the results page and click Verbatim. I've found that I tend to get better results when I'm looking for something specific.

This three click ritual that I have to do on probably 75% of my searches and 99% of my conscious searches reminds me on the 90s when I would have to inconveniently cntrl+alt+del on windows very frequently.

It’s an example of how google is getting more anti user. They obviously have data on this feature usage. They make a bit of a pain and don’t make it a user preference. I fear this means it’s one of the myriad of little features that developers put in that are eventually de supported.

I imagine the following scenario in a design meeting:

PM: SO THE NEW FEATURE IS AMAZING, WE WILL SHOW USERS THE MOST LIKELY RESULTS THAT THEY WANT TO SEE. AND THIS LEADS TO MORE AD CLICKS.

DEV: BUT WHAT ABOUT WHEN IT DOESNT EVEN HAVE WORDS FROM THE SEARCH?

PM: THATS NOT AN ISSUE, OUR ALGORITHM SHOWS THAT 80% OF OUR USERS CLICK ON THESE RESULTS

DEV: HOW WILL I SEARCH STACKOVERFLOW? HOW ABOUT I ADD AN OPTION TO JUST USE THE EXACT TEXT?

PM: WE HAVE TO FOCUS ON SCOPE, CANT WASTE TIME ON BONUS FEATURES

DEV: ITS NOT A BONUS FEATURE ITS HOW SEARCH WORKS BEFORE YOUR CHANGE. IT TAKES 30 SECONDS.

PM: WELL EXACT SEARCH MAKES US LESS AD REVENUE

DEV: ITS REALLY USEFUL

PM: OK MAKE IT HARD TO GET TO SO USERS DONT GET CONFUSED

DEV: OK, AND ILL MAKE A SETTING SO ITS NOT THAT BIG OF A DEAL

PM: NO SETTING THAT WOULD BE A COMMITMENT TO KEEP THE FEATURE

I dislike this meme of thinking that PMs are all useless and devs are the only sane people in product development.

Could you please not use allcaps for emphasis in HN comments? It's basically yelling.

This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html.

100% for years I would try ddg about once a year and end up going back to Google because it just didn't work for me. About six months ago I tried ddg again and have not gone back. Ddg has gotten better and Google has gotten worse.

Google’s search results would probably be useful to me if I happened to be the most generic, average person in the world. I’m guessing this is an explicit business decision and not inferior technology.

Recently, I searched "united states" and Google picked up the word "us" in a page footer -- "Contact us".

It's not just averaging of results. That's a problem, but the aggressive reinterpretation of terms is insane and makes it impossible to search for some things.

Probably it actually meant "Contact United States".

Google seems to prioritize affiliate network blogs that aggregate basic information on a subject.

This makes it difficult to locate information that exists on a term that isn't just surface level facts you can find by browsing Wikipedia.

I've found the quality to be dropping since they started to treat synonyms too loosely in the last few years. Google can mix two completely unrelated topics due to some word being a synonym for your search keyword... in a completely different context.

I vouched for this story when it was flagged dead and I wanted to note why I did.

I think it's more likely than not that the "innocent" explanation is correct here. But even if that's the case it still says something important about how google works and those things are being discussed in the comments.

Further even if the article is wrong, I don't see how it violates any of the guidelines. I think there's a tendency at HN to be very protective of google. I'm not sure if that's external fans or employees or what. There's nothing wrong with that per se, but I think it should be primarily expressed in comments rather than flagging or downvotes.

>I think there's a tendency at HN to be very protective of google

There's a tendency to be very protective of a wretched business model more than there is of google in itself. It may seem people come to google's defense a little too much on HN but that's only because google is the largest, most important corporation at the front of the business model that matters most to hners: advertisement and abusing user data. How else could any of the funded startups that provide free online services survive if they didn't get their money back through some other means?

There is a vested interest on HN and other similar places on the web to show solidarity toward any individual or corporation whose livelihood depends on this because the second you start questioning companies like Google and Facebook, you start questioning the very livelihood of everyone involved here.

One of the first lessons activists (should) learn is the shortsightedness of judging the content of an individual according to what they do for a living; the means of survival they have trained and adjusted for. In an action, strategy will often overrule this but as a conscious exercised and ideally organized process. This along with overlooking petty symbolic straw mans like using cell phones to organize against cell phone companies is like learning to potty train; critical first steps toward maturity.

In other words, evaluate components in proper and sufficient contexts. Software engineers might benefit from a similar conversation, to open more room for discussion around actual problem solving.

>I think there's a tendency at HN to be very protective of google.

Honestly, that's not my experience at all, at least in 2010s. There's a lot of vocal pro-decentralisation folks here, and HN crowd in general seems to be very aware of Google {ab|mis}using its dominating search share. Prior to 2011 or so, that might have been true.

I saw the site on page 3 after googling for "polygraph"

Although searching for "the lie behind the lie detector" gives the first result as the correct one from anitpolygraph.

Is it possible google didn't really do anything shady?

This is a complete falsehood. There is no mention of "Operation Lie Busters" or any kind of program to suppress information about lie detection circumvention in the linked document. No other sources were given.

The site has plenty of reasons to not rank well in search, and secret government control over search rankings is at the bottom of that list in my view.

Yet when I search for 'anti polygraph' [in uk] guess which site ranks top? Like I would expect.

The site title is 'Learn How to Pass a Polygraph Test' yet the term doesn't appear anywhere on the homepage?

I'd agree with the general sentiment that Google search is getting worse but this isn't a great example.

Adding to this:

- search for [anti inflammatory] shows the site as first result and it's on page two for [polygraph controversy]. At the least, the blog post is misleading.

- as per other comments, the site isn't mobile friendly, which has a huge impact on mobile ranking.

- searching for [polygraph] the first page includes several listings discussing the controversy around polygraph results. The results are reasonably high quality stuff, i.e. valid choice.

- a more valid test search might be [polygraph accuracy] or [polygraph controversy] to focus on the debate and less on testing centers and basic information.

Disclosure: former Google engineer and worked at 3 search engine companies on various search products. I've done SEO professionally outside of Google, with success. I've also seen smart people botch SEO, myself included.

I would never have heard about this website if it hadn't been "censored" by Google.

We can be proud to live in democracies, now I know why they envy us in the non free world! /s

https://en.wikipedia.org/wiki/Streisand_effect ;-)

The Streisand Effect is real and hopefully for AntiPolygraph this just gives them a wider audience.

Maybe Google is not trying to censor them, but just changed their algorithms to not show antipolygraph when searching for polygraph as said by someone in the google forum. Their site #1 result when searching "antipolygraph".

Is it appropriate to rank extremely high for something you're directly opposed to? Consider other polarized topics like abortion - should sites enlightening the user on why it is murder, instead of providing useful information, be presented front and center? I'm fairly sure this has come up as a controversy recently, but can't think of the exact topic.

Certainly Google could decide whether it is appropriate on a case-by-case basis, but at that point it starts to be very explicitly political. Once you've stripped away the fig leaf of neutrality, their monopoly would look outrageous to a significant portion of the population.

> Is it appropriate to rank extremely high for something you're directly opposed to?

I'd say that's an important question, which I would answer with "yes", so long as it's understood that "rank extremely high" doesn't imply "at the exclusion of all others".

> should sites enlightening the user on why it is murder, instead of providing useful information, be presented front and center

Possibly. Though, I fear you may have presented a strawman (while betraying which way you, personally, are "polarized" on abortion).

The question of what is useful information depends entirely on the keywords the user entered. Just the single word "abortion" could easily, reasonably, be understood to mean a request for information of general interest on the topic, including moral implications. On the other hand, it would be a stretch to expect that someone would want, say, instructions on how better to administer sedation during the procedure, absent additional keywords.

Of course, there are going to be searches like "get an abortion" that are far more ambiguous than "get an abortion in my area", but I think it's better for something like a search engine (or government or society for that matter) to err on the side of inclusion than suppression.

> Certainly Google could decide whether it is appropriate on a case-by-case basis, but at that point it starts to be very explicitly political

Indeed, and that's a can of worms they likely don't want or need to open.

> while betraying which way you, personally, are "polarized" on abortion

I'm curious what side you think I fall on, seems to me it could be argued either way. I have no strong opinion on it and picked it purely because it's an evocative example. Perhaps too evocative!

Some other examples might be terrorism, animal rights activism, transgender-as-mental-illness. Including both sides can be legitimately and understandably very upsetting for some people; in other cases we demand equal representation.

Are situations like OP's an inevitable product of the tyranny of an algorithm, one that errs to the safer side of excluding results?

> I'm curious what side you think I fall on

That abortion is not murder, as you made the example of the presentation of such an argument in opposition to the presentation of something "useful". I realize, of course, that the word could have been meant more narrowly (such as "strictly pragmatic"), but that tends be less likely, even on here.

> legitimately and understandably very upsetting for some people; in other cases we demand equal representation. [...] > errs to the safer side of excluding results?

That's the the thing, though, I don't believe in a "right not to be offended" or a "right not to be upset" (legitimately, understandably, or otherwise). As such, I don't believe that these divisive cases ought to be treated any differently than the other cases you mention.

None of this is to say that controversial topics don't attract a distinct problem. They do: trolls (for various definitions of the term). However, I believe that's more an issue with the quality of the content rather than the content itself, since a troll could (sometimes just as easily) take either side.

I do agree with this, but think there's enough societal pressure these days that it's very hard for a for-profit company to do.

I'm not sure being for-profit has that much to do with societal pressure as being ad-revenue-driven.

However, this profit (or revenue) above all else model is something Google's unusual stock voting structure was supposed to address.

1) There's no such thing as "the first page of results on Google." Your results depend on your location, previous searches, etc.

2) This site seems to think this is some mass conspiracy to remove sites critical of polygraphs, yet when I Google "polygraph" the vast majority of results I get are critical pieces. Maybe those sites are just as relevant or better? A lot are stories in major publications, so that doesn't surprise me they rank high.

On 29 June 2018, we posted an question about this on Google's Webmaster Central Help Forum:

https://productforums.google.com/forum/?hl=en#!topic/webmast...

(ack! accidentally submitted early; major edits included with this header)

The hypothesis about how the "anti-" prefix is being parsed is interesting; deploying some type of updated (probably machine learning) ranking algorithm with wild and unpredictable consequences to random subjects of that algorithm is consistent with other environments like YouTube (random & unexplained demonetization, etc).

However, this doesn't change much; the results are the same regardless. Google is responsible for the decision to de-rank antipolygraph either deliberately or indirectly with their design of the ranking algorithm.

If it was due to an algorithm, this is another example of the need for algorithmic transparency. You don't get to abdicate responsibility for decisions simply because an algorithm was used; the choice of algorithm (and the data used as input) is itself a decision.

Are you aware that your website cannot be accessed from many Tor exit nodes without solving a google captcha?

Yes, and we very much regret that. We're on a shared server, and our webhosting company recently added security software called Imunify360 that does this.

I cant load the site at all?

Reply on news.ycombinator.com