Too Much SEO? Google’s Working On An “Over-Optimization” Penalty For That

This can't come soon enough.

Too many sites are ranking by stuffing keywords and buying links.

Next, Google need to find a way to rank a page based on it's role in a broader user experience - rather than being measured in isolation.

Sites (especially in retail) are forced to optimise a page in a specific way to get indexed successfully. The result is everyone putting too much content on a single page - detracting from usability and creating homogeneity for users - like cars all optimised in wind tunnels.

Disclaimer: I work for Google, but not on search. These opinions are my own and based only on the linked article.

I took a beating on the SEO blogs for calling SEO a bug a few months ago, but I'm glad the rest of the world is finally realizing that I'm right :)

Ultimately, Google is trying to rank you highly for providing the best content; you shouldn't be spending your time trying to figure out how to game Google by making superficial changes to the presentation of your content. Want to rank better? Write better!

The whole problem is rooted in the fact that Google is a leaky abstraction. It tries to be an omniscient Sherpa, guiding the wary Web traveler with its infinite understanding of the Web and the individual user's needs. The reality, though, is that Google is actually just a computer program. So there is a gap between the user's mental model of Google and what Google actually does, and it's this gap that SEO exploits for its own profit.

An infinite amount of exploitation would mean that Google would just return results randomly, and so it makes a lot of sense to detect signs of SEO and penalize the behavior before it further broadens the perfection/reality gap. Gaming the system is currently profitable, since the worst thing that can happen to you is nothing, but the best thing is that you get more traffic. A penalty aligns the risk/reward spectrum to favor "write better content" rather than with "spam a bunch of wikis".

I gave up trying to optimize for SEO a long time ago. The future is good content, period. Technology will eventually figure out a way to get the good content in front of users, whether it's through a Google-like search engine or something totally different.

I'd rather focus my energy on driving traffic in other ways, such as traditional media, list building, PR - and focusing on creating great content.

I just do the bare minimum stuff when it comes to SEO (title, h1 tags, some anchor text here and there). Other than that, I don't pay attention to SEO in the actual content of the pages.

The problem is that Google actually aren't good enough at search yet.

They still rely on this dumb word-based approach to document retrieval. Example, if my page is about how to manage your time, and it's a really fantastic resource about that, but I don't actually mention the phrase "how to manage your time" anywhere, I won't rank for people searching for that phrase. I should, but I won't.

So I have to write my content for two audiences - humans and Google.

I don't want to do this. I'd be much happier just building great content for my human readers, and if I happen not to mention the exact keyword phrase a searcher might use, it doesn't matter, Google still knows it should put my site at the top for that phrase, because it understands that my site is about that phrase, even though I don't mention it exactly.

But Google just isn't good enough yet. Someone can set up a page which has a bunch of headers and URLs and variations of the keywords and beat me to #1, even if their content is utter rubbish for human consumption.

That's why SEO still exists. It's symptomatic of a bug in Google.

This penalty will help, hopefully, but we're still going to be in a state where we have to compromise our content to serve two masters.

I wonder if Google is getting trapped near a local peak.

I watched the video they released on improving search quality yesterday (http://insidesearch.blogspot.com/2012/03/video-search-qualit...) and was initially impressed that they gave so much thought to slightly improving a small subset of 0.1% of queries. That meeting must have had 40 people attending!

But afterwards it left me with the feeling that Google is becoming so big and entrenched in old ways of doing things, that they may not be focusing enough on the next big improvement in search. Penalties and heuristics can only go so far -- eventually they'll need something approaching AI -- and if any company can do it it's Google.

I am all for continuing to clean up search.

I'd also like to note that for people creating content on the web, especially programming-type people who deal in logic and certainty, the SEO system resembles something like black magic. It's pretty clear that unless you understand SEO and apply it, you're never going to be seen no matter how good your content is. Now it appears that if you understand it too well that's also a bad thing.

The goal here is to let the users themselves inform the search engine as to what content is good -- hence the plus-ones, social search and all of that stuff. But all of this is still indirect evidence. Unless you could plug a computer inside the head of a person and watch their every thought, the only real data you have for input is server logs, click-throughs, and all kinds of other things that computers do, not people.

I just don't see this being solved any time soon. But I do see it getting so complex and unwieldy that it continues to frustrate searchers and content producers alike. Meanwhile the bad actors will continue to have a heyday.

Wish I could be more optimistic about it.

While they can certainly penalize sites that optimize today with respect to the criteria of yesterday, that effectively just means changing the criteria of ranking. Nothing stops site owners to optimize with respect to the new criteria tomorrow. The full implications of the proposition are reminiscent of Russel's paradox, or worthy to be included in a next edition of Hofstadter's Gödel, Escher, Bach book.

EDIT: small clarification

A little off topic but: the idea of SEO has always bugged me: like a store owner investing in having really clean windows, but crappy products inside the store.

Admittedly, that is easy for me to say because I don't need high traffic at my site. I need my web site to have a few high value visitors: people who want to work with me or communicate because we are into the same technology. The way I "optimize" my site is writing about what most interests me, and that attracts people with my interests. Seems pretty straight forward to me.

I hope they'll be able to do it right. But, as soon as they change their algorithms, SEOs will reverse engineer them again and change the optimizations. There can be no definitive solutions, at least not with current technologies, because SEOs are very smart and very fast at adapting.

As an aside, the first over-optimization that I would target if I were google are keyword-based domain names. Keywords in the domain name are given WAY too much weight; how often do you find that the top three results for the search "XXXX YYY ZZZZZ" to be very shallow but keywords-rich websites with the domain names www.XXXXYYYZZZZZ.com, www.XXXXYYYZZZZZ.org, www.XXXXYYYZZZZZ.net?

I have relatively little hope that it takes longer than a few days/weeks until a wave of seo-deoptimization services hit the SEO market. The incentives for beating "the system" are just too high.

Google has gotten so bad. I'm now finding worthwhile results until the second or third page.

Here is my search process

If it is from ehow ignore

If it has my search term in the URL ignore

If it is from about.com ignore

Unless the penalties from Google are long term I can't see this as being very effective, as you would assume competent SEOs will just tweak their processes in response.

From what I've read over the years, basic SEO mainly boils down to :

- Have good, relevant content

- Choose your page titles and URLs carefully

- Get lots of links to your pages , from as many different domains as possible, and try to get them from high Pagerank domains

- Where possible try to get anchor text relevant to the terms you want to rank for - but don't go overboard with a large percentage being the same phrase, as it appears artificial

- An older site can benefit you, as will exact match domains for the main TLDs

Except for site age, they are all easily changeable by the SEO.

Maybe Pagerank is too easily gameable, and what is old will become new again, and some of the approaches tried in the 90s and replaced by Pagerank will return with a new twist.

Sounds like the equivalent of the AMT (Alternative Minimum Tax, designed to increase the US income tax of those deemed to have too many deductions).

I think Google needs to be really careful with this. It would make sense that the first place they should look for over optimization is anchor text manipulation. The problem is that this could open up the door for "Negative SEO" services where you can blast a competitor with high traffic keyword anchor texts. The safest place to look will be on page factors and if they decide to go after anchor text manipulation there should be some type of authority metric that protects established sites.

Google and Matt Cutts, in general, have a larger problem. Cutts never acknowledges that Google works primarily on the language, which is one of the most complex things the human race has ever come up to. And though language is always at stake in Google search, I don't see any serious semiotic study in whatever they make.

And, apart from that, Google totally sucks on languages that are different than English. They think of a new thing for the English based search and then they'll just propagate it everywhere with no substantial modifications whatsoever.

I've got many sites under me, all writing original content. They're not gonna win the Pulitzer, of course, but at least they're edited, accurately followed by teams of human authors that won't even copy PR as a measure to prevent non original contents to appear on the sites. Although all of these things, we got seriously pandalized.

The results where we once used to stay atop are filled in many occasions by scraper-sites who steal our contents and rank 2 or 3, sometimes up to 5, positions above us on the SERP. I can't even count the times I filled the anti-scraping form anymore. That's not enough, because many times popular sites rank higher than us just because they're pretty popular, although their contents are horribly written, short, totally uninformative.

That happens all the time. You know where we still go pretty strong? On Google News, where the human intervention sometimes really applies.

Google should see what's going on everywhere and their insistence on having matt cutts as its only public voice on this huge issue is becoming really frustrating.

He ends up looking like a fake good guy, perpetrating the hypocrisy of a corporation too big and too convinced they have the ability to solve all the hyper-complex search problems, laden with human generated unpredictability and the natural human tension towards deception, just with pretty algorithms.

For years those penalties have often been mentioned on SEO forums. This is nothing new but just another filter they will add. Or maybe its just a little FUD to keep people nervous?

This is completely the wrong thing to do and in my opinion this means that, Google have essentially just admitted they're bad at search.

Firstly, for ecommerce sites etc due to the lack of content - especially unique content (because there are only so many ways you can say something is X length) these sites are forced to SEO themselves. This is even more apparent as Google are placing their “Google Product Search” within the results as well. Sure, you could add yourself to Google Product Search but that’s not the point - Google should be adding ecommerce sites etc in their automatically to make it a level playing field. Sure, you can argue that it is hard to do and that might be the case but, there is proof out there in the marketplace to highlight that this is possible - look at what TheFind etc are doing in this space. Additionally, I think there is a lot more that can be done in the Shopping Search space as well as other areas of search which I will cover below.

Having covered those, I will also highlight another problem which is hindering Google = their search engine is based around the Pagerank algorithm which despite evolving is actually hurting Google in trying to solve the problem of SEO.

I believe this is the case because of PageRank and the general Google Search algorithms which are in place – their search at its core is based on 'citations' like those in academic papers etc (yes it has evolved over time) but it is still even loosely based and developed upon on this ranking system. Hence there is the problem of paid links - although you can report them [1] this reporting method actually isn't effective and doesn't work. Additionally, there are tons of ways to get links really easily which appear natural, won’t appear overly seo’d and are extremely easy to get and game as well.

Google seems to be taking their search into the, Semantic & Social Search approaches which they will probably solve some of these aspects but, everyone is working out (and many have already worked out) how to game social search although, it is a step in at least a decent direction.

Currently, Google just provides links and documents etc to the ones which it believes is the most useful based on their Pagerank/keyword approach and even if it can improve search to a much greater degree the issue is also related to Adsense/Adwords.

This is because; these two print money for Google. For instance, Adwords really is used for Google Search (yes it does feed into Adsense as well but Advertisers really want their ads on Google Search Results). The last statistic I heard about Google Search, is around 2007 when Google was making $0.12 on average per search – you could probably calculate easily how much Google is making now by looking at their income and dividing it by publicly available search numbers but, I’m going to say it’s more than that because in 2004, they were making $0.09 per search. This is actually an issue for the user because, I believe Google really just want their ad results to be perfect – they don’t really want you to have to click on the result because then Google isn’t making any money – as long as their results are good enough so then they’re Ok with that If you don’t believe me, take a look at [2] that’s a whole lot of ads above the fold. Oh, and if you try and do that be prepared to be punished [3].

Now Adsense, sure it doesn’t make that much of Google’s income but it still is highly profitable – if it wasn’t they wouldn’t be doing it. If Google, really wants to fix search then they need to review every single Adsense site and I mean every single Adsense site since Google only reviews the site which you apply with. This leads of a huge problem as MFA’s aka. Made For Adsense slip through the net afterwards and if you look at their sites, they’re generally have poor content etc and are optimised for the search engine and the user to leave either by clicking the back button. Sure, Google will lose some Adsense income on their balance sheet but if they want to fix search this is also a good place to start.

Sure they're trying to solve this issue and get to grips with it but I don't think they will anywhere in the near future. You can argue this fix they have announced will solve some SEO problems but, as I said in my opening sentence I believe this is the wrong approach and some of the above reasons highlight several issues already but there are tons more. In fact, I actually hope you disagree with me because, not only do I value your opinion but am interested in the other aspects you may add to it.

[1] https://www.google.com/webmasters/tools/paidlinks?pli=1

[2] http://searchengineland.com/figz/wp-content/seloads/2012/01/...

[3] http://searchengineland.com/too-many-ads-above-the-fold-now-...

I think this is going to be very interesting. I can understand the benefit of it, but as an SEO, I also feel that because we are working in the dark it is as much a science as an art. To me, this just means I have to work on fine tuning what it takes. Also, I think it will be harder for those who play on the wrong side of the tracks to rank well, which in turn may make it easier for those who do to rank well.

I am probably misconstruing things, as I often do and being simple-minded on top of that, but this seems like a circle-jerk and makes me wonder what happened to Tim Berners-Lee's musings about the Semantic Web, and why search hasn't progressed in that direction in some exponentially powerful and revolutionary way.

Does anyone else find that announcement weird? "over -SEO", is just a kind of low quality content, so been a Google target for years. This announcement seems like it's just trying to scare off SEO fanatics, or an admission that Google has been using ridiculously naive linear scoring models for keywords.

Well, that's pretty good news... if it works. There's a lot of garbage on the first page just because the page/site has a lot of backlinks and has the whole site SEO'd by the book (titles, keyword density, headers, etc.).

As an SEO I am interested to see how this update goes. Google updates usually have a 'shotgun' result. They might kill the intended targets but there is a lot of damage from the spray to sites that might be unwarranted.

The huge elephant in the room is that Google can kill so many startups and small businesses practically overnight with an algorithm change.

People can say "create quality content, don't worry, you can always file reconsideration request if it's an accident" don't realize how Kafka-like Google really is. If an algorithm penalizes your site by accident, and your traffic drops by 99%.. good luck trying to talk to an actual human being working in Google to get your problem fixed!

I wonder how Wikipedia will be ranked with this change since there are litte to no outbound links.

Is there a SEO industry around Bing? What does Bing SEO look like?

I hope this destroys all the scabby blogs that link everything to more of their own garbage - Mashable, Engadget, The Verge etc. Be nice to see them focus on not being shit instead of being rewarded for it.

This is great news. I've always been surprised how the SEO industry can have such legitimacy when it's basically as legit as false advertisement.

This is the Laffer Curve for search.

Thought 1: This may be overly cynical, but it is hard to believe that this isn't just another way to dump more people into their adwords system.

People who "over optimize" on a large scale are probably the same people making money off of well ranked SERP's, penalizing those people pushes them to buy adwords.

They are in essence pre-qualified customers for adwords since they have already demonstrated a willingness to invest real time/money in ranking. Sure this will just trigger yet another race to optimize optimization, but until things are refigured out more money will be dumped into adwords.

Thought 2: This SERP "Market Volatility" is a great way for Google and SEO's to make a little more money.

This is also a good way of reminding some people how much their revenue is dependent on Google, when you see an overnight dive in revenue it captures the attention/mind share of higher ups who have maybe been taking their well oiled SERP machine for granted.

It's like a one night only "Google Dance" reunion tour, and gets people obsessing over them again.

The Google guys are so smart, with enough time they'll get all their algorithms so optimized they'll eventually exclude every site from ranking on the first page of results. It'll just be a blank white page, the ultimate in minimalism and speed. It'll be an engineering feat not rivaled in the 21st century.

The trouble with SEO as a business is that you are always second-guessing the whims of people who's plans and actions are unknown to you other than the fact that they are almost definitely working against you, and furthermore they will usually have much better resources available.

Hard to believe that they don't already do this...?