A domain move disaster | Hacker News Reader

A domain move disaster

paulingraham.com

143 pointslucabenazzi91 comments

Guess what? It shouldn't be so hard period and none of the things that the OP had to do that were mentioned should even need to be done at all. Google should offer a way to make a changes like this similar to the way that you can change an account name or a username at other businesses. If they have to charge a fee for this, fine. If they want to put in conditions so it can't be abused or gamed, fine. But all of this dancing on eggshells in order to please a company that has (for these types of things) no accountability is over the top. I am surprised that nobody has done any kind of class action lawsuit against google for some of their arbitrary and difficult practices or tried to get lawmakers involved (as much as I hate both options sometimes you have to pull the trigger...)

Edit: And you shouldn't need hours and days of test prep, anxiety or needing to hire SEO experts to pull it off either. Ridiculous.

Providing a paid service to help people retain a high search position is basically selling ads, which Google definitely does.

This is very obviously a faulty analogy. Ads help you rank for specific pages with very high ROI per paid click.

Transferring a domain is preserving rank for all pages including those with (comparatively) low value per user.

Google webmaster tools are awful. They treat the regular domain and www as well as https / http as separate, so every website shows up as four sites.

That is 100% intentional. Those could all technically be served by different machines and controlled by different people. It's an abundance of caution in making sure that you can only change settings and view stats for things you've proven you have control over.

I've proven I have control over all of them. I could also happily tell Google they're the same if it can't work this out for itself.

As a user, I've always found it incredibly confusing, unclear, and frustrating.

... to say nothing of the duplicate content penalty for having the same content x4. Even if there's no penalty, there's no shortage of unproductive anxiety over it.

when you change your name, you lose a lot of brand recognition, search engine or no. If ford decided to call themselves "Johnson" next week, next time you wanted to buy a truck, you'd have no idea who these Johnson people were, right? (I mean, barring some huge advertising campaign, or you being the sort of person who follows Ford in the news.)

Yes, there are automated ways around that, if everyone depends on the same search engine for brand recognition, but... I guess I fail to see why the search engine would really care that much. Yes, yes, I know that as a brand owner you spend years trying to game a search engine for your brand, and that has huge value for you, the brand owner.

However, supporting that activity doesn't make the search engine better for the users or for the advertisers, the two groups that a search engine cares about. a search engine has some interest in a user getting a result that isn't too far off from what they want, you know, so said user doesn't leave for another search engine. They have a conflicting (and much stronger) interest in the user getting a paid ad that is similar enough to what they want (and possibly better enough than the organic results) that they might click on it and not be unhappy, because that's how search engines make money. (there's a conflict because the user wants the best result possible, while the search engine wants the organic results to be bad enough that the paid results are relevant enough to warrant a click. Of course, but not so bad that the user goes to a different search engine.)

A search engine has no interest in helping businesses preserve their "organic" search rating. Either enough other people talk enough about the business that the search engine thinks that the link is worth something, or they don't. If you have a good reputation, and then you change your name and nobody knows who you are, well, that sounds like a pretty bad idea, doesn't it? and while yes, it is a problem that the search engine could solve, there's no reason for them, really, to do so.

> I am surprised that nobody has done any kind of class action lawsuit against google for some of their arbitrary and difficult practices or tried to get lawmakers involved

Watch out, or you end up with cookie laws (and Google's fucking ridiculous implementation of that) and right to be forgotten (which is based in sound principles of European right to privacy and control over your data, but ends up with a flawed sub-optimal ruling).

Did anyone else click on this thinking it was at paulgraham.com?

I did

Guilty. Took me a few searches to realize what was up

Yup. Finished the article before I realized the design was all wrong, (figured Paul Graham had a guest poster or something).

I also came here expecting to see this as the top comment and also to mention what exactly I use Youtube comments for.

Yep. Me too. Thought it was a guest post. Got half way before saying "Wait..."

I noticed the difference but thought that maybe it was a parody, like Paul In Graham...I honestly did not expect to see "Paul Ingraham", even though that's not a wildly rare surname.

Reminds me of the "Michelin Guides" and "Michel In Guides" [hilarious] fiasco. [1]

[1] http://www.eater.com/2013/2/7/6483963/michelin-guide-obvious...

Similarly, Mike Rowe Soft.

My first thought was that it was good to see that PG had finally updated his website to have a decent design.

Then I realized it wasn't PG's site.

Considering the subject matter, I thought PG was just letting us know he had made some changes.

Related or not, this is certainly against the "rules": https://www.painscience.com/donate-by-linking.php

This could certainly be critical. Just a theory: Perhaps moving domains triggered a "recalculation" of the link values to the site, and he fell due to violations like this?

Or, more likely, the domain move triggered a manual review and somebody at Google didn't like this page.

I worked in SEO and talked to a guy who manually checked a lot of websites for google. This was a few years ago, but he implied that there was a small army of people manually checking websites.

"Google's manual for its unseen humans who rate the web" -- http://www.theregister.co.uk/2012/11/27/google_raters_manual...

this is a different team than the people that manually review for penalties.

Why is that against the rules? Asking for links is not against the rules.

Asking for links just to boost search engine ranking is at least frowned upon, if not "against the rules". See this[1] for an extreme example of how seriously Google takes this.

[1] http://searchenginewatch.com/sew/news/2321516/rap-genius-no-...

Here's a summary of Google's position on the subject: http://searchengineland.com/google-its-unnatural-to-even-ask...

It's not very clear what sort of asking is permitted, but I'm pretty sure that the sort of asking that Paul Ingraham is doing is not ok. He's specifically asking people to create links in a way that is unnatural.

Sure google hate it as it games the system. You pay for links and your site rises up the natural search. This eats into their AdWords service and pollutes natural search.

But big websites that aren't ad driven use paid links to fund the site. We were paying big sites (think three letter domain names, top five newspaper sites, etc) for links. Say a paper writes an article on car insurance - wouldn't it be nice if they linked to our customers site and 'forgot' to mention a big competitor. Money will make that happen.

So how can google punish these big sites? They are put in an impossible situation.

And he did it again at the end of this article.

Google certainly should penalize spammy, unnatural links ... and any attempt to get them. On the other hand, I hope it’s not against the rules to encourage relevant and earnest links. I should take the page down if there’s any doubt. Meanwhile, I’ve added a very strong warning that links should be editorially relevant and sincere.

In any case, the “donate by linking” page is quite new. The domain move disaster was months old by the time I first published it.

Update: I have now stripped out any possible connotation of soliciting links, in an (over?) abundance of caution. Instead of “Donate by Linking,” the page is now just “Linking Information” and provides only factual information about what constitutes a correct and relevant link to PainScience.com. I believe the original page encouraged only sincere, natural linking well within the spirit of Google’s rules, but with these changes I hope there’s no longer any possibility of misinterpretation.

I don't see a problem with this, he says link only if you mean it.

This is one of the reasons why I would probably never, ever, ever try build a business around any kind of content marketing (as controversial as that sounds.) I know that content marketing is all the rage at the moment and few books would advise against doing it for customer acquisition: but there is always that hidden, implicit assumption with this technique that the organic traffic you build up belongs to you (which is provably false.) It's quite deceptive to think of organic traffic as an asset when Google can take it away at any time (and often without even realizing.) Instead: I would much rather use explicit paid channels and relationships with advertisers then depend on the good graces of Google for my conversions.

I hope that OPs business can recover in time and thanks for posting this. You will probably save a lot of people thousands of dollars in the long run.

I think you need to make a distinction between "a business around content marketing" and having content marketing be one of the many marketing channels for gaining users.

Even if you entirely set aside the hope that you'll eventually gain some organic search traffic, content marketing in the business and startup space still typically has a return 10x of what paid does since it tends to be so much more self selecting, shared on social, etc.

Is content marketing by definition designed for organic search?

This is a lesson for everyone: despite how someone gets to your website the first time, you are in charge of bringing them back.

For many people that means aggressive email capture.

Of your site is not providing any sustainable value to users, then don't be surprised.

"If you have a blog, you can improve my chances by linking to PainScience.com."

Google doesn't take well to people asking for links to their site. Asking for links, buying for links, or spamming your link is something that Google doesn't like. Just from this, I can already see your expertise in SEO. Although having this post at the top of Hacker News is going to help this a bit.

Anyone with basic SEO knowledge could have told you the pitfalls of transferring domains. This reads closer to trying to blame Google for your lack of traffic, rather than what you learned from the experience. From my experience, Google's algorithm is finicky, but if your site is sound and has good content you have little to fear.

> Anyone with basic SEO knowledge could have told you the pitfalls of transferring domains

nope. A lot more people have "basic" SEO knowledge than could execute a seamless TLD transition.

I think we would want to redefine the term basic SEO knowledge here. When I say basic, I mean an SEO coordinator could tell you what's wrong. From reading the article, the user clearly doesn't have basic SEO knowledge.

Knowing what a 301 redirect hardly counts as basic SEO knowledge. Having access to your webmaster tools doesn't count as basic SEO knowledge. My view of basic may be flawed, as I do work in the industry, but I can say that running around complaining about how it's Google's fault and asking for "link donations" isn't going to get you your results.

I agree, asking for links in that context was unwise: I was focussed on the wrong thing. I’ve now removed that. (Note that I also nofollow most links between my sites. I do not use one domain to promote another.)

I don’t know if Google should be blamed or not.

Your contribution to this thread was to mock the author's efforts and then to make some claim about what "basic" SEO knowledge is. HN is a place where experts, apparently like yourself, come to share their expertise. I hope you'll consider that in your future comments.

What's not captured in this writeup is SERP rank (if for equivalent terms the new site is ranking better or worse than previously), it's only showing the resulting traffic.

I wonder what affect people seeing the domain "PainScience" instead of "SaveYourself" in the results caused.

However, it's a near certainty that you're going to lose rank and traffic with a domain name move. Estimates vary but 10-20% isn't an unreasonable guesstimate even if everything goes right.

I can also confirm the deep weirdness of GWT and how it treats sites (seriously, it considered http://domain.com and https://domain.com as two entirely separate sites).

But http://domain.com and https://domain.com are two entirely different sites aren't they? They could serve totally unrelated content.

Technologically, yes. But a well-behaved site shouldn't do that; it breaks the expectations of both people and tools. Either serve the same content, or (even better) serve a redirect.

I can equally imagine someone being irate if Google treated them the same. Might as well at least be technically correct if you're always going to annoy half of the people either way.

You could also serve entirely different sites to chrome and to firefox.

I'm sure people try doing this with Googlebot... serving different content to it than to real "users". I wonder if Google tries to detect this by spoofing its user agent sometimes.

I suspect it does, all the time, since it would get gamed frequently with fake filler content.

It's really fascinating all of the problems they must have to juggle in order to make their shit work. :D

In an extremely pedantic sense yes I suppose they could, but in practice this is so far on the fringes of things as to be a non-issue.

If you want to get really nitpicky about it you could restrict it to only cases where HSTS headers(of which Google + Chrome really took the lead with) are present.

See, now this was my first thought - if I were googling for back pain relief, I would be way more willing to click saveyourself.com than I would be to click painscience.com. One of those is clearly a help page, one of them seems a lot more like a collection of research - and I honestly would really rather help than research, on this topic.

Rank losses were pretty much exactly proportionate to traffic losses. And much, much worse than 10-20%!

This guy actually has some of the highest quality writing I've ever read on a subject. I used it to help my back pain couple of years ago.

If there is anyone at Google reading this, give this guy a manual boost or something, you won't find better quality info anywhere.

> By early 2015, I had an overwhelming impression that Google is a much sloppier technology than I ever imagined.

After working there, I can definitely agree with this. Their engineering chops are pretty high but not only are there tons of bugs with their products (though complexity can cause that), but they also have very poor documentation for everything, so if you run into an awesome bug, your likeliness for finding an obvious solution without having to pnig the team responsible is about as reliable as a coin toss.

If I was to try to take steps to try to avoid this the next time around, it sounds it would be, if moving an HTTP site to a new domain using HTTPS:

  # move to new URL but keep using HTTP 
  # wait for search rank to reach parity, maybe leave it for some buffer
  # upgrade site to use HTTPS

This sounds like it might prevent the issues with migration in GWT.

We saw a similar traffic dip moving openlistings.co -> openlistings.com. Really thought that the domain mover tool in google webmaster tools would have taken care of it but we were unpleasantly surprised that it didn't. Luckily for us, we pulled the band-aid off pretty early and only saw like a 2 week dip before we were beating the previous numbers.

Slightly unrelated but on the topic of SEO in JS SPAs, Google did a really good job with it's latest iteration of the JS crawler [0]. When we made the switch from sending pre-rendered HTML to treating the crawler like a regular user, our rankings actually improved because initial pageload time went from a few seconds to sub 300ms.

[0]: http://googlewebmastercentral.blogspot.com/2015/10/deprecati...

No idea if this is the cause, and I agree the process shouldn't be this opaque, but "pain relief" is a pretty big category of spam. It's possible that those keywords in the domain are detrimental.

Guys, thanks for your comments. I've submitted the link as I've been outraged by this story. I find Paul's website to be a goldmine of carefully selected, beautifully written information that can save people lots of pain (well, literally...). It's hard to say what the solution to cases like this should be, but I believe until the technology gets easier to handle, there should be some sort of regulations, or companies should take the initiative of handling such situations if it's their responsibility. Search engines can't be like black boxes, there's people whose business relies on them and like someone said, I'd expect a level of automation that makes it easy to deal with such scenarios. A content creator shouldn't even worry about all this stuff at all! We are talking about a simple domain migration, that's it. The "Donate by linking" page was published only a week ago, so even though it's a good point trying to address that as well, it can't be the reason for the disaster.

TLDR: Never change a url, ever

Hasn't Google evolved from the original backrub algo? More implicit links than anchor tags, with user personalized results. Wouldn't local browser history get clobbered? My browser autofills URLs; typing in "n" autocompletes with "http://news.ycombinator.com"

Or maybe it is the brand? PainScience is a rather negative in that it focuses on the problem, whereas SaveYourself focuses on the solution.

I observed similar experience making similar move for my blog (idreamcatcher.com -> presentlove.com)

The reason I think is there is some sort of "grandfathered" organic ranking component that belongs to old domain but is not being transferred to new domain, no matter what recommended practices are followed and no matter what Google says.

It's this same grandfathered organic ranking that motivates domain squatters to jump on expired domains.

Exactly.

Apparently, "domain age is an insignificant factor that really carries very little weight in the Google algorithm" --Matt Cutts (ref: http://rapidwebseo.com/matt-cutts-does-domain-age-really-mat...)

Well actual reality and Matt Cutt's words are contradictory.

Besides, "insignificant" != 0

What's "insignificant" for Matt Cutts and Google could be quite significant for someone's family.

I've read similar horror stories. Everyone should be aware of this penalties and the numerous steps that need to be taken:

https://moz.com/blog/seo-guide-how-to-properly-move-domains

The main issue seems to be that Google Webmaster Tools isn't handling HTTPS yet.

Major shortcoming on Google's part, but why not add HTTP support to the new domain and see if that helps? (based on the article it didn't sound like it had been tried)

Why would you make a change like that in November/December, assuming you're selling something which is popular with consumers (or with calendar-year focused businesses with budgets to spend)?

My business is quite oblivious to the season. But, if anything, December sales have always been on the slow side: people seem to think less about self-help info for pain problems at this time of year.

I've seen Black hat SEO tips saying you should set up a network of sites, one with fake/paid links to it, the second a redirect of the first, and some other stuff I don't recall right now. The point was that a ban of the domain would let you change to a new domain and keep all the paid links, and it would somehow fool Google's algorithm.

Just saying, there are legitimate reasons Google may have decided to penalize domain names. People do abuse them.

Why not just keep both domains?

The new site would be marked as duplicate content and the original would rank higher.

What if you returned a 301 to the new URL? You'd think Google would have logic to get that kind of hint.

That's probably what he did - it's the best practise, I believe.

I think they do. If you setup 301 wildcard redirects, google should replace the old site's urls with the new one in their index over time, after they've been crawled. (at least from what I remember when I was digging into it myself)

The old domain (saveyourself.ca) mentioned in the article does do this.

That was the whole point of the article ;) how that didn't work and how it fucked his site.

Yea but can't you use canonical tags on the original site so Google's index is updated with the new URL, thus avoiding duplicate content problems?

This was my thought, as well. It seems like a safer, softer way to test out the transition.

So no redirects for anyone going to an old URL?

I would definitely set up 301 redirects to get the user to the new domain but from for concerns purely about duplicate content, the canonical tags should be sufficient enough to prevent page rank damage.

You can actually mark one of the URLs as canonical if you want to keep both sites up.

Because Google does not like duplicate content.

absolutely everyone clicks thinking paulgraham.com

So I have to ask... why not try to revert after 90% drop? It can hardly get any worse.

I came close to reverting! But I was assured by my consultant that it was temporary, and I was strongly committed to the brand upgrade. And... boiling frog. The 90% losses phase was relatively brief. When things started to improve a bit, I assumed they would continue to improve. By the time it was clear that it wasn’t going to keep improving, it was already late January, and it was getting a lot harder to turn back: lots of energy already put into promoting the new brand, convincing publishers to update their links, etc. “It's a trap!“

I just sincerely wanted to stay with the new brand, even if it was painful in the short term, and I never really dreamed I’d be sitting here a year later and still no recovery.

Makes me wonder why they moved the domain/rebranded in the first place. SaveYourself.ca feels a lot more catchy than PainScience.ca.

Opinions differ on that! It was thoroughly hashed out and tested. “SaveYourself” had over-promising and vague religious connotations that made many people uncomfortable (me included). PainScience.com was more appealing to the right audience. More about the rationale here: https://www.painscience.com/about-moving-to-painscience.php

Good experience to learn from, thanks for sharing

Reply on news.ycombinator.com