Google AMP case study: leads dropped by 59%
kinsta.com
kinsta.com
Google does something with a news carousel that loads fast and the content isn't crap. I get why publishers running things like taboola think it's terrible, but as a user interested in reading some quick news without going blind, I like it.
Hacker news is the exception in terms of a quick loading / clean website.
I was a google reader user heavily - so I like the more stripped down view of things even if it's "terrible".
One other note about this study. This company's goal is not to educate but to sign up users. At least for myself, Folks who want / like a stripped down experience may just be less interested in being marketed to.
This, a thousands times. For both Taboola and Outbrain it's clickbaity trash - usually misleading and low quality content. They're designed to "disrupt" so that people click on them and have in the past featured on blatantly false news stories. I'd honestly be ashamed to work at one of these sort of organisations and it further underlines how awful the web experience can be if you're not using an ad blocker.
It's no surprise to me that AMP is popular among users if news sites are not including such garbage in their AMP renditions.
The difference? I have an ad-blocker in Firefox, but not in Google's app.
So Ad-blocking > AMP > Regular (as far as user-experience from a speed perspective). Obviously Google has a lot of incentive to make sure people don't go the ad-blocking route.
Though not sure what is keeping ad-blocking from growing given its superiority - likely just a lack of awareness about the option. If ad-blocking ever did grow massively though, the internet would change in ways we haven't yet seen. Either ad-serving companies would win the arms battle and ad-blockers would become ineffective, or, the web would have to move to a new payment model (Brave-like perhaps).
I'm not entirely sure what the ideal is - likely a model where users can choose between ads or micropayments is the most pragmatic world.
With AMP in particular, there was no obvious link back towards Reddit itself. And with the redesign, the actual discussion was interrupted with sections showing what other posts you could read on the subreddit. Similarly, both lost the essential hierarchy and it was really hard to follow the threads of discussion.
The whole thing was dreadful compared to using the decade old Reddit design. The modern stuff is crap.
That:
(A) gets Google to return desktop links (rather than cruddy mobile links)
(B) gets the site to show the desktop version rather than the cruddy mobile site.
It is more obvious how to do it on Android (google.com in Desktop Mode), but I do it on the iPad at work when needed.
Reddit has some disturbing dark patterns going on.
You can't view a page on their site without being nudged to use their app. And you're asked every single page view. Like take a damn hint please.
I'd be ok with the asking but they use different popups with the "no, go away" answer in different places with different wording.
This is all exasperated by AMP, I have to click the amp banner twice to load the real url, this then prompts to open in reddit is fun. Prompts because I want this to happen only sometimes.
I set a hard timer on the app and focused on surfing Hacker News instead (since I wanted to keep abreast of tech news). I still browse /r/politics for popcorn purposes, but otherwise no other subreddits. To my surprise today I realised I hadn't opened Reddit and didn't even miss it at all.
The only problem I've ever seen is that stories from tomshardware do not load in firefox mobile unless I replace the amp in the URL with a www.
It's like the "grooming" that a pedophile does or the badly spelled message you get from some Nigerian prince or somebody who says you won some lottery that you've never entered or even heard of before.
If you click on Taboolah links and manage to sit through a 45 minute video of somebody droning on and using words they can't pronounce (e.g. "Bih Seph A Nol A" as opposed to "Bis Phenol a") you just might be dumb enough to punch in your credit card number and get an $80 a month supplement that you can never cancel.
If it wasn't so offputting to ordinary people more complaints might go to the FTC and they might have something to fear from the authorities. As it is, public servants are paid too much to spend hours just to find out what the scam is if somebody reports it.
e.g. some of these "stories" appeal specifically to fans of a particular TV show or actor from a specific time range.
They throw spaghetti at the wall, cater to low quality publishers and use your location to clumsily make it seem like the content is relevant to you with the most clickbaity headlines known to man.
But the thing is ... it works. People are clicking on this nonsense and other people are paying for those clicks (and publishers making money in the process).
IDK, just a really weird juxtaposition that stood out to me.
[1] https://www.theawl.com/2015/06/a-complete-taxonomy-of-intern...
You can get a far better outcome by having search results consider site speed as a factor in rankings, which would make all publishers improve their sites overnight. The reason Google doesn't do this is because they also run the biggest ad network in the world which runs 80% of the slow ads you see on websites.
These projects within Google are common, similar to how it ranks down websites forcing web users to download an app, when their own websites do the same. AMP allows Google to keep people on Google domains, with better data collection and more control over the independent ad networks and analytics that can run on AMP pages.
And Microsoft's and Pinterest's and Baidu's and on and on. There are many search engines and link aggregators that preload AMP pages.
You prefer that publishers make versions of their articles solely for Apple and other versions of their articles solely for Facebook? (Each has a proprietary competitor to AMP that actually have the disadvantages you listed.)
It will also deanonymize the user to the publisher. I said AMP validates as safe to preload, not possible to preload.
What I actually prefer is HTML which is universally accessible by any device and service. No reason to have any proprietary formats like AMP or others.
Unless you want pages to load instantly, which is what AMP and its non-web competitors provide.
The problem is the stuff on the page, like ads delivered by Google AdX. Removing or optimizing those will make any page faster. No AMP necessary.
Slow sites are not a technical problem, they are a business and marketing problem.
They load quickly, not instantly. That's the fundamental mistake you have in understanding why people prefer AMP, Facebook instant articles, and Apple News.
AMP has little benefits to anyone but Google and the work it requires only takes away from the effort that could otherwise be spent on making the original website faster.
We work with 1000s of publishers and every major adtech company, and over the years we've built the fastest ad network on the planet using techniques from TCP window sizes to counting bytes in HTTP2 streams. AMP is not necessary for speed, it only requires business prioritization, but you can't fix that with technical standards.
That is leaving aside the fact that nobody is actually investing the time they'd spend in AMP to make their website faster. It's simply not something the really bad publishers (for example, a lot of the news space) care about, for reasons I have never managed to follow.
I also said "especially if the actual text content is available immediately", which it almost always is. It's the other assets on the page that slow down loading. If you check the top 5k sites, none of them load any faster than 1-2 seconds, but many focus on optimizing for the first paint experience while loading the full site content over a few seconds. This is basically what AMP does, but taken to an extreme degree.
There are many other factors like content types, page depth, dwell time, user activity, etc. that all matter, and it really takes 3+ seconds to start seeing significant effects on user sessions and revenue. Publishers don't make performance a priority because of something called yield management and a host of other complex business reasons related to revenue. Making the site more performant is usually not worth the limited dev resources, which again are affected by things like AMP.
AFAIK, there is no special magic to AMP that makes it fast, that can't be accomplished with simple CSS/HTML. It's just an on-rails subset of web technologies
Now, it may be arguable, in this screwed-up world, that the only way to remotely even get publishers to make better websites is something like AMP, because it's "marketable" or "friendly" compared to just saying "fix your slow website", I just wish this wasn't the case. But I believe anything AMP does can be accomplished through non-proprietary means using standard web technologies.
Not to mention, publishers probably only care about AMP because they feel forced to do so (because of carousel visibility, etc).
The reason AMP loads instantly is that it is possible to validate AMP pages as safe to preload, so they are preloaded (when you interact with the carousel). That's why it's instant instead of just fast.
For “leads” to drop that much it must mean they weren’t legitimate, happy leads in the first place. I bet those “leads” came from a shit letter popup or other similar dark pattern, in which case AMP works as designed and benefits the user by shielding them from such garbage.
AMP is bad for the web in the long term. But it is a decent stop-gap solution given that nobody is willing to make their websites bearable without Google’s pressure.
EDIT: I just tried their website again without an ad blocker and sure enough, I was asked to subscribe to their shitletter.
Somehow I don’t think Google wants them to stop showing ads. Everything else is a distraction that ignores the root cause of the customer experience being shitty.
The fact, that there are no AMP pages in Google's search results on Firefox Mobile, was the final reason for me to make the switch. I feel like I am browsing the world wide web again, not just a Google's snapshot of it.
I can’t understand why the hell anyone would want to use it anyway. The experience is terrible.
Hacker News doesn't need it, but they're nearly unique in having written a good website.
eh? You can't hit 'back'?
Search for some newsworthy word, say "trump". Click a carousel story. Swipe right (another hijack), and the page loads a story from a competitor to the publisher of the original loaded AMP page.
Now, hit the back button. One assumes it would go to the original loaded story, right? Nope, it goes back to Google. Surely that wasn't intentional, right?
Feels a lot like the old "digg bar" (https://moz.com/ugc/die-diggbar-die-die-should-you-kill-the-...)
Publishers usually do not make good decisions. That behavior is for users and (IMHO) it's great.
I'd like to get the non-AMP page to e.g. see user comments (if any) which are often omitted from the AMP version, or for posting the non-AMP URL to friends or social media.
Sometimes there is an easy way to go to non-AMP page, depending on how you got there, but most of the time for me there is not. For example, this article just appeared on my Google feed on my phone, with no way to open the non-AMP version: https://www.vice.com/amp/en_uk/article/wda7jm/where-does-he-...
My preferred option would be to have AMP permanently disabled on my phone.
- stop using Google feed. I can't recommend a one-stop replacement, since I use RSS to follow sites I trust to produce high-quality content and HN to discover new stuff.
- stop using other Google products that link to AMP pages. DuckDuckGo is good enough these days.
- use Firefox. That in itself might prevent Google search from showing AMP results, if you can't bear to use DDG. At least someone said so two years ago in this thread on Google's product forums https://productforums.google.com/forum/?_escaped_fragment_=t...
- install https://addons.mozilla.org/en-US/firefox/addon/amp2html/ in Firefox. It still needs to load the AMP page to grab the canonical URL, but then it redirects to the non-AMP version. You can also add an adblocker like uBlock Origin while you're at it. The ability to install addons is what makes Firefox the best mobile browser in my opinion.
I disagree, strongly. I believe Android is really more about keeping you in the Google ecosystem. So is AMP, so is Chrome. I'm an Android user, but let's not pretend they are really about "choice and user empowerment." That's just the sales pitch. Every single thing Google creates is to collect information on/from us, because that's their revenue model. Selling our data. If you disagree, compare the ad revenue from all of them. Who's is miles ahead? Why? How? You're giving them your data. A LOT of it.
But that said, why do so much for open source? Why not keep big things proprietary? Why sell Chromebooks that can be unlocked, put in dev mode, and flashed with FOSS BIOS implementations? Why allow unlocking the bootloader on Nexus and Pixel phones? There's lots of things they do that clearly aren't for the bottom line or just to trap you in their ecosystem (they do offer versions of basically everything for iOS as well, which makes it easier to switch to iPhone even if you are deep in the Google ecosystem).
Some of my argument is emotionally based tho, I will admit. After being trapped in Apple's very proprietary, anti-competitive, you-dont-own-it-we-do system, I find the ability to reclaim some freedom (tho not all, they could do better) very nice.
So I think our only real disagreement is likely regarding motives.
For the low price of working under an open source[1] license, they bought over a decade of support from a large portion of tech industry professionals. Google (along with the other FAANG businesses) was able to simply ask people to supporting their increasingly monopolistic position.
When some of us attempted to warn about the long term consequences of giving Google/etc even more power, we were ignored or accused of being "crazy conspiracy theorists"; with a lot of open projects to use as counterexamples, and not many current, specific examples of Google actually abusing their power, a lot of people simply couldn't believe that Google might be doing anything bad.
Open sourcing anything that overlapped with a competitor's proprietary software was a highly successful strategy to undermine the market and hurt their competitor's revenue. The minimax best move isn't always the move that maximizes your benefit; sometimes it's the move that minimizes your opponent..
[1] Note that while they use FSF/RMS-style free Software, Google generally doesn't license their open source software under the GPL. Their openness has a carefully considered limit.
It gets them more users for their services if people make use of their open-source libraries that happen to be optimized for Google, e.g. by integrating better with Google's cloud offerings.
> Why not keep big things proprietary?
They do keep big things proprietary. If it directly makes money for Google, you're not going to see it's source code, unless you work for Google.
> Why sell Chromebooks that can be unlocked, put in dev mode, and flashed with FOSS BIOS implementations? Why allow unlocking the bootloader on Nexus and Pixel phones?
Most people don't change their OS ever, so it doesn't really matter whether bootloaders are locked or not. Even if more people did it, many of them would still install Google apps, so it would continue not to matter.
> There's lots of things they do that clearly aren't for the bottom line or just to trap you in their ecosystem (they do offer versions of basically everything for iOS as well, which makes it easier to switch to iPhone even if you are deep in the Google ecosystem).
Google is not in the phone business. Their phones are just funnels for their advertising business. If they were in the phone business, offering those indispensable apps to iOS users would decrease their ecosystem lock-in. But because they're not in the phone business, offering apps for iOS increases their ecosystem lock-in, because even going to Apple, you still stay with Google.
Nexus/Pixel phones exist only as funnels into the Android ecosystem. Android exists only as a funnel into default-installed Chrome, Gmail and Google Search. Chrome itself is a funnel for Google Search, while Gmail's purpose is to make people sign into an account for easier tracking. Then there's the general ecosystem pull of being the one-stop shop for a bunch of online services. But in the end it all boils down to ads.
Because the majority of their products are only funnels, they'll happily cooperate with competitors for some of those products, if that helps them keep people in the wider Google ecosystem. So you get Chrome for iOS, Google Search for Firefox and so on.
When a company seems to be hurting their own bottom line by giving away stuff for free, it's usually because the free stuff isn't that important to them and their real bottom line is elsewhere.
Or the default search engine, which is Google. So most Apple users are still using Google, and Google pays a ton for Apple users data. $12B is not a trivial chunk of change. That's more than most companies make, and that's only from search! http://fortune.com/2018/09/29/google-apple-safari-search-eng...
They take it away from you and then you will bend over and kiss Matt's Butt.
I didn't state any preferences, and how would sites track you on pages you don't visit?
As I stated in GGGP, the benefit of AMP is that it allows a user to safely preload pages.
Apple News is entirely different and more comparable to using any other news app (like Flipboard) or an RSS reader rather than a website.
This would leak information to sites that came up in your search results but you didn't visit. Webpackages should make it be possible to safely prerender sites without AMP though: https://wicg.github.io/webpackage/draft-yasskin-webpackage-u...
(Disclosure: I work at Google, on making ads AMP so they don't get to run arbitrary JS anymore)
- A set of enforced good practices that make sites faster and generally more user friendly
- A way for Google to get more control by introducing a semi-proprietary framework
The disturbing thing is that webmasters usually criticize the first aspect. Basically they want to bloat their sites and AMP doesn't let them do it. They don't really care about the second part. They are already using Google analytics, Google ads and optimize for Google search anyways. It means that the ones who can do something want AMP to die for all the wrong reasons.
For that reason, I think that AMP is good (or less bad). Even better would be to do what AMP does but without the Google framework. That would make it even faster than AMP because there won't be any Google bloat.
If I'm reading this correctly they switched from regular pages with CTAs to AMP pages without CTAs. Seemingly this test was designed to fail.
- decrease in average positions in SERPs on mobile - decrease in CTR on mobile - higher number of impressions - slight increase in total clicks
These measures are correlated. You can't just add them all together.
More impressions with the same amount of clicks means CTR (click through rate) will drop. Average position should also drop.
In this case we see a slight increase in total clicks and almost double impressions!
What if adding AMP makes Google show your site to a wider range of audience (more variance on target audience), while assigning your pages a lower weight (average position). You would get this behaviour. Lower average position but more clicks. This could lower your account creation and similar metrics because users are less interested.
What I mean is for some websites, getting more clicks would be great. If your total clicks is the goal, you would like these results (ex: page with ads). For search users, this is not good because we are shown pages that are less relevant.
I would like more explanations on mobile leads, account creation, and newsletter email sign-ups dropped. Why not show plots? Are these compared to total from previous months or percentages? We should also need to be able to compare both websites (mobile vs AMP) to see if there are any major differences.
I am not an SEO expert or web designer but I personally use AMP on my modest website. I don't even have a desktop version. Only AMP. It is simpler that way and took only an afternoon (for a css noob) to switch. What made me switch was their amp-img carousel lightbox. They are simple to use and just work. I tried a plethora of css/js carousel or album viewers and chose AMP. Bonus page speed and better mobile search cards.
The screenshot of rankings from another tool (looks like Accuranker) with a few +1 ranking improvements after disabling AMP also seems insignificant. Often this kind of fluctuation is very normal. Without knowing the baseline level of ranking fluctuation, it’s hard to read too much into this.
This is the danger of analysing ‘totals’ without segmentation to better track incrementality.
Any test needs to be properly controlled to form clear conclusions and I don’t see enough rigor here.
I would however commend the article on its advice to avoid simply ‘disabling’ AMP after using it for a period of time. There is cleanup to be done as the article touches on, and I suspect many may not be aware.
It never looks right. It's always "off" in some weird (and sometimes very obvious) way. Nearly every time I go to the real page on the real website, it looks better and functions better. Since I use an ad blocker, it's never a problem for me that the real site might want to load heavier ads on mobile.
AMP is a plague on the open web. It's offensive, I never request it, and it never solves any problem for me. I'm glad I use DuckDuckGo as my search engine so I never get kicked to AMP pages from search. As a web developer, I've vowed never to implement AMP on any of my client sites and will explain to my clients why if asked (and so far I've never been asked).
Just say no to AMP.
In terms of performance... this is a situation where the AVERAGE AMP site is much faster than the AVERAGE site not using AMP... but that's a rigged game given the volume of non-optimized WordPress junk and god-knows-what coming from the typical advertising server (images, video, scripts).
A well-optimized site run by knowledgeable and balanced adults will see a moderate improvement in speed; this is assuming you leaned out the pages already and kept the advertising cruft on a short leash. You can generally beat AMP's load times with some elbow grease.
wait until they are kicked out from the google cache (see your trusted google search console, best the useable old one).
then remove the references to the AMP page from the non-AMP pages. then delete the AMPs.
usually takes 2 to 3 days for a few thousand pages.
They list a few reasons why they think it didn’t increase, e.g. they’re already fast on mobile. But they don’t explain a 59% drop in leads.
Crucially they note that impressions and clicks actually increased with AMP. That points to an unflattering explanation: the AMP format is blocking techniques they use to turn clicks into leads.
So why don’t they discuss the reasons for the drop? Perhaps because they know those techniques are user hostile.
It already sounds toxic. Instead of loading your page suddenly I load a google wrapper that includes some parts of your page. Doesn't that sound like a red flags for anybody? Does it sound less toxic if I call it "free and open google wrapper build on existing technologies"?
Publishers certainly wouldn't give up the most important bits of their page (the top), and cede left/right swipe hijacking (on carousel loaded pages) if there weren't some perceived benefit.
It used to be the "World Wide Web" but now it is the "Google Platform" and we just live in it. After Google established that useful web search was possible the realized that the #1 competitor for their ads was organic rankings, and if you could really get a #1 ranking by investing in the best content then why would you spend money on ads?
So since then it is has been about deliberately degrading the search results so you learn to look at the ads instead, giving people bad S.E.O. advice, etc.
No, pretty sure I covered both. Maybe you're thinking of one of the replies to my post?
That post does seem to be paying lip service to actual benefits for users (such as speed), but then ... suddenly ignore them in favour of a Trojan horse metaphor that doesn't make much sense to me.
As for the Trojan horse metaphor, it is my opinion that's what AMP is. It's tricking publishers into ceding control of their websites. I suspect Google will, over time, leverage AMP to further march towards a walled garden. It is, though, an opinion. Hard for me to "prove" something I'm not privy to.
User experience is only valid as an argument in the context of that darn, biased "mainstream media" being dismembered by ad blockers, or held for ransom by "Brave".
They sell managed wordpress hosting.
However, their site doesn't appear to have much in terms of collecting leads. No newsletters, no trial accounts, etc. They just have paid plans you can sign up for. I guess the "contact us" page could be considered a lead generator.
So, I'm as confused as you are. Lead generation would be pretty bad if you don't collect leads :)
I’d love a search engine that gives users control over ranking, over whether the site runs ads, whether they are a business, whether they’re SEOd out the ass, by security, to blacklist results, to control how they’re displayed.
But of course none of these help display ads, so they have no value to companies.
What happened to total leads/signups/creations?
You need to understand how html css and js work and not cut and paste megabytes of cruft.
It usually maps to constraints they can't control.
It's not really the web developers who are the issue; they just implement what the people above them in the org chart wants. You could argue that the web developers should push back on it more than they do, but I don't buy that they're not knowledgeable enough to fix it.
That's not to say that there aren't actual technical issues; the recent trend of throwing front-end frameworks on content-heavy pages, making the users' low-powered phones parse and run hundreds of megabytes of JS before the page can be interacted with, is an example.
The worst culprit is not knowing how to use photoshop to optimise images properly.
1 A lot of the front end production staff wont be comfortable using a cli - its designers that should be providing the images not developers.
2 Photoshop in some ways produces better results and is already part of the work flow.
There are a multitude of issues that can be tweaked to speed things up more so than javascript elimination. You can switch to HTTP/2, implement server push, image compression, header compression, compressing HTML/CSS/Javascript downloads, preloading, above-the-fold downloads, and much more. A jQuery plugin user might not know half those things and I only mentioned maybe half the things one should know.
The average JS dev (like average anything dev) may not write performant code, but the functional parts of the website are not where the main problem lies.
As such, sure, it's useful to know the details, but it's usually far from necessary.
One example would be lazy-loading of images. I've worked at multiple agencies that definitely knew how to do this, but didn't, causing in the tens of megabytes of unnecessary data. Even just a small change to the templates + a jQuery plugin could have solved this one issue.
Of course, there are lots of reasons why a lot of this low-hanging fruit is left to rot, but they're not primarily to do with the developer's ability to understand all those details that you describe as necessary.
And yet somehow they decided to not use that knowledge until Google pushed them into corner.
In depth knowledge of the DOM would serve you a lot better than knowing about CPUs. What triggers a repaint, how you can avoid it, and so on.
It's in the second part of your statement where you're completely and totally wrong. Speeding things up on the whole does not require a lot of knowledge, and it does not require understanding how the cpu works at a low level and how javascript is compiled to machine code. Knowing how that stuff works can help you squeeze the last 1% of optimization out of code, but in practice, you get the first 99% of speed from decisions:
- Which frameworks do you use?
- What "add-ons" that are key to business do you end up embedding in the front-end and how much page size and slow down do they add?
- Where are your hot loops? Are you doing anything expensive inside them?
- Do you have things that are synchronous where they could be in parallel?
- How much eye candy are you adding to the page? Does it all need to be there?
- How much CSS are you using? Are you using it in a manner where CSS optimizations can do heavy lifting?
- What does the frontend and the backend API contract look like? Are there places where excess requests are occurring, and could they be rolled up so that there is less waste?
You may have noticed that many of these decisions boil down to architectural concerns as well as product and business level decisions, which are tangentially related to the labor of front-end engineering. I don't want to lob ad-hominems at people, but I find this kind of attitude one of the most tiresome parts about certain parts of the engineering community. There's this haughty, holier than thou mentality that places data structures and algorithms at the very top of engineering skills. There's a giant world out there where those skills are not at the top of the hierarchy, and actually are least useful because any sufficiently advanced development there ends up being commodified and available as open source software or as a paid SaaS (IE AWS).
Based on this short, flippant comment, it's obvious you actually have no idea how to actually speed up a web site because you have no idea about what the actual top ten things are that you'd do to speed it up in any kind of commercial production usage. What's even worse is that instead of trying to figure out something you don't know, you're instead making up an answer that sounds reasonable but is actually completely wrong and something any experienced frontend or full-stack engineer would understand is poor judgment and an ineffective approach. This is an extremely dangerous attitude to allow into an organization, because you end up with a culture where people are focused on one-upping each other and attempting to look elite as opposed to pragmatically arriving at the right approach for the problem, specific to all of its constraints. I've worked on these kinds of teams before, and it ends up being a miserable waste of time for everyone involved, and I've endeavored to work on teams that don't behave this way and to create teams that don't suffer from this.
Any engineer that gave this kind of a response to an interview question of "We've got a web app that's slow on the frontend and exhibiting XYZ symptoms -- how would you determine the root cause and diagnose it" would be rejected on the spot by me. That's the kind of attitude that can cause engineering organizations millions of dollars a year in engineering resource misallocation.
As a professional community, we need to evolve away from attitudes like yours. They symbolize an idealized, fictional world that is anything but the pragmatic reality of what good software engineering is.