Hacker News Reader

Comment by vie00001 | Hacker News Reader

> I'm not sure of how it works (does it subscribed to them all?) but https://archive.ph/ is a good way to see the content in those cases.

I think for search engine crawlers there are versions without a paywall so these articles can get fully indexed. Archive.ph, and similar services, might get the full content this way somehow. But I am just guessing.

View on HN

Yes, it's called dynamic rendering when you serve different content based on the user agent, but it's apparently not recommended anymore.

https://searchengineland.com/google-no-longer-recommends-usi...

archive.ph also uses (donated) logins to archive (paywalled) content, however those accounts do get blocked from time to time.

https://blog.archive.today/post/678202832257794048/why-cant-...

While pretending to be GoogleBot used to get you full articles (or grabbing them from cache) this doesn't seem to be the case for some sites anymore.

They just give the first part of the article without the paywall, as that's usually enough for SEO purposes.

> They just give the first part of the article without the paywall, as that's usually enough for SEO purposes.

Many consumers often wouldn't read more text anyway. About one paragraph might even be too much to fill the modern attention span.

you're spot on, this is exactly how sites like archive.org, archive.ph, or even if you click on "view cached version" on Google get the non-paywalled versions.

Reply on news.ycombinator.com