Hacker News Reader

Comment by simonw | Hacker News Reader

Does this really matter?

That 100KB feed is served by GitHub Pages, which should be pretty efficient - it gets gzipped and comes with ETags/last-modified headers, so good feed readers should be able to access it efficiently.

As a consumer of Atom feeds I find whole content feeds like that quite convenient, provided they don't get into the thousands of entries.

(Fun aside: I got the new Gemini 2.5 Pro to write me a quick Python script to confirm that GitHub Pages does indeed support gzip and ETag/last-modified: https://gist.github.com/simonw/7f6b167bd8288dccfbebbaf7e143e... )

View on HN

> As a consumer of Atom feeds I find whole content feeds like that quite convenient, provided they don't get into the thousands of entries.

Atom does have an RFC for supporting pagination so it is possible (client support provided) to have both a short feed as well as full content history: https://www.rfc-editor.org/rfc/rfc5005#section-3

For Wordpress sites, `?paged=X` can be used to paginate through RSS feeds, although that's more of a Wordpress standard than an RSS/Atom standard: https://codex.wordpress.org/Pagination

My personal gripe is that clients checking for query params would seem to rule out pagination of static feeds (so you'd end up shipping a single huge response to the client) but now that I look closer, I guess you could achieve it via Atom, you'd just end up having to reshuffle every feed whenever pages get updated.

I went through some of this recently when standing up some home software to generate RSS feeds for sites that don't have them :)

> Does this really matter

Well, there is a matter of efficiency. Even when the feed reader uses the ETags/last-modified checks - which it should, but may not be granted practice after providers like Google ditched the technology (as some were apparently flooding them with "are we there yet" requests) - with an average for this year of 200 entries per XML you would be using 1/200th of each per download - and increasing...

On the average of 20kb of gzipped material per XML (rough average for this year for that feed) it may not seem much - but if you multiply that for all the feeds users register to, inefficiency starts building a volume...

Optimally, I would say, a feed provider should have (1) the "whole collection" XML - "download once to get all the past records in your DB", and (2) a "latest" XML, which should probably better calibrated by time (e.g. all the records of the past 48 hours for a newspaper; all the records for the past month for an every-few-days blog...).

... I decided to give people a choice, so I've updated the script to produce TWO atom feeds now, one of which is just the most recent 20 items:

https://simonw.github.io/ollama-models-atom-feed/atom-recent...

Reply on news.ycombinator.com