Hacker News Reader

Comment by __all__ | Hacker News Reader

How the author knows its website has hundreds of subscribers? AFAIK is not possible to identify subscribers to RSS feeds and counting hits won't help. Am I missing something here?

View on HN

Hi! Some feed aggregators include the subscriber count in the User-Agent header, so I can pick these counts from the access logs and add them up. This is how the logs look:

  [14/Jun/2024:00:03:46 +0000] "GET /feed.xml HTTP/1.1" 304 0 "-" "Feedly/1.0 (+http://www.feedly.com/fetcher.html; 87 subscribers; )"
  [14/Jun/2024:00:06:29 +0000] "GET /feed.xml HTTP/1.1" 304 0 "-" "Feedly/1.0 (+http://www.feedly.com/fetcher.html; 34 subscribers; )"
  [14/Jun/2024:00:09:31 +0000] "GET /feed.xml HTTP/1.1" 304 0 "-" "Feedly/1.0 (+http://www.feedly.com/fetcher.html; 31 subscribers; )"
  [14/Jun/2024:00:16:36 +0000] "GET /feed.xml HTTP/1.1" 304 0 "-" "Feedbin feed-id:2815708 - 3 subscribers"
  [14/Jun/2024:00:29:42 +0000] "GET /feed.xml HTTP/1.1" 304 0 "-" "Feedbin feed-id:2330316 - 3 subscribers"
  [14/Jun/2024:00:40:58 +0000] "GET /feed.xml HTTP/1.1" 304 0 "-" "Feedbin feed-id:1714691 - 8 subscribers"
  [14/Jun/2024:01:21:01 +0000] "GET /feed.xml HTTP/1.1" 200 188077 "-" "Mozilla/5.0 (compatible; BazQux/2.4; +https://bazqux.com/fetcher; 5 subscribers)"
  [14/Jun/2024:01:44:21 +0000] "GET /feed.xml HTTP/1.1" 304 0 "https://susam.net/" "Inoreader/1.0 (+http://www.inoreader.com/feed-fetcher; 24 subscribers; )"

Picking a few days of logs where the subscriber count has not changed much, I get a rough estimate of the total count of subscribers reported by the feed readers like this:

  $ for i in 1 2 3 4 5; do echo $(head -n 1 access.log.$i | grep -o '../.../....') $(awk -F'"' '{print $6}' access.log.$i | sort -u | grep -o '[0-9]* subscribers' | awk '{s += $1} END {print s}'); done 
  13/Jun/2024 335
  12/Jun/2024 335
  11/Jun/2024 336
  10/Jun/2024 334
  09/Jun/2024 337

In case anyone is wondering why we see multiple entries for Feedly and Feedbin in the first log snippet, that's because in an older design of my website, I had multiple sections each serving its own feed at paths like /blog/feed.xml, /maze/feed.xml, etc. Later I consolidated all of them into a unified feed at /feed.xml. So the feed readers still hit the old feed URLs and then get redirected to the unified feed URL.

You can get a rough estimate based on unique IPs hitting the RSS feed. Moreover, some of the online feed readers report the number of subscribers of your feed as part of their User-Agent. An example from my blog logs: `"Feedbin feed-id:2688376 - 9 subscribers"`

Some sites like Feedly and others show the subscriber count in the User-Agent string.

Interesting, didn't know. Will check that!

In my own logs, the ones that show are Feedly, Inoreader, Newsblur, Feedbin, The Old Reader, and a few small/personal ones.

Of course, they only show the subscriber count for their own platform. And then you can also pool together all the separate requests fetching /feed/ and add it all up.

Reply on news.ycombinator.com