Log parsing seems like the logical choice for the static site crowd but it seems like there's little interest there. I must be missing something.
Any site is constantly being accessed by bots, only some of whom announce themselves in the user agent. Some are deliberately designed to mimic human browsing and you can only tell by carefully following their access pattern.
Filtering out the bots from the logs in an automatic fashion seemed like too much work so I implemented a javascript beacon. I figure anyone without JS probably doesn't want to be tracked anyway.
We host websites and the bots are super annoying, because even the well-behaved ones throttle requests per domain, which means they just hit all of our customers at once. If our cache architecture were a little more rotten, like I’ve seen on other jobs, then bot-driven evictions would get ugly, instead of just spiking our traffic, increasing our overhead, and making it harder to get clear metrics.
others have pointed out:
- Client side SPAs sometimes don't hit server logs
- Some static sites are hosted places where you don't have access (github pages, netlify, etc)
- Bots are sometimes defeated by a simple js file
But another one that is not mentioned is one that effects large apps and services. Many large apps and services don't exist on a single server. Furthermore servers are launched and destroyed on a whim to meet scalability needs. Javascript analytics easily surmount this. I suppose it is still possible to feed multiple server logs into a single source of truth for analytics, but I dont know if such a solution exists right now. JS Analytics easily overcome this obstacle.