Ask HN: What analytics do you use when Google Analytics reaches its limits? | Hacker News Reader

Ask HN: What analytics do you use when Google Analytics reaches its limits?

9 pointsdumbfounder13 comments

From Google's page on data limits: "Data collection limit: If you exceed 10 million hits per month, there is no assurance that the excess hits will be processed." - We are doing more than a million hits per day, so what now?

Right now Twicsy continues to use GA, and also Quantcast, but their numbers (on page views in particular) have started to diverge. On some days the page view counts are 60-70% different. We are not sure who to trust.

We have tried hosting Piwik ourselves and also a hosting provider, it just stopped working after a few days for some reason in both cases. Clicky doesn't support websites that get more than 1 million page views per day on one website. We can't find anything else appealing.

We don't have a lot of money to spend, at most we would like to spend a few hundred dollars per month. Is there something out there that is worthwhile at that price? If not, what are our alternatives?

My suggestion would be to look into a workaround inside of GA.

Eg: Create a new suite within GA (UA-XXXXX-2), and use the new ID for a subset of your page views. Look into how your page views are happening across your site and plan on distributing the analytics to fall under 10M. For instance, use your main ID for landing, top and trending pages and a second one for the rest of the site. If you are near or over 20M views you might want to consider a third or forth ID.

Good luck!

===

Edit: On a second thought, one of our sites reports over 10M a month and we never run into issues. GA doesn't process more than a small percentage of visits for the reporting, so your issue might not be related to the 10M cap.

Thanks for the suggestion, but I see a few issues with this approach: 1. You can't track the number of unique visitors anymore 2. I will lose the data history I have now with GA. I am actually not sure that it is incorrect, I just am suspect at this point and need validation. If it is indeed correct then I don't want a break in the data.

But, something to think about, so thanks.

Re: Edit: Are you sure? I mean, really sure?

Are you sure it's GA? Our numbers differ between GA and quantcast because google isn't tracking opera mini on feature phones but quantcast is. We are running 450m pageviews a month through ga and its typically a day behind processing. Outside of that its missing feature phones, but I don't believe it's because of our numbers.

Nope, I am not sure, so thanks for the input. The numbers are 60-70% different for us between Quantcast and GA, so I don't think opera mini would make up anywhere near that difference.

And, are you sure it isn't dropping data? I mean, really sure?

Reddit uses GA. There may be some special arrangements you need to make. http://blog.reddit.com/2011/02/reddit-billions-served.html

They'll use the GA Enterprise version which serves up to 1 Billion "hits processed per month" its £90,000[1]/$150,000[2] as a flat fee per year.

[1] http://www.google.co.uk/intl/en_uk/analytics/premium/feature...

[2] http://www.google.co.uk/intl/en_us/analytics/premium/feature...

Server access log analyzers (awstats, webalizer, etc)? Those are free, and get you most of the same basic reports. I don't know of any SaaS analytics that aren't out of your price range at 30M+ data points per month.

http://clicky.com/

http://clicky.com/help/pricing#/compare/

They only allow up to 1 million page views per day per site. We are over that number.

On the register page it says

"If your site has more than 1,000,000 daily page views, please contact us before registering!"

I take that to mean they'll work something out with you, and based on their pricing it should be well within your price range.

Consolidated logs + Splunk.

http://www.splunk.com/

Maybe I am wrong, but I don't think that analyzing logs is anywhere near as effective as the javascript based solutions that use cookies to track users. We receive a lot of bot traffic and I don't think log analyzers do a good job of telling the difference.

It is trivial to filter out the bots IMO. You can also correlate system logs with application logs.

Reply on news.ycombinator.com