What to do when your site is "too large" for Google Analytics?
webmasters.stackexchange.com
webmasters.stackexchange.com
According to the terms of use I'm familiar with, linking an AdWords account to your GA account, and spending money on it (even nominal sums like a buck fifty) has been sufficient to be exempt from any hit limits on GA. Your AdWords account does need to be explicitly linked to the GA account, and you do need to be spending some non-zero amount of money, but they're easy to connect and that's honestly still quite cheap for an analytics implementation. Omniture SiteCatalyst will run you in the six figures per year, minimum. It's possible that Google has changed its policy, as Google has a reputation for being somewhat fickle as an analytics provider, but I haven't heard anything via official channels.
That aside, only having your data update once a day should not be a problem. There is surprisingly little value in data updating multiple times per day. There's no reason to have data update more frequently than you can take action upon it, and most sites of most companies move at the speed of a glacier. There are a few business models where it is beneficial, for example news sites can benefit from knowing what stories are really popular right this minute. Besides, if you need data more often than once a day, you shouldn't be on Google Analytics anyways. Average data delay is a few hours, but it's not unheard of for data to be stale by a day or two.
Apparently you need an adwords account 'in good standing', I'm not sure what they mean by that because we do have an adwords account, and as far as I know and can see it is in 'good standing', only we're not advertising any more using google because we found that it isn't making enough money in return.
See:
http://news.ycombinator.com/item?id=1593898
I'd pay for analytics, it's a very useful service and I can't see the adwords angle, after all with something as vague as 'good standing' as the deciding factor how about simply charging for analytics?
I always figured that analytics was free because it provides google with useful data as well.
Maybe it's time to get my old stats suite dusted off and re-installed, I don't like operating against the TOS even if it is apparently something they'll let you get away with.
I wouldn't worry about this too much. It doesn't cut you off, just limits it to updating once per day (basically just a day behind). Its possible that your business/company is dependent on information from hourly reporting in realtime, and if that's the case you should probably augment GA with some other paid service or roll your own BI internally.
I see adwords, adsense and analytics (as well as search) as unrelated offerings by google.
Link directly to their help center: http://www.google.com/support/analytics/bin/answer.py?hl=en&...
I told her that I blocked google analytics because I didn't want it to find out things about me for advertising etc. She responded saying that Google doesn't do that, but now that I think about it she was probably saying that Google doesn't profile me specifically, nothing about anonymised data... yeah, now I'm not so sure. :)
I wonder if I'm alone in that perception or if this is more common.
You could just take how much you'd pay for analytics and buy some ads. :-)
Sampling your data by visitor/cookie (not by page view) is the number one thing to do that that no one does. Just sample your data down by a factor of 10 or 100. The costs drop by a factor of 10 or 100, query speeds improve, and the business value of the data drops by very, very little.
The only unfortunate prerequisite for this approach is that you need a company/boss that understands that sampled numbers are OK even if they are not exact. (So, explain, if you must, that your industry insider source tells you that the notion of 'exact' in web analytics world is very loose indeed.)
http://code.google.com/apis/analytics/docs/gaJS/gaJSApiBasic...
A couple of solutions I'm aware of:
1) Omniture Site Catalyst: A lot of big sites use this but it will cost you dearly. My site runs about 85mm pvs/month and they quoted me close to a million dollars a year at those traffic levels! I didn't even bother negotiating it down, even the negotiated price would still have been way more than I'm willing to pay.
2) Urchin: I don't know how similar/dissimilar Urchin is from Analytics, but the Urchin 6 software runs about $3k. This option could prove labor intensive since you would effectively be running and logging your own metrics on your own server. So the actual costs could go up (wrt to time, equipment, etc.)
There are plenty of smaller players in this space, but the problem that always comes up is, "Will these guys be around in a year?". When looking for an business solution, it's nice to know that your data won't up and disappear should the company go bankrupt.
I do wish Google would provide a paid service for those of us that could use it and would pay for it. I'm already using Google Doubleclick for Publishers for ad serving (which costs money), you'd think they'd also want to expand their other complementary offerings (like metrics).
On that theme does anybody know what the maximum limit of money is before your bank account crashes? (or you have to start buying hotels instead of houses)? :-)
Avoid Omniture SC. It has a terrible interface, is too expensive, harder than GA to implement, and Omniture doesn't seem to be improving it much, if at all anymore.
More seriously, what Google Analytics does that you can't do yourself when you have complete access to the logs of your web site?
Things like unique visitor, unique referer, unique whatever tracking just explode when you have to compare each IP address you get with 20M-entry list.
If you try to do it on your webserver, you'll have a 3 hour window each day where a significant portion of your CPU and Memory is not available for serving web traffic. If you try to do it elsewhere, you get to ship 20gb daily logfiles around.
It's just not any fun. Which is why even Google has difficulty with it.
Running your own analytics is great for relatively small sites. For big ones it starts to bog down once you hit a certain size.
That, and the fact that I prefer to process my logs on my computers. As a matter of principle (decentralization, privacy, freedom, blah blah).
Now, I barely know Analytics. But it looks cool, so I'll check, and see if I can reap most of their benefits myself. (If I can't, I'll have a tough choice indeed.)
(Something about that bar not being updated until the first byte of the stream was read -- so if a static.foo.com was hanging connections, it would get stuck blaming whatever loaded right before it.)
GA now has asynchronous tracking though, which solves this pretty nicely. Though I'm not a JS guy, my understanding of it is that instead of sourcing the script and then issuing the tracking calls, you put some instructions into a global variable, and then source the script. Nothing blocks on the script's loading, so even if GA is down the rest of the page is supposed to load correctly. Once the GA script loads, it looks at the global and runs the tracking commands.
But still, I would have liked the analysis process to be completely invisible from the client.
Now I recognize that it is a complex question: If Google Analytics has some advantages that you can't reproduce even when you fully control your web server, I would like to know what it is. If it does not, then we may want to stop sending all our logs to Google right now.
(Also, doing it on a pageview basis instead of visitor basis means you lose all the nice visit path tracking Analytics does)
Another commenter on this topic has a good post about convincing management of the validity of sampled metrics, I'd read it then tip your chair back and stare at the ceiling for a while.