Hacker News Reader

Comment by randomstring | Hacker News Reader

The team fails to mention that the deployment of Garmin Connect was over a year late and the migration from motionbased.com (which Garmin bought) was a total disaster.

It looks like the motionbased.com blog and forums have been taken down. That makes it hard for those who didn't go through that hell, like I and many others did, to appreciate just what a total cluster fsck the transition was. The new site, while much prettier with all its shiny web 2.0-ness, is functionally on par with the site it replaced.

Here is a cached forum link talking about how the December 2008 launch is slipping.

http://74.125.95.132/search?q=cache:YnEfNC3lDnwJ:forums.moti...

View on HN

Yeah they definitely lagged from what I can see, but if they are accurate in their statements of 45,000 logged activities a day, that's a TON. To put it in perspective, a single XML log file from my Garmin GPS bike computer is around 200-300kb for a 30-40 mile ride. Imagine that infrastructure buildout!

Not defending a slow release schedule, but it's not "trivial" after working in this industry for the last couple years.

That, and any corporate structure slows a development team down considerably. Nice dig for the cache, thanks for the info :)

This is coming from the confused part of my brain. 45k transactions a day isn't that much and processing 300kb xml files should be trivial. What am I missing? Was it tongue in cheek?

I guess my response was a bit intense. but I will stand by with saying processing 45k 300kb files a day is a non-trivial task. Making the site handle that amount of traffic (serving the pages) is easy. Processing the data however, isn't. Consider it's being done through their proprietary firefox extension that works on two browsers and operating systems in order to pull info off your devices. Then it has to go to some cluster of background job processors. If you think growing your database/storage scheme by 2-5 gigs a day is 'trivial' then I commend you. From my experience, it's an easy recipe for setbacks. For example, our dataset is only 5 gigs, however at 5 gigs we are to the point of multi-hour schema changes in our data storage format. So, say you want to support handling laps from an uploaded logfile. A data migration of how many terabytes spanning how many hours?

It's ok, it seems as if you have real experience doing this, in contrast to the armchair "scaling experts" flooding HN that balk at problems like this.

Noticing this a few days late, but feel the need to defend myself. I've worked for two data mining and data aggregation companies. Currently working for a real time vertical search company. We do 45k in the blink of an eye. real time search over millions of documents is HARD. Granted we have lots of hardware and bandwidth. I'd put 45k at entry level scaling problems and see no reason to brag or get excited over it, which is why I asked for clarification.

Reply on news.ycombinator.com