Hacker News Reader

Comment by ajays | Hacker News Reader

The figure I've heard is that the data generated doubles every year (here, "data" can mean web pages, logs, transactions, etc.) . Therefore, it follows that every year we create as much data as in all the previous years combined ( sum_i 2^i = 2^(i+1) ).

If we created X amount of data in 2003, then, 7 years later, we're creating 128X as much data; which roughly works out to X every 3 days.

View on HN

It's still BS. A consumer level analog camera captures around 1GB of data with each picture. The peak of film sales was in 1999 when 800 million rolls of film were sold and 25 billion images were captured and printed which works out to around 25 EB of data just from analog cameras in 1999.

Your example also neatly illustrates that not all data is equally meaningful. You lose a lot of information when you do JPEG or MP3 compression, but it doesn't feel like you're losing much of value.

Until you try to enlarge that JPEG later.

"Doubles every year" is just a simple way of summarizing geometric growth to the layman. I'm sure info generation doubles every t, but the chance of t being very close to one year is quite low.

Other, less dramatic kinds of growth can appear exponential in its early stages. I'm always amazed at Internet growth data that are only fit to an exponential -- "at this rate, our startup is going to take over the universe in five years!"

Has everyone forgotten about logistic growth? There is probably a ceiling to most growth, wouldn't it make more sense to ask where it lies? http://en.wikipedia.org/wiki/Logistic_function

I agree with you in general, but you picked a funny example to make this case - generation of information doesn't seem like something which will level off after a while.

The rate at which it is being generated might.

Edit: if you think of the logistic function as modeling the rate we create information, this is one possible "story": initially there's slow growth due to technology. Then the technology picks up, and there's the exponential phase we're seeing right now. That settles into a nice linear trend as the tech matures. Finally, we hit either natural (e.g., satiation) or technological limitations and that slows the rate of growth. At the very limit, we're still creating information, but at a constant rate. The rate of growth might be near 0, but the rate of production is still bloody high.

Sorry, that is what I (obviously?) meant. It would have been trivial to state that information will continue to be generated.

But the doubling every year claim really only makes sense over a short period of history too. Starting from the article's 27 exabytes every 7 days in 2010, tracing back in time (halving each year), you get down to about 1 byte of information being produced in 1940.

Forecasting these kinds of relationships back to the dawn of time seems a bit tricky to me - I would guess we're really seeing something like an S curve.

But it is also worth pointing out that data != information. The human eye captures more data than a fly eye, but both we and the flies manage to get the same information out (i.e. avoid predators, find food, shelter, mates, etc.) and both survive.

Reply on news.ycombinator.com