Hacker News Reader

Hacker News

Comment by noblethrasher | Hacker News Reader

Full thread

noblethrasher

I posted this spreadsheet in another thread to give an idea of the cost of storing all of the nation's calls.

I figured an annual cost of about $11 million.

https://docs.google.com/spreadsheet/ccc?key=0AqWtA_3af-R0dE5...

(the spreadsheet allows editing, so you all can play with the numbers).

View on HN

I'm pretty sure 96,000 bits/s vastly overstates the required storage. Common VoIP codecs uses as little as 5,300 bits/s[1] and military communication systems typically only require 2,400 bits/s and can go as low as 600 bits/sec[2]. Note: this is before all the transport overhead.

And this is for live communications. Recorded voice communication can be compressed further. For example the popular G.729 VoIP codec compresses down to something like 1,000 bits/sec in archives from it's normal 8,000 bits/s required during a live call[3]. Although lower bandwidth codecs probably don't compress as much.

Please correct me if I am wrong - not an expert.

[1] http://www.cisco.com/c/en/us/support/docs/voice/voice-qualit...

[2] http://en.wikipedia.org/wiki/FNBDT#Voice_compression_using_V...

[3] http://call-recording.tmcnet.com/topics/call-recording/artic...

Agreed. I didn't have any reliable numbers at the time so I overestimated in favor of making it more expensive.

Brewster Kahle of archive.org ran the numbers last year as well:

https://blog.archive.org/2013/06/15/cost-to-store-all-us-pho...

https://docs.google.com/spreadsheet/ccc?key=0AuqlWHQKlooOdGJ...

That's remarkably cheap.

What does it cost to transcribe All VOICE calls ->TEXT for faster analysis, apply DEEP learning to it, and apply a graph of Everyone bigger than Facebook to these patterns?

I was just talking about the cost of storage, not acquisition or analysis.

Just going by energy need I back-of-the-envelope estimated the Bluffdale center could run voice recognition on all U.S. voice calls, yes. I'd guess it's not the best use of that much power, currently, but it'll only get easier.

The voice->text is inaccurate enough (especially because the many different languages that can be used) so you have to keep the original recording and not just the text; but yes, you can easily do things like "find all conversations where the word 'nakamoto' is heard" with mass scale voice/speech analysis.

Reply on news.ycombinator.com