Claude Shannon: Tinkerer, Prankster, and Father of Information Theory | Hacker News Reader

Claude Shannon: Tinkerer, Prankster, and Father of Information Theory

spectrum.ieee.org

166 pointsfauria29 comments

Those interested in reading a more narrative-style recounting of Mr. Shannon's efforts ought pick up a copy of "The Information: A History, A Theory, A Flood" by James Gleick. It is one of the most fantastic books I have ever read. Informative, entertaining, engrossing. From Babbage's learning machines through the present, I can think of no better introduction to these critical concepts.

I recommend reading Gleick's book in conjunction with Simon Winchester's "The Professor and the Madman" and Walter Isaacson's "The Innovators". Gleick's work is deeper than the other two, but the three books combine to form a fascinating picture of how humans have shaped - and been shaped by - information theory over the last ~150 years.

Gleick's book is one of my favorite books, period.

A good followup would be Jon Gertner's The Idea Factory: Bell Labs and the Great Age of American Innovation. A great read and has a chapter or two devoted to Shannon.

Gleick's book encouraged me to read Shannon's The Mathematical Theory of Communication.

Shannon's conversational tone in the first half is amazing. This is a primary work, laying out a novel idea, but it reads as if it were the most self-evident thing there is, and you're just being nudged along a little bit. Was this the norm in the 40s for technical papers, or was Shannon just a great writer? (The second half delves deeper in the calculus behind it, which I found difficult).

Anyway, The Information is a great read indeed, but Shannon's original is surprisingly engaging and accessible.

No, Shannon was just a great writer. For instance, Turing's writings haven't aged as well.

Economically, can such a life exist anymore? Two masters degrees, a pure R&D job at Bell Labs, earned enough for a nice house in Cambridge, MA, to raise a family. Are there any young men (or women) on a similar path? And if so, what are the organizations involved? (The closest analog would be the startup scene, but the primary "R&D" for startups is usually either done or just needing to be tweaked for market fit.)

I'd imagine the closest modern analogue to Bell Labs would be Google X. (Or whatever they're calling it now after the Alphabet reorg.) They have a lot of the same hallmarks - huge company in a backbone industry that can afford to pay top dollar for the smartest folks they can find. From what I understand about their internal cultures, it sounds like there's plenty of similarities there as well - loose oversight, just the directive to make Cool Stuff.

This is one of the things that really concerns me! It's really no longer possible for one person to work a "normal" job and afford a house/condo in Boston/Cambridge/Somerville, etc. Now it takes two doctors living together to get anywhere.

>“I’ve always pursued my interests without much regard for final value or value to the world,” he said cheerfully. “I’ve spent lots of time on totally useless things.”

>“A lot of practical people around the labs thought it was an interesting theory but not very useful,”

I think that it's good to pursue your interests, no matter how trivial (so long as they don't harm others, I suppose); however, it helps to work in a field paying you to do just that. Bell Labs was notorious for having a laissez-faire approach to managing its researchers.

Now we have to resort to spending company time posting on news sites and browsing wikipedia.

Amongst other "useless" pursuits, Shannon built several machines to play board games, justified as part of his interest in artificial intelligence. See a quick summary here: https://boardgamegeek.com/geeklist/143233/claude-shannon-man...

More than a few of these are on display at the MIT museum in Cambridge, MA. Very awesome. Shannon made more intricate things as toys for his children than many of the things I've made professionally.

Out of the various pieces published on the centennial in Time, NYT, etc. this is by far the best. It goes into much more detail, personal and technical, and includes wonderful photos

The Transoceanic crypto system Shannon created so Churchill & Roosevelt could conference call was brilliant.

It was a sonic one-time pad, L.P.s were sent to Britain and America.

They were played over the conversation, simply adding noise - precisely calibrated by Shannon's signal to noise ratio formula.

The noise was subtracted at the other end, but any eavesdroppers on the transatlantic cable would have been unable to listen without the L.P.s and circuitry.

So while Enigma was being cracked at Bletchly Park, Churchill and Roosevent could chat on the phone, provable securely.

Anyone tried a demo of this one with (say) Audacity generated noise sample and a few words mixed well down then 'decoded'?

Would be fun to see how far you could degrade the encoded voice recording before decoding became problematic.

That sounds worth a go, although Shannon's system was analogue rather than digital which might have made a difference to the point where signal is unrecoverable from noise.

Then there is the noise from the transatlantic cable.

"Then there is the noise from the transatlantic cable."

Sure would be, and then there would be severe bandwidth limitations, frequency dependent phase delays and so on. I can remember transatlantic cable telephone calls in 1960s/1970s (I was a small boy at the time) and they sounded like the Daleks.

That is the interesting bit: US President says stuff into microphone and we have

(President + Local Recorded Noise) convolved with Transatlantic cable = Input

Then we have Churchill in his Siren Suit with a brandy or three playing his noise LP (I gather that they were 16" acetates at 78rpm)

Input - Local Recorded Noise = something like President.

I'll have a play over the weekend: just how bad can the transatlantic cable get before you lose the signal?

I have generally found that one of the best ways to make young engineers and scientists appreciate the importance and difficulty of interpersonal communication is to show them some of Claude Shannon (and other Bell Lab'ers) work on information theory and communication research.

We think about Shannon as the architect of many many advances in technology but his impact has been much broader. This includes a huge amount liberal arts focused work in the field of communications.

That and he has a doctorate in juggling...

I didn't know Mr Information Theory had Alzheimer's.

Man, the universe has a cruel sense of irony sometimes.

I've attempted to learn more about Shannon's battle with Alzheimers, but found little information on the subject. Are there any interesting readings on this, perhaps in his biography?

I just saw a footnote saying that he died at 83 after a long battle with Alzheimer's. Unfortunately that's as far as I looked.

(I find the notion of losing my marbles or my vision terrifying, so I usually run away from such discussions).

Shannon is one of those names, like Riemann or Boltzmann, that I keep running across in quantum mechanics. This was a very interesting article to read.

Didn't know Shannon was a figurehead in QM, but then again I hardly know any quantum mechanics - mind summarizing his contributions to the field for a lay person.

Might have to do with the role of information in QM.

Makes sense, I thought it was something completely unrelated to information theory.

Entropy and information theory in general as discovered was Shannon has been generalized to the quantum case and a lot of the mathematical tools Shannon created are used in quantum information theory (QIT is a big part of QM). QIT is even often called quantum shannon theory, but I'm not sure Shannon dealt with anything quantum per se.

A great book on Shannon's work and other things is "Fortune's Formula: The Untold Story of the Scientific Betting System That Beat the Casinos and Wall Street". Easy and entertaining to read, as are all William Poundstone books.

http://www.amazon.com/Fortunes-Formula-Scientific-Betting-Ca...

I wonder if there is any correlation between genius and a sense of humor.

I'd love a biography of Shannon!

Reply on news.ycombinator.com