Robert Fano has died
eecs.mit.edu
eecs.mit.edu
https://archive.org/details/fano-tr65.7z
This was an original conception of information theory, somewhat beat to the press by Shannon, though the two had talked about it (and indeed Shannon cites this in "A Mathematical Theory of Communication").
I have a copy of TR 149, Transmission of Information II, as well, but no scanner anymore.
How delightfully ironic! ;)
Otherwise, the 63 combined pages are available for the low, low price of $275.
There is significant research effort these days on going beyond unbiasedness and asymptotics, which in many cases has been beaten to death and is "classical", though there are still plenty of questions.
Here is a simple illustration of unbiasedness/going beyond it. Consider n iid (independent and identically distributed) coin flips of coin of P(heads) = p, and based on the observations, we want to estimate p, call the estimate p'. Optimum estimate in the sense of minimax - min (over estimators) max (over p in [0, 1]) E[(p-p')^2] is not the simple maximum likelihood which is the sample average. Sample average yields a score of (1/4)(1/n), while optimum is (1/4)(1/(sqrt(n)+1)^2), which is slightly lower. It is natural to wonder why people care, typically the motivation is in high dim statistics where there is dependence on the dimension as well. In such a case fine grained characterization of performance sometimes affects the scaling in terms of the target loss amount and dimension, i.e the asymptotics now has at least 2 parameters, sometimes more.
My personal unhappiness with these problem formulations is their lack of robustness to transformations - suppose I now want to estimate p^3 instead of p, the optimum min-max will change in most cases, and definitely does here. Plugging in the sample average and then cubing is far more intuitive, and often works fine even though it is not optimal. There are more extreme examples of this, but that will take the thread too far afield.
If one is interested in the topic of biased estimators, I suggest reading about James-Stein estimators and more generally shrinkage estimators.
So he was 90 years old when he wrote that. That is really cool, especially in the context of this section of the obituary:
> In many respects, Fano was one of the world’s first open-source advocates. He frequently described computing as a public utility that, like water or electricity, should be accessible to all. His writings in the 1960s often discussed computing’s place in society, and predated today’s debates about the ethical implications of technology.
> “One must consider the security of a system that may hold in its mass memory detailed information on individuals and organizations,” he wrote in a 1966 paper he co-authored with Corbató. “How will access to the utility be controlled? Who will regulate its use?”
It sounds like he was way ahead of his peers in understanding the privacy aspects of future computers.
In those days [batch processing] programmers never even
documented their programs, because it was assumed that
nobody else would ever use them. Now, however, time-sharing
had made exchanging software trivial: you just stored one
copy in the public repository and therby effectively gave it
to the world. Immediately people began to document their
programs and to think of them as being usable by others.
They started to build on each other's work.
Glen Langdon and David Huffman I knew from UC Santa Cruz. Shannon and Fano, I only heard stories about. Jorma Rissanen is still alive at the age of 98.
Pillars in compression and information theory.
One of my favorite memories of David was how he got grumpy about being best known for that term paper. He had done much more.
(Sorry for side track memory)
Who?