akhayam

Finally… took a while.

Now CNCF needs to figure out how to get Istio to work nicely with the networking k8s addons

Let me send a PR to the project with this addition.

The e2e argument was key to internet design. But we regularly (and actively) violated it in design of largescale systems at AWS infrastructure--most of the times with great results. The key difference here is that the cloud owns many (but not all) pieces in the e2e systems puzzle. Do we need another paper that rewrites the e2e argument in light of cloud infrastructures?

Would someone like to work with me on writing it?

Can we commit other people's blogs to this?

E.g. I am surprised to not see Julie Zhou's blog in this list--which, I believe, is one of the most influential blogs in the tech product space: https://www.juliezhuo.com/.

From someone who has dabbled in information theory (the real one), I am just as confused as you. What I have observed in the past decade is that calling things “information theory of something” makes it somehow more palatable for a broader audience.

I wonder how this synchrony can be used for efficient coding. I love and hate pair programming for its efficacy and inefficiency respectively. Is there a way to make it more efficient, as it’s already infinitely more enjoyable?

Would love to see a study around this aspect.

Retrospective from the OG authors of the seminal 1993 old internet traffic modeling paper.

Every picture in this article is worth a million words. Well done!

I also used a lot of pictures when teaching Discrete Cosine Transform in a grad course back in the day:

http://rmarsh.cs.und.edu/CLASS/CS446/DiscreteCosineTransform...

Students loved it and could instantly relate to the theory because of these pictures. Embarrassingly, this continues to be my most cited publication to date :)

I feel it's all rooted in bad leadership. Leaders who don't understand who's actually doing the work hire more folks like them and then there is a chain reaction. Unfortunate, but true, part of most big companies.

Let me provide an adjacent perspective: I started my career as an academic and excellent writing was survival in that jungle. While that's a profession where research and writing can actually "pay your bills" (metaphorically), it resembled Liu Cixin's dark forest:

“The (academic) universe is a dark forest. Every civilization (professor / grad student) is an armed hunter stalking through the trees like a ghost, gently pushing aside branches that block the path and trying to tread without sound. Even breathing is done with care."

Profs and students would poach ideas, submit before the research was even complete blatantly violate double-blind reviews, and submit to multiple venues in parallel (a huge no-no in academic publishing).

Arvix is the only silver lining on that cloud: https://arxiv.org/.

Nicely written.

Before taxonomies and remediation, I would watch Tim Urban's absolutely legendary TedX talk that humanizes procrastination--despite all its ills--and how to empathize with procrastinators (inside and around you):

https://www.youtube.com/watch?v=arj7oStGLkU&pp=ygUacHJvY3Jhc...

Indeed, Hamming and Golay codes are an absolute thing of beauty. For folks that wanna read more about Hamming Bound, Kevin Boone does a good job in starting from the basics and explaining the intuition behind the proof:

https://kevinboone.me/hamming_bound.html?i=1

What is often overlooked is the philanthropic and entrepreneurial impact that Andrew Viterbi had in this world.

In 2004, he donated $52 million to University of Southern California (USC) School of Engineering:

https://en.wikipedia.org/wiki/USC_Viterbi_School_of_Engineer...

He was also the co-founder of Qualcomm with a market cap of ~$115B today:

https://www.google.com/finance/quote/QCOM:NASDAQ

https://en.wikipedia.org/wiki/Qualcomm

Seems like a lot of folks have tried starting from the theory and getting stuck in their tracks.

Would anyone be interested in building a low-code implementation of Viterbi algorithm? I am happy to collaborate on it as long as we commit to releasing it in open-source.

Which I must sadly agree with :(

With internet censorship being used blatantly in Pakistan (yet again), I was reminded of a research paper that I wrote on this topic with some peers and students in 2014--a time when bad technology was impacting day-to-day lives of 191M+ internet-connected people in Pakistan. It seems like not much has changed since then.

Great to see someone writing this as a structured thought. Thank you!

I would add one thing that someone recently said to me and I felt it was the core reason of why I write blog-style docs (for my team or the general internet populace): "These are notes to myself so I can structure learnings and not repeat my own mistakes."

Similar to your thought on "Distilling information" but a bit of a nuance that I think is worth highlighting.

For folks that are interested in learning more about bitwise operations, here is another article (slightly more nuanced) about XORs that you may enjoy reading:

All About XOR: https://accu.org/journals/overload/20/109/lewin_1915/

I have also been using BBEdit for a decade now, but didn't know that it had such a history behind it.

Long live BBEdit + Vim...

Just going through this goldmine and honored to be reading implementations of IP in Mesa using Proc[]'s.

My favorite book on this topic is The Dream Machine which tells how Licklider believed in the "Intergalactic Network" and then worked through all the bureaucracy to make it a reality.

* https://www.amazon.com/Dream-Machine-M-Mitchell-Waldrop/dp/1...

Org will be orgs… and I find it hard to disagree with you :(

The only counterexample to Conway that I have seen working is “away teams”:

https://pedrodelgallego.github.io/blog/amazon/operating-mode...

My world started with LaTeX so I'll support an editor with the audacious goal of making editing "fun". Wasn't aware of mmm, but it looks and "feels" great.

PS: I am still in love with LaTeX--one of the only strongly-typed editors still standing the test of time.

Werner's advice in my own words:

1. Don't pick an architecture because it's all the rage right now.

2. Don't pick an architecture that mimics your org's structure. Aka don't fall prey to Conway's law: https://en.wikipedia.org/wiki/Conway%27s_law.

3. Don't pick an architecture that your team can't operationalize--e.g. due to lack of skills or due to business constraints.

100% agree. Jack Kilby's Masters thesis that realized an actual integrated circuit was another one that makes my list of GOAT.

I would read "Lifespan" by David Sinclair, who used Shannon's information theory to model epigenetic changes that cause us to age. Fascinating read and goes to show how far reaching impacts a foundational mathematical theory can have.

The real moats in this field will come from the hardware industry. It's way too expensive to train these models on general purpose compute. Vertically designed silicon that brings down the unit economics of training and inference workloads are already being designed, in industry and in academia.

Since we are on the topic, does anyone know what Shannon was up to after disrupting 3 fields? ... He got into ball juggling and built the mathematical theory that led to creation of an automated juggling machine: https://cs.stanford.edu/people/eroberts/courses/soco/project....

He actually showed that ball juggling is one of the most complex cognitive tasks that a human can undertake: "The physical constraints that affect mastery and limit the number of objects juggled arise from gravity - more specifically, Newtonian mechanics (h=1/2gt2). Each ball must be thrown sufficiently high to allow the juggler time to deal with the other balls. The need for either speed or height increases rapidly with the number of objects juggled."

Yes, yes. How did I forget this one? Thanks for calling it out.

Wish we had more MS theses today that at least attempted something as ambitious as this one, instead of just checking a box :(

It's incredible how Shannon published "A Mathematical Theory of Communication", while actually working on the "Communication Theory of Secrecy Systems":

https://pages.cs.wisc.edu/~rist/642-spring-2014/shannon-secr...

https://pure.mpg.de/rest/items/item_2383164/component/file_2...

What an absolute genius who disrupted two fields in two years.

I was exposed to Shannon's channel capacity theorem of a binary erasure channel in a grad comms theory course, and totally fell in love with the beauty and elegance of his mathematical formulations. Since that day, teaching Information Theory has been my most favorite hobby--which also used to a career. I love to see students' eyes pop when you eventually show how elegantly simple it is to express disorder (entropy) and to find fundamental limits on source and channel codes using this expression.