Intuitions for Tranformer Circuits

connorjdavis.com

67 pointscjamsonhn5 comments

I was glad to discover this is an analogy to electrical circuits further down the page. It's a good topic to understand, the write-up is comprehensive.

> They are the only man-made technology in history that we don’t fully understand from first principles

…what? What about bicycles? Ice skates? General anesthetics? I feel like there are a ton of commonly-used man-made items we don’t fully understand from a first-principles perspective.

Am I missing something here?

No, I don't think you are missing anything. Only recently have engineers been inventing things from "first principles". I think for the majority of human civilization we've mostly invented and improved through trial and error.

How many computing or mathematical constructs fall into that category? You don't accidentally land on a new algorithm typically.

The author shouldn't have discussed inventions when the nugget is about "original thought".

yea all of modern semiconductors were built on a guy going "wouldnt it be cool if i could write using metal instead of ink"

https://en.wikipedia.org/wiki/Czochralski_method

Reply on news.ycombinator.com