marsten

Intel should have spun out their fab in 2009-2010 when the signs were clear: Mobile was taking off, AMD spun out their fab, Intel had missed the boat on mobile CPUs, and Apple had acquired PA Semi and was investing heavily in custom silicon.

High-end fab is a volume game and that was the time frame when Intel was still process competitive and could have competed for Apple's business (and Nvidia's, ...). But that would never happen as a division of Intel, nobody wants to send their designs to a competitor.

Biological weapons are probably the more worrisome case for AI. The equipment is less exotic than for nuclear weapon development, and more obtainable by everyday people.

I attribute the curriculum shift to something slightly different, which is the changing perception of CS as a career.

When I was in college in the late 1980s, CS was not perceived as the moneymaking career it is today. Accordingly the kids who went into CS were typically the nerds and hackers who truly loved the field.

Many kids now perceive CS as a safe, lucrative career option akin to becoming a doctor or lawyer. It attracts many students who are smart but perhaps not as intrinsically excited about the field. The universities adjusted their curricula to what these students care about: Less beautiful theory, and more practical training.

A similar thing happened in statistics. At one time it was hardcore stats nerds. Now "data science" has brought a ton more people into the field and the teaching methods have changed dramatically.

The problem isn't that time blindness is a fake issue.

The problem is that many people incorrectly self-diagnose as suffering from conditions like time blindness. Which they do for a variety of reasons: To externalize accountability for why they're late, to feel special, and so on.

A comparison is the large number of people who claim "gluten sensitivity" and maintain special diets. Now there are serious medical conditions like celiac disease that require one to avoid gluten. But the vast majority of self-diagnosed "gluten sensitives" do not have such conditions. Researchers conclude that for many of them there is no physical basis for their self-diagnosis.

Among other things this phenomenon makes it harder for people with actual conditions to be taken seriously, because there are so many impostors.

> Over the years, the magic was never lost on me. However, I can never see LLMs as more than a "token prediction machine".

The "mere token prediction machine" criticism, like Pearl's "deep learning amounts to just curve fitting", is true but it also misses the point. AI in the end turns a mirror on humanity and will force us to accept that intelligence and consciousness can emerge from some pretty simple building blocks. That in some deep sense, all we are is curve fitting.

It reminds me of the lines from T.S. Eliot, “...And the end of all our exploring, Will be to arrive where we started, And know the place for the first time."

You raise a good point that this isn't a low marginal cost business like software, telecom, or (most of) the web. Efficiency will be a big advantage for companies that can achieve it, in part because it will let them scale to new AI use cases.

With the race to get new models out the door, I doubt any of these companies have done much to optimize cost so far. Google is a partial exception – they began developing the TPU ten years ago and the rest of their infrastructure has been optimized over the years to serve computationally expensive products (search, gmail, youtube, etc.).

Interesting that they converged on a memory/network architecture similar to a rack of GPUs.

- 152 cores per chip, equivalent to ~128 CUDA cores per SM

- per-chip SRAM (20 MB) equivalent to SM high-speed shared memory

- per-board DRAM (96 GB across 48 chips) equivalent to GPU global memory

- boards networked together with something akin to NVLink

I wonder if they use HBM for the DRAM, or do anything like coalescing memory accesses.

A surprising number of 401(k) plans default to a money market fund for invested assets. Imagine retiring after a decades-long career and realizing what could have been.

I've always found it amusing that mathematically it should be the rule of 70, but it's commonly rounded to 72 because the latter has more convenient divisors.

70 is divisible by 1, 2, 5, 7, 10, 14, 35, 70

72 is divisible by 1, 2, 3, 4, 6, 8, 9, 12, 18, 24, 36, 72

The standard answer is that the US government broke up Ma Bell in the early 1980s to encourage competition. Hence the quid pro quo that had existed for decades – you let us have a monopoly, we'll do research in the public interest – vanished.

In short the fact that you can now make free long distance calls is why Bell Labs died.

I wonder if we'll see a similar change now that the DOJ is going after Google for antitrust violations. Google has been very open about publishing many of their innovations (like the Transformer architecture), not to mention DeepMind's work e.g. the protein folding work that got a Nobel prize.

Unfortunately C++ ended up with a set of defaults (i.e., the most ergonomic ways of doing things) that are almost always the least safe. During most of C++'s development, performance was king and so safety became opt-in.

Many of these can't be blamed on C holdover. For example Vector.at(i) versus Vector[i] – most people default to the latter and don't think twice about the safety implications. The irony is that most of the time when people use std::vector, performance is irrelevant and they'd be much better off with a safe default.

Alas, we made our bed and now we have to lie in it.

Length is 100% marketing. There is no commercial market for 15-20 page essays, which would be the ideal length for a lot of business books. (The famous Harvard Business Review cases are typically this long.)

The reason many business books feel fluffy is that these 15-20 pages of solid content are spread over 300 pages to meet the expectations of readers and booksellers.

This would be a planet by anyone's definition, if it's near the predicted 9 times the mass of the Earth.

By comparison the entire Kuiper belt – including Pluto – is estimated to have a total mass of about 10% of Earth's mass.

No wonder Varian is still here after all these years. A prepaid 99 year lease for $41k!

Terman gets a bad rap because his father, Lewis Terman, was a proponent of eugenics. There used to be a Terman middle school in Palo Alto named after Frederick, but the bad association with the Terman name compelled them to rename it. Which is unfortunate because if anyone deserves to have a school named after them in Silicon Valley, Frederick Terman should be high on the list.

I remember that! 237 was a country road through a lot of grassland. The only thing of note in that part of the bay was a few streets of houses in Alviso. Great America was surrounded by empty space when it was built.

I think what may save Google from an Innovator's Dilemma extinction is that none of the AI would-be Google killers (OpenAI etc.) have figured out how to achieve any degree of lock-in. We're in a phase right now where everybody gets excited by the latest model and the switching cost is next to zero. This is very different from the dynamics of, say, Intel missing the boat on mobile CPUs.

I've been wondering for some time what sustainable advantage will end up looking like in AI. The only obvious thing is that whoever invents an AI that can remember who you are and every conversation it's had with you -- that will be a sticky product.

> When I worked there Google routinely funded all kinds of things without even the foggiest idea of whether it was profitable or not.

You're talking about small-money bets. The technical infrastructure group at Google makes a lot of them, to explore options or hedge risks, but they only scale the things that make financial sense. They aren't dumb people after all.

The TPU was a small-money bet for quite a few years until this latest AI boom.

Google was changed by two things, neither of which were much fun. But very understandable.

One was the transition to a mature product line. In the early days it was about how do we do cool new things that will delight users: Gmail, Google Maps (Where 2), YouTube. The focus was on user growth and adoption.

Then growth saturated and the focus turned to profitability: Getting more value out of existing users and defending the business. That shift causes you to think very differently, and it's not as fun.

The second was changing market conditions. The web grew up, tech grew up, and the investment needed to make a competitive product skyrocketed. Google needed more wood behind fewer arrows and that meant reining in all the small teams running around doing kooky things. Again not fun, but understandable.

What do you think made Lenat work so secretively? I suspect his ideas could have advanced a lot quicker if he'd been plugged into the community, sharing and getting feedback. In research it's almost never a good move to pursue a dream in isolation.

Given the secrecy surrounding Cyc we don't know how close or not they got to something AGI-like. Whatever conclusion you draw will be from very little data.

From a practical standpoint Cycorp seems to have pivoted at least 10 years ago into providing knowledge and inference systems for clients. If customers are willing to pay them then perhaps we can conclude there is value in the technology. But again I'm not aware of any public examples to look at.

My own thought process is that it's like Usenet or arXiv: The collective value of Github is so high that even if Microsoft loses interest, someone else will buy it from them or clone the content elsewhere. That said, I do wish there was an alternative that didn't involve a for-profit corporation.

This underscores the difficulty companies have trying to navigate through this. Even if Trump doesn't change his mind tomorrow, as he's liable to, he's only around for another four years and for most companies that's not enough time to justify a supply chain overhaul.

It's a poem about self-deception, and Frost has cleverly extended that theme to encompass the reader.

He clearly wrote it the way he did to invite an easy, comfortable interpretation, one that is blatantly contradicted by the very evidence he presents. He's rubbing our noses in our own gullibility and tendency to deceive ourselves with comfortable lies, just as the narrator has done.

All in simple, direct language. Frost sure was a sneaky one.

For an LLM conscious thought is the tokens in its context window, and subconscious thought is the training embedded in its parameters. A person thinks in a similar manner, with subconscious gut instincts modulated (more or less) by a thin veneer of consciousness.

My shocking 23andMe story:

I reached out to a guy on 23andMe that was a DNA relative (2nd cousin) and said hi. Curious how we were related, I gave him some family names going several generations back, and asked if any of them rang a bell in his family history.

He responded quickly and said no, they did not.

Then a few weeks later I get another response. He said that one of the names had faintly rung a bell, and when he dug into it, it turned out that was the name of his mother's boss for much of her career as a secretary. His heart sank when he realized it. He and his siblings did genetic tests and confirmed that he, the youngest, was only a half-sibling. Both of the parents were deceased so there's no way to know what really happened.

After dropping that bomb in the poor guy's lap I stopped using the DNA Relatives feature.

I got my start climbing at City Rock in the mid 90s when I was in grad school at Berkeley. It was a great place and really defined the model that other climbing gyms copied and built upon.

Peter would also show up at Indian Rock sometimes, a popular outdoor climbing spot in Berkeley. He was an incredibly good climber and a friendly guy to boot. It took me a fair number of years to realize that the outgoing guy at Indian Rock was actually the founder of City Rock!

He's one of those who seems to get a lot of satisfaction from building something cool and inspiring people, without much regard for money and traditional status. We could use more people like that.

> But it shifts the need to memorize undefined behavior with the need to memorize the borrow checker rules.

You have a choice: Fight the borrow checker on the front end until your code compiles, or chase down bugs on the back end from undefined behavior and other safety problems. No single answer there, it depends on what you're doing.

Cognitive load is a big issue. In Rust I find this comes in a couple of phases, the first being figuring out what the borrow checker does and how to appease it. The second is figuring out how to structure your data to avoid overusing clone() and unsafe blocks. It's a bit like learning a functional language, where there's a short-term adaptation to the language, then a longer-term adaptation to how you think about solving problems.

You mention doubly-linked lists, which are a good example of a "non-star" data structure that doesn't nicely fit Rust's model. But: for how many problems is this the ideal data structure, really? They're horrible for memory locality. Those of us who learned from K&R cut our teeth on linked lists but for the last 20 years almost every problem I've seen has a better option.

No court in the land will agree with your interpretation. The first amendment protects speech, but it doesn't grant you the right to publish that speech wherever you want. If it did then Facebook couldn't ban people from its platform, for example.

For the harder Euler problems you usually need a combination of mathematical insight and good coding technique. Each problem typically provides sample values for smaller instances of the problem, which you can confirm with a straightforward solution approach. Then the actual problem will be to calculate f(10^500) or some absurd value that requires insight to make tractable. They are extremely good challenges overall.