mikgp

Couldn’t AI and tech be paying AI and tech? Or I’m not sure how this is calculated - the government? Consumers?

There was a post today or yesterday where Paul Graham and Nate Silver accused the AP of being woke for saying you shouldn’t anthropomorphize LLMs. But I can imagine there to be quite a few unpredictable consequences to encouraging your customers to project emotions onto your product that humans generally project onto each other.

I’m not sure SV is prepared for what happens when you go down that road.

I feel like there’s a weird conflicts in how folks mental model of LLM development.

It’s not the model I don’t trust, it’s myself. The model is wrong _all the time_ because - it’s easy to verify the code - it’s hard to verify that I knew what I was talking about when I prompted it.

So the idea that you can broadly speaking take the human out of the loop. I think suggests to me a level of consistency in the contextual environment that would probably never exist.

At some point it’s politics. The model can come up with a better answer than my boss, and then my boss can just ignore it. Taking the human out of the loop broadly speaking implies that we all agree on what we’re trying to optimize.

I wonder if - there are sort of broadly speaking a few different types of learners.

Jensen Huang has been CEO of NVIDIA for 33 years. That is a role with a very specific type of information environment. He is sort of this weird combination of specialist and - he necessarily has to operate within a certain level of abstraction.

I’m somewhere in the middle, I’m a high achiever but not the highest. I would say I’m above average in my usage of AI at my tech job. The way I’ve learned systems thinking is by being a bit non-specific in what I learn. British history, psychology, software engineering, queuing theory, cooking.

The idea of learning systems but not basic math - the idea of being too discerning in what I’m willing to learn. The entire idea of passing up the ability to learn something like basic math.

So many mental models of the world are developed by engaging with things like basic math. How do you learn systems without learning patterns behind numbers?

If the whole argument is something like it’s now about taste or creativity or being a builder? The way you learn those skills is engagement with all the things. It’s not abandoning all the things to read a book on systems thinking and product management.

It’s not never focus, but if your default position is “maybe I shouldn’t be curious about that”. You’re operating from a deficit.

The future is long.

I get the whole “we don’t have to read the code anymore” Schtick from most product devs, and DHH I guess is at his core a builder / product dev whatever.

But the development of something like rails pushed forward the state of the art in ways that LLMs now replicate.

The point being like - someone has to keep writing programming languages and frameworks and libraries. Like - someone has to be thinking what the code looks like for LLMs to have patterns to match against, right?

I understand this persons frustration, but I feel like…

It I were running YCombinator and I were writing next years request for startups. I would say:

Here are the top twenty household expenses / bill for the average person. Tell us which one you’re going to lower and how.

There are a few variations on questions you ask that get at the same thing, but while I understand the complaint of the poster, I also somewhat understand the dissent.

Why are you selling us this thing that maybe creatively in the right context could have an indirect impact on people’s lives. Solve one of the problems I have in front of me.

It is kinda crazy - I remember Elon musk showing off that you could play video games and watch movies in the Tesla.

And there’s something super dystopian about the idea -

When I was a kid basically four things governed the emotion you felt about a car. What it looked like on the outside, how far the gas pedal was depressed whether or not the window was open and what was playing on the radio.

The experience of driving a car was one of the things we most looked forward to about growing up, and now we’re trying to abstract that away.

And I feel it myself - there is something exciting or stimulating in modern cars about - where the screens are, how big they are, what colors are on them. But how backwards that is to the whole original notion of design.

It is something I often think about in life that, inefficiency is where our humanity is.

I’m curious how Anthropic Pivot to biology will work out. I imagine the real world is much harder to manipulate than numbers/code.

Curious about this lawsuit - I assume that there are legal means to “collude” on safety. Car companies coming together to agree on crash test regulations will “slow down” auto manufacturing but I’m pretty sure this mechanism is legal.

If the four companies came together to start an industry organization and agree on safety protocols, I assume that would be legal done correctly.

An assumption built into RSI is it speeds up iteration cycle right? I clearly don’t know anything about frontier model development, but in theory, you could have naive RSI with the right loop inputs right? But the assumption is RSI makes the next unit of work incrementally faster not slower.

A mental model I was thinking about was - I remember when Travis Kalanick was talking about using the chatbot to discuss “vibe physics-ing” on the all-in podcast.

And like - I think there’s a presumption you could make that AI models could overfit to asymptote towards just the capabilities and knowledge we currently have.

And that would be amazing! And crazy useful. And there are probably a whole world of complex problems that remain unsolved because they’re adjacent to knowledge we have but they haven’t been invested in.

But can a human reliably tell the difference between “can do 99.999% of the things we currently know how to do which includes a small subset of things we didn’t know we had the capacity to do” and “super intelligent math and science research pushing the frontier of what we know”

A physicist that knows all the things we currently know in excruciating detail feels like it should be able to make the leap beyond the frontier.

But since these are computer models it might just be that it can ride that line extraordinarily well while the line remains firm.

The 70’s were arguably more Grim than today in life and film.

None of them! That was my point. High prices is unpopular, and datacenters are unpopular, so I’m trying to figure out why 1 and 3 are written in the negative and 2 is written in the positive. I’m just confused that’s all.

I’m trying to understand your position here. Resistance to datacenters is like wildly bipartisan popular the other two are inversely so. I assume you’re trying to make a satirical point, but what is it?

I’m curious if something akin to the Streisand effect is going to take place here. It’s not like you can stop them from knowing about the rest of the world and it sounds like current efforts are heavy handed.

I think it was Kara Swisher and guest who made the observation that the reasons social media as it currently exists may not be long for this world is that part of the reason these lawsuits and laws to restrict access to children are making traction is because adults realize that social media isn’t good for them either. But restricting access for adults is a longer more difficult process for first amendment reasons. The tobacco lawsuits took decades.

“ I have repeatedly argued for strong antitrust enforcement to keep industries competitive, especially Big Tech.”

Has David Sacks actually called for stronger antitrust enforcement? Not that he has total control of this administration but it’s gone pretty hard in the other direction in my observation.

I think SWE fundamentals matter - because it will be a long time before software is a closed system.

And the problem is that as long as humans are in the loop building software that dynamic will have to be maintained.

We use Loki for logging at work. There’s certain types of queries it just doesn’t support. And so the question becomes -

Do you change logging providers Adapt to Loki’s capabilities Create a third layer / tiered storage.

And each of those decisions have multiple downstream consequences. It’s not that LLMs can’t make those decisions per say, it’s that

What does an LLM do when five different people ask for a system optimized to do five different things.

It could figure it out itself, but like I don’t think that’s how the human software contract works.

“Debuggable? It can add extensive instrumentation in seconds.

None of this requires expertise, prompting, or mention of TDD. It's the default.”

I’m pretty sure it takes some level of expertise just to use the term “instrumentation” correctly in a sentence.

It seems like, particularly in scholarly works, we should cite AI contributions the same way we cite a third party, with similar consequences around plagiarism. If you’re copying AI whole hog, not quoting and citing it, and not catching basic failures. Is it really “your” work?

The point on Radithor seemed suss to me:

“Introduced in 1918, Radithor was William J. A. Bailey's biggest commercial success, selling 400,000 bottles between 1925 and 1930. The Food and Drug Administration (FDA) issued warnings against the use of Radithor but did not have the authority to ban it during this period.”

Many of these examples seem less like settled science and more like a set of fads. But it also seems to leave out all of the fringe things people tried and didn’t work.

While I’m not completely obtuse I don’t actually understand the point this person is trying to make other than knowledge is hard and political.

“1930s: a chemist dissolves a new sulfa drug in diethylene glycol, which is antifreeze. It kills 107 people, mostly children, and only then does America give the FDA power to demand safety testing.”

Is this a pro-institutional screed or an anti-institutional one?

I feel like what’s weird is people don’t really have a solution here. We’re simultaneously criticizing cutting edge science while simultaneously trying to lower the bar around getting things like “peptides” to market.

I don't know if this is sort of a nerds vs normies argument, but sometimes I think we get stuck on AI and not the simpler ideas it represents.

Things are easier to validate in person so there's a sense you understand the context. If you go to a painting club because you want to learn how to paint better, and someone or someone(s) show up with photography instead. You don't hate photographers you're just there to do a specific thing. Or say a club that uses traditional woodworking tools. You may not hate power tools or the people who use them, but they're not invited to the club.

I wonder if all the noise is around a need for better tools. Some level of consternation is good to signal that there is a problem in need of a solution. Hopefully there are solutions to this that aren't too invasive, but I think the counter-argument that it's wrong to have a like preference for human generated content is like - I think it should be ok to want to try and specifically define what a community is for.

On the one hand I want to believe in spec driven development - I think there’s this awkward gap between code and not code that is the cause of the sort of “code review crisis” that gets written about a lot. And

But the reason you don’t have to read assembly when you write in a higher language is that, over time that abstraction has become largely complete and is deterministic.

Like when you write I dunno Python to add two numbers and calls class, that spec is universal and agreed upon. ~99.99% of the time when you read Python documentation, and replicate it, it does what the designers intended.

Spec driven development is none of those things. So while I’d like to see people experiment with higher level languages, I think many folks find there’s now way to consistently reconcile the spec with the code.

As much as no one wants to read the code anymore, it seems like most people seem to agree it’s the only source of truth.

I always find Superintelligence inevitability interesting. And the author dismisses George Hotz argument about supply chains and the way that in order for ASI to truly take off it has to dominate not just the world of bits but atoms as well.

But I imagine there’s this problem that measuring intelligence in AI is not the same as measuring intelligence in humans. How is fable not already super intelligence. How is it not way smarter than the 99th percentile human? In terms of speed, broad ability and pattern matching behavior. If intelligence is loosely correlated with lifetime income, Anthropic billions in revenue should be a decent measure that LLMs are way smarter than most folks.

Because sometimes I think the inconvenient truth is that when people ask, you don’t think we’ll have AI that’s smarter than humans. It feels to me like we had AI that was smarter than humans a long time ago that somewhere around GPT three or GPT four we had AGI and if we already have AGI and we already have AI that’s smarter than humans then maybe the real question is is intelligence not his intelligence actually the right measurement for what we’re trying to achieve.

Or maybe it’s that ASI inevitability claims to be a very general argument, when it’s a very specific argument (or vice versa)

The AI inevitability argument seems to be something like - are you so anthropocentric and unimaginative that you can’t see a world where human creations outdo humans.

When the specific claim is more like “The specific technology of large language models will hard takeoff in the next few years”

There are infinite years ahead of us. Maybe hard takeoff is coming! And maybe it’s a thousand years away.

I think I learn a decent amount from AI so I’m legitimately torn on this, but I guess an analogy I might turn to would be something like “fork lifts are great! But don’t bring them to the gym to do your heavy lifting.”

Cara are great! But a one hour drive serves a different purpose from a one hour run. Running doesn’t make me a Luddite.

So I think - it will continue to be important to “do the reps” so to speaks. Build mental models. Read long form. Do crossword puzzles. Think on your feet and have in person conversations.

I think / if a university carries the responsibility of credentialing you, they equally carry the responsibility of ensuring you didn’t bring a fork lift to the gym.

I was accused of being AI a few posts ago. I think it’s hard get the cadence out of your head if you talk to AI a lot. Especially when making certain kinds of arguments. I was thinking about dropping a few more f bombs, maybe that would seem less AI like.

I think it’s a maturity model question, and an individual ability question. And maybe an industry tooling question.

Individual ability? Newer engineers should read and write more code than experienced engineers.

Maturity model? Teams with better access to objective specifications can read less code than teams without it. Teams with more experience with building loops and validation frameworks the same.

Industry tooling? I think we need more work on the spec driven development front. Some set of standards as to how we define contracts that LLMs can operate against that are more deterministic.

Slowing us down from what?

In the context of many other measures I think speed is an important measure. Maybe even the key measure. I’ve even written a blog post entitled “reading code is an anti-pattern.”

But if the metric is, slowing us down from having more lines in production then the answer is unequivocally yes.

If the answer is providing customer value, or having a sustainable engineering culture. Then I’m not so sure. I’m not saying I believe the opposite, but it feels like excessively optimizing on the wrong thing.

Ultimately I think this asks the wrong question, I think most other surrounding questions is the right one which is - how do you safely and quickly deploy the right code to production that delivers customer value.

I think that will continue going forward involve doing so in an automated fashion, but then the right question isn’t, “should we stop reading code”. But something like “what is the right way to ship intent to production”

Because if you do the first without the second…