5,276 karma · joined July 11, 2012
One big technique it sounds like the authors of the OAuth library missed is that LLMs are very good at generating tests. A good development process for today’s coding agents is to 1) prompt with or create a PRD, 2) break this down into relatively simple tasks, 3) build a plan for how to tackle each task, with listed out conditions that should be tested, 3) write the tests, so that things are broken, TDD style and finally 4) write the implementation. The LLM can do all of this, but you can’t one-shot it these days, you have to be a human in the loop at every step, correcting when things go off track. It’s faster, but it’s not a 10x speed up like you might imagine if you think the LLM is just asynchronously taking a PRD some PM wrote and building it all. We still have jobs for a reason.
One of the more bullish AI people has said the models performance scales with log of compute (Sam Altman). Do you know how hard it will be to move that number? We are already well into diminishing returns with current methodologies and there is no one pointing the way to a break through that will get us to expert level performance. RLHF is underinvested in currently but will likely be the path to get us from Junior contributor to Mid in specific domains, but that still leaves a lot of room for humanity.
The most likely reason for my PoV to be wrong is that AI labs are investing a lot of training time into programming, hoping the model can self improve. I’m willing to believe that will have some payoffs in terms of cheaper, faster models and perhaps some improvements in scaling for RLHF (a huge priority for research IMO). Unsupervised RL would also be interesting, albeit with alignment concerns.
What I find unlikely with current models is that they will show truly innovative thinking, as opposed to the remixed ideas presented as “intelligence” today.
Finally, I am absolutely convinced today’s AI is already powerful enough to affect every business on the planet (yes even the plumbers). I just don’t believe they will replace us wholesale.
Either teaching them is going to get easier, which is somewhat plausible actually — or you think LLMs are going to get dramatically easier to prompt with more reliable output. I actually think that’s somewhat implausible. Log(compute) is a pretty hard number to move past a certain point and it seems we’ve hit that point already. RLHF is great, but it doesn’t scale well and the result is that some tools are hard to build. Giving me millions of context tokens is neat, but if it’s wildly expensive and confuses the model on top of that, it’s not actually useful.
I do think there is room to improve and of course there are new techniques to be discovered. The models may even help discover some of them, although I am skeptical that they will find Everest by hill climbing when they’re currently on Pike’s Peak. Or maybe we really need to find Olympus Mons.
Tl;dr — maybe!
The reason you want Whoop to have a subscription (and why I wear a Whoop now) is because it incentivizes the company to ship great hardware _and_ software. If you don’t have a subscription, your company becomes stuck on the treadmill of needing to have a new device to sell to the public every year (or even twice a year) so that you can continue to fund your business. Pebble found this out, too, and it led to their sale eventually.
Worse, the launch dates are not movable. People are largely not going to wait and buy your new watch in January if they wanted to get a Christmas gift for their spouse. The same goes for Mother’s Day. The scope is also not movable, because it has to have certain things for people to be interested vs last year’s model. We all know what happens when you fix scope and date in the iron triangle — quality suffers.
The subscription model has a great property — you can ship the device to your customers when it’s ready and meets your quality bar, and you can theoretically do it for free, because they are already paying a subscription. I realize Whoop did not take this path with their latest device release, they are clearly trying to goose their revenue for a quarter or two. That said, at the end of the day, their full product offering will either earn your subscription or not, which means that you can be confident that they are aligned with your interests. You generally cannot say that about a pure hardware company, unless they are remarkably disciplined with respect to hiring. I have not seen an example of this in the wearable/health space.
I think you can derive a general principle from this, which is that if a company is incented to sell you crap, they will eventually do so. If instead they are required to repeatedly earn your business, they will either maintain high standards for their products or they will go out of business. I mostly choose to spend my money on products made by companies with the latter model.
Now think about how much of software development is typing out the code vs talking to people, getting a clear definition of the problem, debugging, etc. (I would love an LLM that could debug problems in production — but all they can do is tell me stuff I already know). Then layer on that there are far more ideas for what should be built than you have time to actually build in every organization I’ve ever worked in.
I’m not worried about my job. I’m more worried my coworkers won’t realize what a great tool this is and my company will be left in the dust.
Biggest laugh I’ve had all week!
If Canon started trying to sell cameras that literally only work with their software (not the case today) then maybe you’d have a semi-valid beef, although such a camera would also sell very poorly in the market given the many alternatives that exist, including Canon’s own previous lineup. Even then it wouldn’t be illegal, just harder to justify from a business perspective. Perhaps they could give away a DSLR for a yearly subscription and the math would pencil out for some people. That would be mildly interesting. Canon would have to do a lot of work to close such a product, though, as all of their existing hardware is extremely open.