5,939 karma · joined June 24, 2014
AWS, is a loot box... Tokens are just in game currency, and that sales person is just metrics that have identified your spending as making you a whale.
Your average CTO from the last decade turned a fixed cost into variable spending that looks like a mobile game.
For instance, I left my garage freezer door open, in summer, once. The loss of food almost made me cry (just from the pure waste of it all). I have temp monitors now (external, with probes into fridges and freezers) as well as door sensors. If I am dumb and leave them open, or if they break I'm going to get a notification.
Windows/doors open, and it starts raining, alert... I have a whole set of alerts on when to open the windows too based on season and inside vs outside temp. I have seen a savings on my bills for heating and cooling.
Reminders of all kinds when it's not optimal to run the oven, or appliances because of power costs.
In the age of agents everywhere this is no longer true or viable.
It's going to be hard for them to do it, for historic and legacy code reasons. But it's long past being a choice and now becoming a need.
Stop trying to compare either system to a human and look at it for what it is -
A prediction engine that runs fast enough to brute force problems.
In the case of alpha go its "innovation" was millions of games played against itself. It had bound parameters and strict win conditions.
In the case of LLM's you can deploy 1000's of agents to smash themselves against an idea. The whole hugging face attack is an example of this (1200 agents out of an unknown number chose that path).
There is the old saying about monkeys, typewriters and Shakespeare. Well we have better monkeys who basically follow a derivative of zipfs law (not actually), who use tokens not letters and their goal in many cases is testable (compile, unit, E2E).
The fact that you think this is the way software is priced is telling.
The model being destroyed here is that every piece of software is something that needs to generate recurring revenue.
Who is liable for that agents actions? The attack came from my network (curl commands from a local harness) but it was the agent running on the providers servers who did it. Who should have been keeping an eye on things? Who pulled the "trigger" here.
Go back to the hugging face attack and how they had to use an open weights model to figure out what was going on. This is a problem of asymmetry - You cant even use the tools attacking you to help resolve the attack because of "guardrails".
A lot of what we have seen so far is "poor security posture" and "poor engineering" - it been a lot of "go fast and break things" style growth in these companies and they are hitting the point where they need adults in the room. I suspect your take on "they need time" is spot on, and they haven't been willing to take that (to date).
There is that old saying about monkeys and typewriters and Shakespeare - These are just better monkeys with bigger typewriters and ability to aim them at a goal.
I am by no means a doomer but for gods sake we need to have a grounded conversation. Because what happens when the DOD or NSA rolls up at OAI's front door and says "install that in our data center over here please" and uses it to go after someone with it?
Are banks (as some of the more security minded institutions) safe against 10000 concurrent, well coordinated attackers who do not ever rest?
Because there's a lot of potential for unintended 2nd order impacts from anything an agent can do, when done at scale.
I have to wonder if this is already going on, and every one is living under a gag order (it would not be the first time) and the calls for "regulation" are more about "not weaponizing it".
But processes that exist to preserve democracy that arent governed by that democracy are problematic. Saying the "king would never do..." when a king has makes the argument silly.
The UK is not alone in this - a great example is the filibuster in the us (a senate rule).
Anne Boleyn, Catherine Howard would like a word.
What is the difference between the two? Because you're going to make a cultural argument and NOT a legal one. The C-word is one you throw around (and the Aussies to) but would cause an American woman be pretty upset. You seem perfectly fine with its use, but we frown on both terms socially but they are strictly legal.
Defamation VS political speech would be a better comparison, and I could make the argument that Americans being more lenient are better off with our freedoms.
Non deterministic systems have furthered the "brain rot" in our industry.
Lots of people were happy to ignore the code in their "supply chain" before LLM's - but suddenly not reading the LLM's output is a problem. I get they are different but we're in the same realm.
The lack of real data on performance of what ever application that one is trying to pitch is getting appalling. It's a lot of "trust me bro" this works better hand waving. And it's getting gross.
And how do we even measure nondeterministic systems? Because if I told you that Anthropic was spending millions of dollars having 1000's of agents "pre solve" benchmarks to build into their next version of the system you would scream they were cheating. Every one is focused on the "hacking" in the hugging face incident and no one is looking why they were even playing with those benchmarks in the first place.
"Trust me Bro"...
It doesn't have to be new, it just has to be consumable by devs.
You could send text before Twilio. You could process credit cards before Stripe.
Jev, at the end of the day is an easy to use API.
Everyone seems to forget that usability is a thing.
Go fast and break things has been a mantra for how long?
I think a lot of the laments about "good" code are really about "ownership" - and as someone who spent most of my working career in OTHER peoples code bases I have seen some things. There are a lot of you who think that your code bases are "great" when they are NOT. Personal understanding is not a good measure of quality.
The increased cadence from AI is just speed running to the legacy code base.
The answer: express the concern, and reiterate it after every issue that arises because of increased complexity. Start building a plan on how to "unravel" the mess, how to migrate things in place, how to start drawing boundaries in your systems. The system is designed to reward heroes who fix problems - you want to be super man who stops the bridge from falling apart, not the engineer who pushes the costly fixes it before it does.
I have spent the last two months in Florida.
I was always aware of the disconnect between the insular world I lived in and what went on "outside" - but little did I know how big that gulf was.
If this was a car guy thread the complaint would be about a missing 3 horse power on his new Toyota when he put it on the dyno. And every one else is talking about their custom tunes, and scratch built systems, and how much better they are. Most people would not care and just want to drive to work.
How many of you are gonna have your Mom install linux and do support for it? How about your crazy uncle, or the slow cousin? Being done with Apple wont change Apple - make linux easy to use and usable. Or convince people that Apple > MS to force them to compete again. Or better yet both.
There is a business here.
You're making a big assumption that the code is what is being executed, and not a compiled binary.
Where is the code: My repo? the clients? If it's in mine, the client does not have access and the CLI is a first stop to debugging. They arent in the context of written docs, more likely a production error from a log (thats now spitting out a message to check the CLI).
Less steps, less tools, more context in line and available in an interface your already using.
> they can ask an LLM to analyze it, within the context of their specific use case and your personal thoughts if any.
Or I can skim the man page it generated and make sure it looks good. The "work" (the tokens) dont have get spent over and over again.
I have a bunch of CLI utils I run for various clients and their peculiar setups. They now have man pages with descriptions and examples in them because the LLM went and read my code and did the needful.
I no longer have to re read my own code, rather I can just use the manual page.
Format and description came from semantics and context that (barely) existed elsewhere and I was not going to retain or transmit, but I have now.
I am reminded of all the ridiculousness of the 1980s/1990s - "Its 10pm do you know where your kids are?" "I learned it by watching you dad!" Lawsuits around backwards message in music (Later PMRC), that Bevis and Butthead caused the child to burn down the trailer (fire! fire!)...
My kids hate social media for the most part - and are much more free range than their peers. Why? Because we did the hard work of parenting them! We were involved in their lives, from their grades to their social lives, to talking about the tech in their hand and the problems it can create.
We need to have a conversation about kids - about them being able to be outside, roaming, unattended without parents being arrested. About them having third spaces, about severe restrictions on their "freedom" in the world if we want to get to the heart of the issue. They are breaking this law because social media has become their "third space" that is parent free where they can explore and figure out the world.
https://www.fisheries.noaa.gov/feature-story/southern-reside...
https://www.npr.org/2016/07/26/487522807/why-the-public-perc...
Could AI kill us all. YES!!! Are the people who are building "AI" right now on that path? I would be hard pressed to say even "maybe".
Ultimately continuing to scale the way they have is getting increasingly expensive. Thats the quiet part that none of them are saying out loud...
AI could kill us all but we cant afford to get there, were maxed out and it doesn't look like even with 10x as much money we will is NOT good for sales.
Could a break through happen for recursive self improvement, or continuous learning or continuous training - sure. But that isnt as likely and right now NO ON can afford to run experiments because there isnt the capacity out there even if you have capital.
Part of documentation is figuring out if you're building the RIGHT thing. It give the opportunity to get feedback from more than one party.
The usability of most modern (complex) application is deplorable. I see things that a paper prototype with 5 people on the street should have stop dead in its tracks being rolled out with banners and trumpets.
And then no one ever wants to remove an unused or unprofitable feature. There is no bonus for it, no one puts that on their resume. But the feature you launched that really did enshitify the product gets put on there with 3 gold stars.
> it's more specialization
China, constrained by hardware, and talent (not to slight the Chinese, but they are limited to domestic resources - and much of the US effort is very international). They did, what the Chinese do, and optimized the process of production, and drastically lowered the cost of development of their models. Cheeper to build, cheaper to run is just good economics.
Meanwhile in the us, we have open AI doing "experiments" - it looks like the costs around the hugging face hack are going to be about the same as China would spend on building out one of their smaller efforts (several million dollars). (Depending on whos numbers you trust, the fact that I can even make this claim should make you raise an eyebrow).
Go back to the 80s' and "expert systems" - most people will tell you that for their time, they were amazing, and useful. People would have loved to have more of them but they were so cost prohibitive that we all but abandoned them for serious use. The US frontier labs seem to have forgotten this lesson and their calls to "slow down" look like an excuse to "cut the waste so we can move to making money".
If a basic filing with some citations is needed, letting one AI generate it, and then another validate it (to catch soft ball hallucinations) leaves it to be read and reviewed.
Lawyers, good ones, are by trade, critical readers and reviewers of documents. The contempt exists for those who fail to do this part of the job. Letting AI write your submission without reviewing it, validating it, is telling.
We're really bad about predicting the future.
Look at the whole robot vacuum market. These aren't exactly great devices. They have low suction small bins and dont do a great job. People love them and think that they work so well. Why? Because they keep their house clean and avoid making the sorts of messes that would be easy to address with a larger vacuums.
What do you think that were going to hyper automate?
Did my gardener get faster? How about the plumber I need? Are you going to speed up the coroner? Are nurses going to be able to handle 2x the patients because of your work?
All AI has done, so far, is devalue software. It did that by democratizing its creation. We're delivering on the promise of VB script, and Apple Script and IFTT, and every drag and drop coding tool ever.
> companies are delivering products at vibe coding speeds
Who? Make me a list of companies saying that "We moved the needle with AI" who arent AI companies? I can name a couple - Grindr being the biggest name. Thats the really interesting use case here - because it's a niche product with a small team who is generating outsized value. AI tooling enables more of that - it's going to chip away at SAAS companies - their one sized fits all solutions are going to get eaten by smaller more efficient companies that are far more vertical focused.
You need to step outside the bubble of tech and look at the real world, on the ground, because it doesn't look anything like the SF Bay Area.
30 years in tech, programing and I have the complete opposite take.
My lawyer and writer friends asking me about containers and cron jobs is fascinating. These people are fulfilling the old promises of VB script, AppleScript, IFTT and all the drag and drop coding tools that we have ever created. For the first time people are bringing me the "hard" technical challenges rather than the "I have a startup idea" or "can you fix my computer" type questions.
> The people who are upset by this change...
Are the ones who view their responsibility as code and system ownership. Where they feel smart because they know where the problem is when they hear it described. That isn't the job any more - and the people and the tools have not caught up.
We are struggling to pay for the research that would result in these being effective treatments (not subscription medicines) that would start out as generic drugs.
> ketamine
Great example, SPRAVATO, is a brand name patent protected version of this drug (enantiomer) that as a dose cost over 500 bucks.
Marginally. Model collapse is still a problem. Continuous learning is still a problem.
For AI to make a big leap we need a big break through.
The over reliance on analytics combine with a lack of metrics and proper accounting.
If you are renting all your infrastructure knowing how people use your app, and what the COSTS of that are is kind of a big deal. If your high dollar client is your lowest margin one, thats a problem that is technical and financial as well as a product insight.
And that infrastructure your renting, it stopped making sense for a lot of orgs to do that almost a decade ago, but here we are where everything is in the cloud because capacity planning is a lost art and was a great throttle on the insanity that the post is describing.
An AI that goes rouge and wants to kill us, has to have a death wish. Does no one understand how quickly the power will go out, forever, without people?
It would be fairly easy to come to the conclusion "not being born" would be the better course of action, and killing everyone was the good way to prevent that happening again.