jbstack

It halts and refuses to carry on after finding a security risk, which is exactly what it's supposed to do? What's the point of it then?

Large profitable corporations with very poor customer support have existed long before AI, so why should it be different? Using AI to provide poor customer service is just an implementation detail.

Yes. It's often hard to hear the dialogue.

It's hard to imagine a frontier model being proficient at finding vulnerabilities, but not so proficient at exploiting them.

Surely finding is the hard part, and any LLM should be able to easily exploit a vulnerability it already knows about?

If it amounts to fraud, then it's also criminal.

For example, if you take out a loan and you sign terms and conditions which say you agree to pay it back and then later your circumstances change and you can't pay, that's probably just a civil matter. But if, at the time you signed the agreement, you had planned to deliberately not pay it back, that's fraud and therefore criminal. The fraud is in the fact that you are making a false representation: you are saying you will pay it back, but you know that to be false. It's a very different thing from believing it to be true, but it then later turns out not to be true.

The same principle applies to LLM subscriptions (or any other contract).

Could you elaborate with examples and numbers?

That's a more risky strategy. What if you added new files since the last time you updated the deletion profile? It's also technically more challenging. You have to think about what might be in RAM, caches, backups, etc.

The good thing about a total wipe is that it's very easy to implement, and it's hard for it to go wrong. You just encrypt the whole drive and, when you want to wipe it, erase the key.

> as already happened with Intel I guess

In which sense? Intel is neither a full monopolist (AMD, ARM) nor has progress stalled (we're still getting a regular cycle of new generation CPUs which improve on the previous).

You might take a different view if you live in Taiwan.

That's fine. It's my choice whether to upgrade my OS or not. If I do so, and my app no longer works on the new OS, then I can purchase a new version. Why should I pay a monthly fee just in case I might upgrade my OS and just in case the app doesn't work there?

Also, it's far less likely than you're suggesting that I would need the exact major OS version I started with. The majority of apps survive an OS upgrade just fine. In my case, I use NixOS, so that isn't even an issue at all since every package can have all of its dependencies frozen for as long as you like, even with a new major OS version.

That isn't the same thing at all.

If I buy version 2 then it's mine and I can continue to use it for as long as I want, if I'm happy with the feature set. I can ignore version 3 or choose to buy it. If I buy it then I now own two products instead of one. Sometimes the older version might even be better (e.g. features I like have been removed or made worse).

With a subscription model I'm forced to pay again and again even if I'm perfectly happy with version 2.

There's some merit to your arguments, but not enough to justify a subscription:

- Subscriptions are marketed as being a lot more than just bugfixes - new features being the big one. But there's usually no cheaper "bugfix-only" subscription, which means that someone who doesn't care about new features has to pay for them anyway.

- To be honest - yes, I do expect bugfixes for free if I've paid to buy the product. After all, a bug is a defect, and products are usually sold with the expectation that they will be fit for purpose. That's the principle which applies to physical consumer products, so why should it be any different for software? If I bought software that calculates my taxes for me, and it turns out a bug means that it applies the wrong tax rules, then I haven't got what I paid for. Why am I expected to pay every month just to make my software do what it was supposed to do in the first place?

- The developer is still incentivised to fix bugs in order to attract new purchasers.

- Subscriptions aren't a magic solution financially anyway, because there's an average limit to how long a customer stays subscribed for.

The thing you have to remember is that there are thousands of products out there all trying to charge a subscription, and most people aren't going to justify taking on more than a very small handful of them at any time. Plus (not directed at you specifically, just in general), your product almost certainly isn't as good or as important to the customer as you think it is unless you've genuinely identified some niche nobody else is operating in, of you have an exceptionally polished product with few competitors.

People are however consistently willing to regularly make one-off purchases to get something they can "own". I've bought way more lifetime licenses for software than I've taken out subscriptions. When you consider that nobody stays subscribed for life, there's always a number that you can charge for lifetime which is in fact functionally equivalent to a subscription anyway.

For the majority of software I use, I don't really care about continuous upgrades and new features, as long as it works with the feature set I signed up for.

A great example for me is the Xodo app on Android. It's by far the best PDF editor that I've found on Android, particularly for annotating with a digital pen. Some features are locked. If I want to unlock them, it's $5 a month. I get nothing out of that which isn't already in the app. I'm happy to pay a one-off fee for the work the developers have done up to that point. I'm definitely not happy to add another $5 a month to my pile of subscriptions.

For me the boundary is this: (1) If I get something of value every month (e.g. use of a cloud server, or something which obviously needs regular updating like Netflix) -> subscription justified; (2) If I just want to use what I can already see in the app -> very unlikely I'll ever subscribe unless the product is absolutely essential to me and there are no competitors.

A good example of the latter is Skritter. I don't care about new functionality, but there literally isn't another app which can do what it does, so I pay the subscription.

> there is no way to sustainably develop a product without subscriptions

This is clearly incorrect given that there are plenty of software developers who offer lifetime purchases. In fact there was a time that subscriptions for software were virtually unheard of.

On a lower level, all that matters is the numbers. If your average customer stays subscribed for 24 months, then charging a lifetime fee equal to 24 months is equivalent to a subscription model. At that point it's irrelevant what's "sustainable" since 24 months is the max you can expect to charge on average anyway.

> an unusual ... definition

I don't think it's that unusual. It seems to me just to be a narrower version of panpsychism:

https://en.wikipedia.org/wiki/Panpsychism

The LLM can only generate text. The harness can do more than just generate text. By joining the two you're allowing the LLM (through text) to carry out whatever actions the harness can take.

My brain can only generate electrical signals. My hand responds to electrical signals and can interact with the real world. The two together can do more than just what my brain alone can do.

If you don't trust a particular brain, don't put a gun in the hand which is connected to it. If you don't trust a LLM, don't connect it to a harness which has access to your production database and only recent backups (https://www.theregister.com/2026/04/27/cursoropus_agent_snuf...).

To be fair, there's sufficient of it publicly available (e.g. on https://www.bailii.org/) that you can easily disprove the conspiracy theory that laws and interpretations are "blatently inconsistent". In fact most judgments are very well thought through and carefully written and cited, because those that aren't tend to be appealed if they are establishing an important principle and getting it wrong. There just isn't enough publicly available to base a full legal AI assistant on.

> would that then be a much better tool

Better than before, yes. Good for general legal work that doesn't require robust legal research, yes. Sufficient for full legal research, no.

The problem is that "a lot of case law" isn't enough case law. You need close to everything. Otherwise this can happen: Canlii case X -> Legal principle Y. Westlaw case Z not on Canlii -> X overriden, Y no longer good law. Or you might simply not find a case which cogently supports your argument, when one does in fact exist. Or, conversely, you are unaware of a detrimental case which your opponent knows about because they have Westlaw.

You can have as much RAG as you like, but if you're missing the data itself (the legal judgments), it's useless. The fundamental problem here isn't technical, it's that a very small number of corporations have complete control over the source material.

Not if you set the VPN up on your router. You can also buy small portable routers designed specifically to sit between your machine and your wifi / ethernet.

Yes, it sounds crazy and against the principle of open justice, but unfortunately this is the reality. Certainly in the UK which is my jurisdiction - and I believe in the US too although I don't know for sure.

In theory, any member of the public can obtain a judgment by applying for one at the court and paying a fee. That's fine if you just need a one-off judgment, don't mind paying the fee, and you're not in a hurry. It also assumes that you know which case you need.

For realistic legal research, you might need to wade through dozens of cases just to even know if any of them are relevant, you might have a deadline of tomorrow to get it done, and you might not want to pay that fee for a bunch of cases that you aren't going to end up needing. Only a company which already has a comprehensive copy of virtually every important case can help you here.

A typical workflow for a complex piece of legal research might look like this:

1. You need to research a legal topic.

2. Do some Googling, or chat to your LLM, to get a rough overview and some pointers for further research (but don't completely rely on what you find).

3. Read some professional content (e.g. Practical Law articles relevant to the topic, or a legal textbook).

4. Read the relevant legislation.

5. Use a legal database to download all the cases you found from steps 2 and 3 which seem like they might be relevant.

6. Use a legal database to download all the cases which cite the relevant legislative provisions you found in step 4 and seem like they might be relevant.

7. Use the legal database to confirm that those cases are still good law (not overridden or criticised by a later case).

8. Skim read them, discard those that turned out to obviously not be relevant.

9. Read the remaining ones more closely.

10. Note any useful-looking cases which are cited in the ones from step 9, and recursively work your way through those cases as well.

Relying on court-provided copies of judgments won't realistically help you with most of these steps.

Agreed, but that's the fault of the lawyer, not the LLM.

Humans make mistakes too. In many cases, more often than LLMs. Humans are still useful for doing work.

I use AI extensively in my legal work. But I check every citation myself, manually. That means that I read the entirety of every case that I plan to cite in my output, and I check on Westlaw that it hasn't been overridden by a later decision. If you're just producing the AI's output verbatim, then you have only yourself to blame when things go wrong in the courtroom.

Legal professional here. This is NOT a replacement for proper legal AI assistants (e.g. Westlaw, in my jurisdiction). As far as I can tell, this is just a wrapper around regular LLMs i.e. nothing that you couldn't achieve yourself with the right prompting.

What legal professionals actually pay for, and that is virtually impossible to replicate unfortunately, is to give the AI access to a legal database of case law. Without case law, you can't do accurate legal research, and you are inviting disaster if you're doing things like drafting statements of case or skeleton arguments.

There's a reason why companies like Thomson Reuters have an oligopoly on these types of products, and can get away with charging thousands a year. They are the only ones with access to a comprehensive set of case law, and they've entrenched their position by having exclusive contracts with the law reporting companies. Without that, your model is just relying on publicly available cases that it can find on Google etc., and that's just a fraction of the full set.

With that said, these types of competitor products can be useful if you're just doing simple tasks like drafting letters or reviewing contracts and you accept that you need to do the legal research separately. But again, you can get that with just ChatGPT + a good prompt.

VPN. Simple as that. Most companies aren't bothering to check anyway, most that do aren't detecting VPNs, and for the few that do that, there are ways to circumvent detection if you are really determined.

This is a common misconception. Namely, that increasing lifespan just means extending the part where your health degrades continuously. That's actually a very unrealistic outcome for life extension technology. In general, the things that cause your health to degrade as you age are interlinked with the things that cause you to die. If you find a way to increase lifespan, chances are you've also found a way to increase healthspan. In fact, all of the best methods we currently have to live longer do exactly that (e.g. exercise, eat healthily, avoid smoking, etc.).

You seem to be implying that at after a certain number of years (e.g. 79) you wake up one day and say "I'm fulfilled and have nothing left I'd like to achieve".

As someone who occasionally works with terminal patients, I've never seen that in practice. In reality most people desperately wish that they could carry on living, and have plenty of unfinished business that they'd like to see through. The only exception I've seen is when someone is in so much pain that they just want to end the suffering.

If we turn your argument on its head, a person who dies at 20 is just as fulfilled as a person who dies at 79. So why should anyone bother trying to live a long and healthy life?

You want it, but then you closed by explaining exactly why you shouldn't want it. Plus, the new baseline isn't neutral (as in, everyone is the same again). If humans can now do 10x the work as before, the employer doesn't need the same number of humans to carry out its work. So the new baseline is actually "let's keep 1 employee and fire the other 9", unless the business can find a way to suddenly expand 10x so that it needs 10x as much work done.

Not sure why you're getting downvoted. People shouldn't get punished for asking questions.