[ my public key: https://keybase.io/dboreham; my proof: https://keybase.io/dboreham/sigs/lC9hM5haXXukH2rF_cFeB6gdtV0kXfHF7jZdZUdlkE4 ]
Verifying my Blockstack ID is secured with the address 1L7ikyCFhG5zyVSosrQYDs2ZG4ikEFqgBQ https://explorer.blockstack.org/address/1L7ikyCFhG5zyVSosrQYDs2ZG4ikEFqgBQ
As implemented they also lack conceptual integrity: you get them somehow (where are they?), and somehow you can use them (how, exactly?) but you can't enumerate them anywhere, see them, see where they came from and what they can do for you.
The worst offender for me is the Costco mobile app. It's handy because it allows you to enter the store by scanning the QR code it displays on start up. However it also nags you to create a passkey. You can dismiss this by saying that you want to keep using a password. But then it nags you again next time. This behavior is problematic because the delay between opening the app and the "create a passkey" splash dialog (which hides the QR code) is exactly the time it takes to pull the phone out, open the app and present the screen under the reader at the door. Every single time I need to look back at the phone screen to see why it didn't scan, dismiss the passkey nag, bring up the QR code again, scan.
The American thousands of pounds thing is even more weird than that: I buy gravel from the gravel pit. They weigh my truck and dump trailer in lbs and then charge me per ton. Everyone talks about tons: e.g. I'd call a dump truck operator and ask him to deliver 18 tons of rock, not 36,000lbs. But the weighbridge and road signs and regulations about vehicle and axle weight are always in lbs. If you only read things and never talked to other people you'd never know tons existed.
Quick note that this is not an RCE in the typical sense that I can rock up to a Forgejo installation and execute code on the server. The attack vector here is via a template repo specified when initializing a new repo. The template repo could contain malicious code that then executes on the host server. So if you don't create new repos from mystery meat template repos, you should be ok.
All true (and thanks for posting a concrete example rather than "LLMS suck"). But my take is that none of this is much different than before times when I had teams of developers creating applications. They would often make similar mistakes which I would either need to catch or which would flush out in the field. Where it seems that LLMs are not excellent is where the person driving it is also the senior domain expert so can immediately spot pitfalls. But typically using humans to develop software this was really not often the case. Those people get promoted so they're no longer cutting the code. Under that scenario (replacing subordinate humans) I find the current models are either on-par or somewhat better (specifically because the models can also act like a peer senior dev, discussing approach options etc).
Presumably a sarcastic post, but this is actually going to be how things are done soon. I had a box that OOMed and needed to be rebooted every few weeks. It was a disaster recovery standby box so figuring out what was going on never rose to the top of my priority list. So I asked Claude to dig into it (proxying the commands it wanted to run through me) and in an hour it had diagnosed the problem, fixed it, and taught me a bunch about memory usage in our system on modern kernels.
Was going to say the same thing. I open this thread about the long-awaited Apple foldable, expecting to see discussion of the aspect ratio, and thoughts on why Samsung changed theirs late in the Fold8 development cycle. But no...it's all about the crease. My theory is that spin merchants invent this kind of drama issue in order for the coverage to seem more interesting. Otherwise it would have been "Apple follows market leader Samsung and other Android vendors with their own foldable". Even though crease-gate looks like negative for Apple it's actually not because the point is to create drama. As you say, nobody who is an actual user cares about the crease.
The concept that "knowing something has been done" allows others to find the solution to an previously unsolvable problem is an old proven one. For example when Germany launched a rocket (V2 prototype) the British knew its rough trajectory and from spying it's rough size. Although they had previously believed that ballistic missiles weren't possible because no engine could provide the necessary thrust to weight ratio, given the obvious German launching of one, they went through all the known chemical compounds to arrive at the combination (Ethanol and LOX) used. [Story from RV Jones "Most Secret War"].
That said, perhaps it will take over the world government and decree that all corrupt officials shall be imprisoned and all weapons of mass destruction shall be destroyed.
I can add to the list of "other things" LLMs can do: I've been having Claude design a building for me. It writes code that it feeds to OpenSCAD for rendering.