Everything your building is now streaming through a third party who could steal it, the same way Amazon steals open source software and repackages it.
All you need to do is look at palentir's forward deployed engineers. Once you let a third party observe your processed, you're cloneable and replaceable _as an entity_.
I think we can look at our own human psychology as it developed into social groups. If our brains were obviously deterministic, it would be easy for people to prey upon us via structured inputs. Some level of randomness in our thought process is almost evolutionarily required to avoid becoming part of the larger super structure, similar to an ant colony.
So these AI companies are definitely more dangerous than just the "AI is super intelligence". They're basically a intelligence platform everyone is wiring into.
That's today. Tomorrow, they'll be run by an MBA which is the enshittification.
If you can make a facebook clone it doesn't mean you can steal facebook's lunch. there are exceptions of course.
Now even if you want your idea to compete with all the other todo apps with your special sauce, youre giving a third party your recipe. That third party ostensibly has no reason to steal it, their business model does.
So, sure, if your app is a cult of personality and nothing special, i admit it doesnt matter.
But the number of software home runs is dwarfed by all other auccess on individual products
At their scale I'd expect it to be only worth stealing potential home runs and even so, imagine they did this a couple of times, how would that make them look? Do you think it would be worth the bad blood and mass exodus that would follow?
Not really. Ants are superorganisms, they share more than half of their DNA with each other. You could think of each ant as being like a cell in a human.
So atleast pare of our collection of genes has added value as with random decision factors rather than find some supergene that everyone benefits from.
Whenever I see these vibecoded apps, my move is to just do what you've described: point an agent at it and tell it to reproduce it (usually with my own little customizations). It's a bit absurd, but I don't trust that they vibecoded the app as well as "I" could lol.
Hopefully someone smarter than me is working with trying to develop a new programming language given what we know to be possible with LLMs
The open models that are chasing the frontier labs will stop being open once things slow down and there is less incentive to undercut the front runners. Time will tell if GPU compute gets cheap enough to run stuff locally.
I think the next generation of exploit will be putting up code all over the internet that does x, y and z and then when asked to one shot whatever that code does the agents bake in the exploit that was indirectly included in their training corpus.
https://github.com/seb3773/ntfs-repair-rfc
https://github.com/Kotivskyi/screenshot-tool
I think it was more popular around the beginning of the year to mid-year
Again, I think it heavily depends on if people are writing comprehensive specs with sufficient detail.
It makes sense to not trust AI models with the ability to read and alter your screen for many many many reasons, but the developer of the tool knows that as well. A tool that lets your ai do something doesn't also let it do anything.
So, as always, it's a question of if you trust the software producer. You're giving them the ability to draw on your screen too. Would you have the same fear if there were no AI involved?
For stuff like this I imagine AI misbehavior as a novel failure trigger not a novel failure type. It can only do the harm that the software was able to do on its own already.