doesn't seem like it makes sense to train AI around human user interfaces which aren't really efficient. It is like building a mechanical horse.
New technology is slow due to risk aversion, it's very rare for people to just tear up what they already have to re-implement new technology from the ground up. We always have to shoe-horn new technology into old systems to prove it first.
There are just so many factors that get solved by working with what already exists.
Both Open Pilot and Tesla FSD use regular cameras (ie. eyes) to try and understand the environment just as a human would. That is where my analogy is coming from.
I could say the same about using a humanoid robot to log on to your computer and open chrome. My point is also that we made no changes to the road network to enable FSD.
I am more leaning into the idea that an efficient self-driving car wouldn't even need to have a steering wheel, pedals, or thin pillars to help the passengers see the outside environment or be seen by pedestrians.
The way this ties back to the computer use models is that a lot of webpages have stuff designed for humans would make it difficult for a model to navigate them well. I think this was the goal of the "semantic web".
We always make our way back to trains
While the self-driving car industry aims to replace all humans with machines, I don't think this is the case with browser automation.
I see this technology as more similar to a crash dummy than a self-driving system. It's designed to simulate a human in very niche scenarios.
[Looks around and sees people not making APIs for everything]
Well that didn't work.
That'd be neat.
But for now: The web exists, and is universal. We have programs that can render websites to an image in memory (solved for ~30 years), and other programs that can parse images of fully-rendered websites (solved for at least a few years), along with bots that can click on links (solved much more recently).
Maybe tomorrow will be different.
It'll never happen, so companies need to deal with the reality we have.
If you want to make something that can book every airline? Better be able to navigate a website.
That's just one obvious example, but the principle holds more generally.
If you read through the thread - to me its apparent.
Even if you make burner accounts, this behaviour doesn't help one grow.
But hey gramps could be wrong eh