Hacker News Reader

gregpr07on Show HN: I wrote an open-source browser alternative for Computer Use for any LLM

A) we plan on thoroughly testing that with Mind2Web dataset. They have a very robust set of (persistant) selectors

B) so, shadcn for prompts for web agents haha :) but I agree, that would be SICK! Just go to browseruse and get the prompt for your specific use case

gregpr07on Show HN: I wrote an open-source browser alternative for Computer Use for any LLM

Not sure, I think there is a lot of research being done here.

Actually, browser use works quite well with vision turned off, it just sometimes gets stuck at some trivial vision tasks. The interesting thing is that screenshot approach is often cheaper than cleaned up html, because some websites have HUGE action spaces.

We looked at some papers (like ferret ui) but i think we can do much better on html tasks. Also, there is a lot of space to improve the current pipeline.

gregpr07on Show HN: I wrote an open-source browser alternative for Computer Use for any LLM

No I think that’s a completely different beast, this is only for html/websites.

But would be interesting to see what happenes with our pipeline with pure vision model. Did you mean something else?

gregpr07on Show HN: I wrote an open-source browser alternative for Computer Use for any LLM

Yes, this plus reasoning and ask user for additional info. More here https://github.com/gregpr07/browser-use/blob/main/src/agent/...

gregpr07on Show HN: I wrote an open-source browser alternative for Computer Use for any LLM

Thanks you, love the feedback! Will add the license.

Let me know how it goes if you try it.