quarkcarbon279

love the simplicity of this article. Reminds me so much of foundational software engineering. i love databases def not cosmosdb

"These weirdos have eyes up above. They can read lips from space." from Pluribus. Watched Pluribus and couldn't shake one idea: Everyone knows Carol. Whoever she calls has the full context and just does the thing.

Isn't that the best customer experience every product should provide?

I'm so surprised you didn't add your agent on your own website? And the same fundamentals of a browser agent why you can't achieve everything with APIs alone apply here too inside a website. What's your take?

We have been building in the space balancing both with priority to GUI tools currently building on top of the great GUI experiences of a product, rtrvr.ai/rover

As a developer I have many times seen Claude's models confidently hallucinate, jump into conclusion. Fable though I used just for 2 days, didn't experience it much in the short-term.

The reason we open our client side code is to bring in the trust in putting rtrvr's DOM intelligence in your web apps - https://github.com/rtrvr-ai/rover/tree/main . Our monetization is super straight forward with subscription - https://www.rtrvr.ai/pricing . The experiences of some extensions shipping anything or selling user data comes in when people build them as side-gigs not when we pour more than year in building the highly accurate automation engine. We have cloud sandboxes too if you prefer executing with the same intelligence on cloud and not on your own device.

PS: Also, our data policy if you are interested: https://www.rtrvr.ai/blog/rtrvr-ai-privacy-security-how-we-h...

PageAgent doesn't have the strong page understanding - semantic tree representation of the pages - it's just a flat DOM basic stripping of HTML - which makes it hard to navigate shadow DOMs, even same origin iframes for that matter or diff frameworks. And also they do element marking - CUA style not sure if they use it in the actual calls to Qwen. And yeah, as arjun takes 30 steps to even do a basic task of find some info.

What we strengthened building agents working on 2M+ web workflows in the past 4 months - is our representation of pages that seamlessly helps agents go through any page old to new iframes, shadow-DOMs and more. Best part of Rover if you as website owner enable cross-origin reqs, say Doordash has Rover and a merchant be like get my restaurant menu from my website and update in Doordash. Rover agent determines the 3P website need, launches our cloud browser to securely execute 3P site actions gets the menu and updates the merchant menu on Doordash so your users never have to leave your site to do a task - one of a kind enabling cross-site interactions

Hey, we have various configurations available even in this very early preview: https://rover.rtrvr.ai/docs/configuration and you can turn off telemetry so we don't collect the data . As a website you can also configure to host the script on your preferred endpoint and our 'apiBase' helps with that pointing to custom domains. And based on project data needs we are also open to tighter data policy rules.

We have seen people hire consultants to build RAG pipelines, maintain them only to be QA Agent when the world is shifting losing traffic to the likes of ChatGPT and Google comes up with a protocol that makes you do more work with WebMCP essentially maintain a lot of APIs for your UI and DBs. Rover is the answer to these painpoints. No RAG pipelines, just one script tag your users engage, retain, get work done on your website without ever leaving it and fully conversational.

what permissions are you talking about? No user permissions/any insecure permissions are needed to navigate cross origin iframes, shadow DOMs and likewise. It comes down to your architecture choices and capabilities - rtrvr can navigate these diff realms without ever taking debugger or such insecure permissions

Let's start the year of vibe scraping! Curating websets and powering apps and sales pipelines from data curation to data entry all in one prompt. Also, if you got hot takes/ideas on accessing the web: Security, Public info Vs Bot Blocking, and more chime in!

let's go!! after a long time

Yeah when building for consumers on their devices, security is of outmost importance. I tried on couple use cases too. They currently use Sonnet and this is very initial version so it sometimes fails on picking the right tools. Some actions fail, for example I used their calendar template it failed to Save in the end.

Would love to see if they have cost breakdown for the task haha. It took so many actions, can be optimized + screenshots for one task.

World of LLMs or not, development should always strive for being fast. In the LLM World, users should always have the controls on accuracy Vs speed. (Though we can try for improving both and not one way or other). For eg at rtrvr.ai we use Gemini Flash as our default and did benchmarking on flash too with 0.9 min per task in the benchmark still yielding top results. That said, I have to accept there are certain web tasks on tail end sites that needs pro to accurately navigate at this point. This is the limitation given our reliance on Gemini models straight up, once we move to our models trained on web trajectories this hopefully will not be a problem.

If using off the shelf LLMs always have a bottleneck of their speed.

How did you keep your costs so low? Eval costs especially with Agents can go up a lot and what did it cost for other agents?

It's coincidental that Anthropic also published recently on similar finds and approaches on multi agent orchestration and memory management https://www.anthropic.com/engineering/built-multi-agent-rese...

I have been following your product for a while now. Pdf editing is a hard problem, but good to see the progress you have made!

The credit card data is massive. Lemme play around with your agent.

What's your thought behind having an exchange? You want everyone to collaborate on creating big datasets for training or individual use?