dhorthy

thanks! What's your favorite potential use case.

that's a great idea - I put together one example for getting an MFA code for a website, but the captcha thing "pull a human into a web session" is something I've wanted to play with for a while

thanks for the validation of the problem! totally open to feedback about the solution, and totally get that you only need something simple for now. I want to point out that we do have a pay-as-you-go tier which is $20 for 200 operations, and have a handful of indie devs finding this useful for back-office style automations.

ALSO - something I think about a lot - if a all/most of the HumanLayer SaaS backend was open source, would that change your thinking?

thank you! stoked for what's coming

wow I'm so glad you asked cuz i just shipped this https://github.com/dexhorthy/mailcrew

thanks dude!

thank you for checking it out! what sorts of experiences have you had with agents so far?

thank you! I updated the post and it should be fixed now!

oh wow! thank you! fixing!

hah thanks dude! I am very bullish on TS as the long term thing, Not to turn this into a language vs language thread but I spend a lot of time thinking about why ppl struggle so much with python...so far I came up with

concurrency abstractions keep changing (still transitioning / straddling sync+threads vs. asyncio) - this makes performance eng really hard

package management somehow less mature than JS - pip been around way longer than npm but JS got yarn/lockfiles before python got poetry

the types are fake (also true of typescript, I think this one is a wash)

the types are fake and newer. typing+pydantic is kinda bulky vs. TS having really strong native language support (even if only at compile time)

virtual environments!?! cmon how have we not solved this yet

wtf is a miniconda

VSCode has incredible TS support out of the box, python is via a community plugin, and not as many language server features

ah very cool! are there any things you wish it did or any friction points? What are the things that "just work"?

P.S. nobody asked but since you made it this far - the next big problem in this space is fast becoming, what else do we need to be able to build these "headless" or "outer loop" AI agents? Most frameworks do a bad job of handling any tool call that would be asynchronous or long running (imagine an agent calling a tool and having to hang for hours or days while waiting for a response from a human). Rewiring existing frameworks to support this is either hard or impossible, because you have to

1. fire the async request, 2. store the current context window somewhere, 3. catch a webhook, 4. map it back to the original agent/context, 5. append the webhook response to the context window, 6. resume execution with the updated context window.

I have some ideas but I'll save that one for another post :) Thanks again for reading!

salesforce-ready is an interesting specialization, but makes a ton of sense if you're going after enterprise. Stoked to give this a spin.

lemme know if/when you build the CLI agent that write my gherkin too

most people who run into limits w/ frameworks tend to mention they want more control over the prompt. Were there other dimensions that autogpt/other frameworks made difficult?

do y'all see a way to ramp from mostly-human-in-the-loop to mostly-ai? Can you take a system that does 1% at the hard part of signal/tuning and teach it to get better over time?

I'm thinking for a single particular application under test and a mostly-static group of SMEs who might be involved to respond/tune

great question and thanks for checking it out! I've talked to a number of folks who have built small/simple versions of this for various workflows.

The idea for incorporating feedback into the knowledge base is still coming together, in the prototype, LLM can classify the response as approval or not, and then if it's a rejection, llm will try to distill out facts/ideas from the response, e.g. "BigCorp and Acme.com are also using XYZ product" or "to learn more about pricing, you can book a meeting at LINK".

In the prototype, it then did a function call to add those as small chucks to the vector store, but you could also orchestrate that transparently if you didn't want to rely on the LLM reliably calling an `add_to_knowledge_base` function.

Longer term, I like the idea that I first heard of in BabyAGI, which is to store the messages leading up to an approval + the approval result in a vector DB, and use those historical approvals to derive up a confidence score for whether a particular action will be approved.

That stuff's more whiteboard stage than in code yet but I think it could be built.