Winner of the SF Mistral AI Hackathon: Automated Test Driven Prompting | Hacker News Reader

Winner of the SF Mistral AI Hackathon: Automated Test Driven Prompting

prompting.flyflow.dev

95 pointscarlcortright15 comments

Isn’t this just a very naive implementation of what DsPY does?

I don’t understand what is exceptional here.

(author) It's very similar. I think the extension of this is that we could build a frontend gui for something like this for prompt management. Love what they're doing

I built something like this too:

https://github.com/hitchdev/hitchstory/blob/master/examples%...

Just yesterday, I wrote an article about FT and learned about services like Entry Point AI.

Seems like an awesome idea. I'm curious how long it will take to get a model on a reasonable level.

Phind is pretty good and also the fastest model I used recently, so I'd assume it's quite small, no?

Post Author: getting a lot of requests so scaling the backend. Standby.

You should add a very short description of what this is and how it works. I kind of get that it is a way to tune/generate prompts for specific input output test cases, But its not clear exactly what it does.

can you please add more info on the page to show why it is important and how its helpful

Yeah I'll do that now!

It'd be great to get a little write up of the technique

Added it!

Sorry to ask, but would you mind expanding a little more? For example, when you state "our system iteratively rewrites the prompt to better fit the desired behavior", what is the use case for this? Is it to make personal datasets? Thank you.

Shouldn't this be LoRA training?

Yeah I've been thinking about this lately.

LLMs come and go.

Prompt engineering techniques come and go.

But eval / labelled dataset is always useful once you built it.

In my experience, architecture stays. A nice modular codebase (where you can switch-out the various models and have control on the minutia of the pipeline from data to model) brings a lot of value and can fairly easily be kept up-to-date with best practices.

“Learning” by prompting, calculating the loss against evals, and updating the prompt

Reply on news.ycombinator.com