You are better off coding up your workflow in some normal language, like Python, using normal programming techniques.
There is the (1) essential complexity of your problem and (2) the accidental complexity that is added by your tools. If your framework doesn't confront (1) your project will fail. Adding a few more features that your framework is lacking will probably take more effort than building a minimal viable product from scratch. As for (2), the framework authors thought a lot of things are important that are not important for you. The framework will make you do things you don't want to do. Suppose it has to keep some state in a database: do you want it in a database that you know is reliable and know how to manage or do you want to add risk by adding a database that is unknown to you, etc.
Years ago I wrote a general purpose model trainer and it was one hell of a mistake because it had to face many different scenarios, not least that it was supposed to support models that take 30 seconds to train (but gets thoroughly model selected) and models that take 5 days to train across a cluster. Either case brings in a lot of requirements that the other case doesn't need, trying to support everything made something that wasn't simple or reliable for anything.
See langchain, for instance, if your first impression is that makes no f'in sense, it's not because you're not smart enough to understand it, it really doesn't make any sense.
It's natural that software in this space will evolve quickly, but that's not a proper argument for writing everything on your own.
If I was advising somebody how to do a really difficult agent task (anywhere I didn't have an idea on how to bootstrap a large training set) I'd consider building a system that lets a human do all the steps manually first, then build something that can automate the steps.