Can you describe your use case?

We also faced the same problem with other tools and hence building UpTrain with customisation at the core of it. Would be interesting to see if your use case fits