I've been using self hosted langfuse via litellm in a juptyer notebook for a few weeks for some synthetic data experiments. It's been a nice/useful tool.

I've liked having the traces and scores in a unified browser based UI, it made sanity checking experiments way easier than doing the same thing inside the notebook.

The trace/generation retrieval API was brutally slow for bulk scanning operations, so I bypassed it and just queried the db directly. But the is the beauty of open source/self hosted code.