helping build https://portkey.ai/ where we try to take care of storing all prompts, responses, erros, costs, latencies etc and make it available to debug.
Thanks for the link! I skimmed the articles on the blog, but couldn't find one describing the monitoring architecture behind all. Do you have any public references for that?
Not yet, but it's on the roadmap to publish. Broadly, we use clickhouse, streams, mongo, cloudflare workers as the mainstay of our monitoring infra.