109 karma · joined October 2, 2022
We are using CF Worker cache under the hood https://developers.cloudflare.com/workers/runtime-apis/cache...
We might use D1 for some of our other features like rate limiting.
Two key advantages that we have over OpenAI. 1. We are open source and are trying to build a community of developers that can out run any single LLM provider. 2. We are completely provider agnostic, which allows us to aggregate and share common features across tools. (One analogy we like to use, is we want to emulate how dbt is database agonistic, and Helicone will grow to be provider agnostic)
I opened this issue up last week https://github.com/xiaoyang-sde/reflare/issues/443
I kicked off a thread here https://github.com/Helicone/helicone/discussions/164 and would love your input!
P.S. that for the <title> note