You can do it easily with object caching / function memorization patterns in any modern language which should fit your desired solution. Best of luck!

there's an open source ai gateway - https://github.com/Portkey-AI/gateway