If you aren’t caching LLM functions during development, then you’re an even greater glutton for punishment than the normal engineer.

My local file cache Python decorator also allows the decorator to define the hash manually, either by the decorator’s parameter function call that plucks a value from the cached function params, or by calling a global function from anywhere with any arbitrary value.

What’s cool about caching results locally to files during development is the ease of invalidating caches — just delete the file named after the function and key you want.