Thanks for the clarifications. Token summation is a cool optimization :)
Query optimizers in SQL database systems typically optimize based on the time to execute the function on a local server. The token summation optimization generalizes time-based optimization of local functions to dollar-based optimization for remote functions.
Execution Time-based optimization: FunctionFoo(input 1) = 2x FunctionFoo(input 2)
Dollar-based optimization: ChatGPT(prompt with 100 tokens) = 2x ChatGPT(prompt with 50 tokens)
We are also exploring dollar-based optimization in EVA, and will check out your openai-to-sqlite tool for ideas [1].