Hacker News Reader

Comment by simonw | Hacker News Reader

I've been having an interesting challenge relating to this recently. I'm trying to calculate costs for LLM usage, but the amounts of money involved are so tiny. Gemini 1.5 Flash 8B is $0.0375 per million tokens!

Should I be running my accounting system on units of 10 billionths of a dollar?

View on HN

Fixed point Decimal is your friend here. I’m guessing you buy tokens in increments of 1,000,000 so it isn’t too much of an issue to account for. You can then normalise in your accounting so 1,000,000 is just “1 unit,” or you can just account in increments of 1,000,000 but that does start looking weird (but might be necessary!)

No, billing happens per-token. It’s entirely necessary to use billionths of a dollar here, if you don’t use floating point.

In which case, I’d look at this thread https://news.ycombinator.com/item?id=44145263

Accounting happens on the unities people pay, not the ones that generate expenses.

But you probably should run your billing in fixed point or floating decimals with a billionth of a dollar precision, yes. Either that or you should consolidate the expenses into larger bunches.

You're better off representing values as rationals; a ratio between two different numbers. For example, 0.0375 would be represented as 375 over 10000, or 3 over 80

Sounds hard to model in SQLite.

Two columns?

From Forth, here's how I'd set the rationals:

    : gcd begin dup while tuck mod repeat drop ;
    : lcm 2dup \* abs -rot gcd / ;
    : reduce 2dup gcd tuck / >r / r> ;
    : q+ rot 2dup \* >r rot \* -rot \* + r> reduce ;
    : q- swap negate swap q+ ;
    : q\* rot \* >r  \* r> reduce ;
    : q/ >r \* swap r> \* swap reduce ;

Example: to compute 70 * 0.25 = 35/2

70 1 1 4 q* reduce .s 35 2 ok

On stack managing words like 2dup, rot and such, these are easily grasped under either Google/DDG or any Forth with the words "see" and/or "help".

as a hint, q- swaps the top two numbers in the stack, (which compose a rational), makes the last one negative and then turns back its position. And then it calls q+.

So, 2/5 - 3/2 = 2/5 + -3/2.

Convert to money as late as possible

This is surely the right answer: simply count the number of tokens used, and do the billing reconciliation as a separate step.

As an added benefit, it makes it much easier to deal with price changes.

Ethereum is 1e18 or 1 wei.

https://ethereum.stackexchange.com/questions/158517/does-sol...

I've used Auroa Units to do this. You can define the dollars dimension, and then all the nano-micro-whatever scale comes with.

Reply on news.ycombinator.com