2,564 karma · joined June 4, 2020
jkalstad@santurcesoftware.com - write HN in the subject and i'm infinitely more likely to find you!
Founder of FlaskTrack (https://flasktrack.com). FlaskTrack unifies ELN, LIMS, lab workflows, instrument connectivity, AI agents, and compliance in one platform, replacing the fragmented stack of software and manual data entry that slows research teams down.
Not nearly as sophisticated as myself who would mutter "When in doubt - Charlie out" before marking C.
thanks
For anyone who wants to dork around there is https://github.com/rasbt/LLMs-from-scratch which is something amazing that I think anyone who wants to engineer things around LLMs should at least blast through and read.
If I have a user input and then sanitize and inject that into a prompt to do something, I have no idea how much that is going to cost at all and no real way to measure this properly. A parallel example is digital ocean or aws, i can go and measure/limit my compute/fs/memory/startup times/etc and while it can be impossible to get down to the last flop of money allocated - i can run things on a real budget with real constraints, opposed to an LLM where I have to .. prerun a sanitized user prompt through a tokenizer and then ask an LLM to guess what it may do and give token consumption estimates and then act on those in any sane manner for the user?
Perhaps i'm missing something to do realistic and static rails on things but I don't see a serious way at scale to use the token billing model handling things requiring a users free text input short of having to go pander to VC money to throw money at it until someone else figures it out.
*to clarify my rambling... We should be billed and given controls based on resource usage itself and not an opaque token concept on top of not being able to spin any knobs that control it's resource usage.
It has worked great but i've spent more time beating LLM output into parseable output than I have reading and appreciating the prose it sends when i'm asking it something about some snippets of code.