rybosome

That’s cool, I’d love to see the advanced ToolController when it’s available!

Great points about not updating priors. I also thought about it a bit more and realized that there’s a way you can largely mitigate the out-of-distribution inference requests after local tool selection, if you wanted to.

The tool use loop in an inference framework builds up history of each interaction and sends that along with each subsequent request. You could create “synthetic history”, where you send the LLM history containing the prompt, your local tool selection masquerading as though the LLM generated it, and the tool response. This would be in-distribution but still rely on your local tool routing.

If this works well enough, then I think your approach is very powerful once you’ve decided on a task and set of tools and are able to commit to training on that. Definitely want to try this myself now.

Looking forward to seeing more! I take it your substack is the best place to follow along?

Thanks for the informative and inspiring post! This is definitely cool, and I can imagine very useful.

However I do want to mention that the “recommended” flow these days isn’t to separate out a tool request in the way you have. Eg instead of asking an LLM to route a tool, extracting that, running the tool, passing output back to the LLM, etc. - you simply pass the tool definitions, prompt, structural output expectations, and let the LLM (and your caller library) manage the tool use loop.

That’s how these modern LLMs are trained in post-training, and so I suspect it’s likely you’ll get different (and potentially worse?) results in trying to subvert this with a small, local model.

It comes with all the downsides you mentioned to let the LLM do this, but is also more likely to be in-distribution, and it’s easier to compose multiple tool calls.

Anyway, thanks for sharing! I’d love to see evals on a task where it compares the result when an LLM is involved in tool selection versus when it is handed tool output only - if I’m wrong about quality degradation then there’s a lot to like about your local tool routing.

One could find a 5-minute slice of any highly successful project I’ve worked on where my actions look foolish and my tools look broken.

Isolating your analysis of this to a single unflattering interaction is intellectually dishonest; you have a bone to pick.

This appears to me to be intentional and ironic to make a point rather than in earnest.

I am interpreting this as a statement about snap judgements in an age where AI will increasingly play the role of a judge or assessor of humans.

Perhaps I’m wrong, but it seems too on-the-nose to be serious.

EDIT:

> This website just puts reductive numbers on the superficial calculations we make every day

From the website. If it is in earnest then I’d be embarrassed to have shilled for it, because I agree that the idea is stupid and gross.

A very interesting idea. I am curious about this sharing and blending of the various nets; I wonder if something as naive as averaging the weights (assuming the neural nets all have the same dimensions) would actually accomplish that?

I cannot for the life of me think of what you are referring to.

If it’s COVID-related mandates like vaccines and lockdowns, then surely it’s obvious that NASA had nothing to do with that?

There is no single issue that I can see linking all of these science organizations together. Even if it’s about budget, there are bigger targets.

Agreed, especially when in this context of training a smaller model on a larger model’s outputs. Distillation is generally accepted as an effective technique.

This is exactly what I did in a previous role, fine-tuning Llama and Mistral models on a mix of human and GPT-4 data for a domain-specific task. Adding (good) synthetic data definitely increased the output quality for our tasks.

Echoing this advice in the USA. Don’t fuck with this, the language in the laws is loose enough to prosecute almost anything security-related, even well-meaning efforts like this.

This is quite an impressive list, and many of the things ranking low in difficulty for the author would have been quite high for me. It's definitely inspiring, makes me want to dust off a toy of my own.

That said, I feel the conclusion on learning with LLMs deserves some more nuance. It really depends on how you use them.

This is a terrible prompt for learning:

> Implement this solution for me

This is an AMAZING prompt for learning:

> Please give me an overview of ELF. Focus on the highest level abstractions, more of the 'why' than the 'how'

Certainly one can argue that it takes something away, in that you did not have to conduct the research yourself whenever a question occurred. However, if you are engaging in honest intellectual work (i.e. really thinking), then having a Socratic teacher always ready to discuss your questions is a profound accelerant to the learning process.

You clearly have an emotional connection to people that you feel are being harmed by AI, so I’m not going to gaslight you about that.

But I will tell you that real humans asking private, real questions of LLMs is also happening, and these two things aren’t related. Many people who don’t have the technical literacy to understand the implications are sending messages with extremely sensitive personal, medical, and financial information.

Straw-manning all of these users as tech-savvy, horny IP thieves is ridiculous. I could find your argument more persuasive if you actually considered the privacy needs of the people who had nothing to do with building or perpetuating the systems.

EDIT: to be clear I’m not sure what I think the solution should be, as I also understand the need for discovery.

I was wondering if it is function overloads of the same name, defining it non-variadically:

add_function(PythonObject)

add_function(PythonObject, PythonObject)

add_function(PythonObject, PythonObject, PythonObject)

There was a similar pattern in the Guava library years ago, where ImmutableList.of(…) would only support up to 20 arguments because there were 20 different instances of the method for each possible argument count.

The first response definitely wasn’t. It laid out the hazards in great detail, then asserted the likelihood of making it implausibly far. I poked at that conclusion and it backed off until we arrived at shear winds in the outer edges, where I agreed with the analysis that this would be lethal.

It was a thought provoking conversation, regardless of whether it was absolutely accurate.

I’ve wondered about this a lot.

More so the grim question: if you were in a typical space suit sitting in a ship just outside Jupiter, then propelled yourself towards the planet - what would kill you first?

Assume you are close enough that from the moment you are launched out, you are “in” the atmosphere at the outer edges. Also assume moving fast enough that the answer is not “dying from dehydration”.

I discussed a bit with GPT 4o and came to the conclusion that shear wind gusts of over 300mph in the upper atmosphere would probably do it. You’d hit that almost instantly, before high pressure, temperature or highly corrosive materials.

Global dependencies were disallowed back in 2018 with a tiny handful of exceptions that were difficult or impossible to make fully regional. Chemist, the service that went down, was one of those.

Generally GCP wants regionality, but because it offers so many higher-level inter-region features, some kind of a global layer is basically inevitable.

Must be hell inside GCP right now. That was a big outage, and they were tired of big outages years ago. It was already extremely difficult to move quickly and get things done due to the reliability red tape, and I have to imagine this will make it even harder.

Exactly the question on my mind. I have an ongoing fantasy of a super powerful, private inference server sitting in my closet that I can throw at stuff like this.

The honest answer is that we are still figuring out where to draw the line between an agent and a human, because that line is constantly shifting.

Given your example of the resume screening application from the post and today's capabilities, I would say:

1) Should agents decide when to create a job post? Not without human oversight - proactive suggestion to a human is great. 2) Should agents design the job description itself? Yes, with the understanding that an experienced human, namely the hiring manager, will review and approve as well. 3) Should an agent email an opening round of questions to the candidates? Definitely allowed with oversight, and potentially even without a human approval depending on how well it does.

It's true that to improve all 3 of these it would take a lot of work with respect to building out the tasks, evaluations, and flows/tools/tuned models, etc. But, you can also see how much this empowers a single individual in their productivity. Imagine being one recruiter or HR employee with all of these agents completing these tasks for you effectively.

EDIT: Adding that this pattern of "agent does a bunch of stuff and asks for human review/approval" is, I think, one of the fundamental workflows we will have to adapt in dealing productively with non-determinism.

This applies to an AI radiologist asking a human to approve their suggested diagnosis, an AI trader asking a human to approve a trade with details and reasoning, etc. Just like small-scale AI like Copilot asking you to approve a line/several lines, or tools like Devin asking you to approve a PR.

AI code generation tools work like this.

Let me reword your phrasing slightly to make an illustrative point:

> so for an employee, instead of specifying explicit if conditions, you specify outcomes and you leave the human to figure out what if conditions apply and how to deal with them?

> Are there any serious applications built this way?

We have managed to build robust, reliable systems on top of fallible, mistake-riddled, hallucinating, fabricating, egotistical, hormonal humans. Surely we can handle a little non-determinism in our computer programs? :)

In all seriousness, having spent the last few years employed in this world, I feel that LLM non-determinism is an engineering problem just like the non-determinism of making an HTTP request. It’s not one we have prior art on dealing with in this field admittedly, but that’s what is so exciting about it.

Adding to that excellent high level explanation of what the attention mechanism is, I’d add (from my reading of the abstract of this paper);

This work builds a model that has the ability to “remember” parts of its previous input when generating and processing new input, and has part of its intelligence devoted to determining what is relevant to remember.

This is in lieu of kind of saying “I need to keep re-reading what I’ve already read and said to keep going”.

I’d welcome better explanations. :)

I have a very clear memory of offending someone with the use of the word “crap” years ago.

As a kid I worked in a restaurant that sold Cincinnati-style chili - noodles with sweet chili and cheese on top. We were encouraged to offer customers who ordered a plain bowl of chili this noodle concoction instead.

Late one night, I had a customer order a bowl of plain chili. I gave her the spiel I was supposed to, suggesting that she try the noodle dish. She said, “so you won’t sell me a bowl of chili?”. I replied, “sorry for the confusion ma’am, I am happy to sell you chili. We are asked to say this crap because management is worried customers don’t know what they want”. She replied, “I don’t think it’s appropriate for you to use the word ‘crap’ with me”. I apologized again, gave her her order, then was removed from my position 3 days later when she emailed management to complain. I had “refused to sell her chili”, and “used vulgar language”.

Scale has also built massive amounts of proprietary datasets that they license to the big players in training.

Meta, Google, OpenAI, Anthropic, etc. all use Scale data in training.

So, the play I’m guessing is to shut that tap off for everyone else now, and double down on using Scale to generate more proprietary datasets.

A wonderful approach generally and something we also do to some extent, but not a substitute for fine-tuning in our case.

We are working in a domain where there is very limited training data, so what we really want is continued pre-training over a larger dataset. Absent that, fine-tuning is highly effective for non-NLP tasks.

Yep!

So my use case currently is admittedly very specific. My company uses LLMs to automate hardware design, which is a skill that most LLMs are very poor at due to the dearth of training data.

For tasks which involve generation of code or other non-natural language output, we’ve found that fine-tuning with the right dataset can lift performance rapidly and decisively.

An example task is taking in potentially syntactically incorrect HDL (Hardware Description Language) code and fixing the syntax issues. Fine-tuning boosted corrective performance significantly.

Haven’t tried it personally, as this was a use case where a classic SFT was effective for what we wanted and none of us had done LoRa before.

Really interested in the idea though! The dream is that you have your big, general base model, then a bunch of LoRa weights for each task you’ve tuned on, where you can load/unload just the changed weights and swap the models out super fast on the fly for different tasks.

I think the point the author misses is that many applications of fine-tuning are to get a model to do a single task. This is what I have done in my current role at my company.

We’ve fine-tuned open weight models for knowledge-injection, among other things, and get a model that’s better than OpenAI models at exactly one hyper specific task for our use case, which is hardware verification. Or, fine-tuned the OAI models and get significantly better OAI models at this task, and then only use them for this task.

The point is that a network of hyper-specific fine-tuned models is how a lot of stuff is implemented. So I disagree from direct experience with the premise that fine-tuning is a waste of time because it is destructive.

I don’t care if I “damage” Llama so that it can’t write poetry, give me advice on cooking, or translate to German. In this instance I’m only ever going to prompt it with: “Does this design implement the AXA protocol? <list of ports and parameters>”

I wish that I had the deep biology knowledge to make a joke (not joke?) Large Mushroom Model based on actual signal data.

This also makes me wonder about the idea of attempting to expand on this research by artificially stimulating a fungal colony with the various “words” and seeing if different effects can be produced.

This is a great point - communication is a vast array of information channels. “Motion” is communication whether or not it evolved to be, and so is every other aspect of occupying space in a biome.

And given that the organism under scrutiny is fungi - which contains many sprouting heads of the same underlying organism - one would have to have to have no familiarity with life at all to be surprised by intra-organism communication.

Yeah, very much agreed that in the spirit of good discussion I should’ve at least asked about their experiences and use case before jumping to finger wagging.

But that said, let me reiterate a couple important points from my post:

> With no disrespect meant

I’m not calling anybody an idiot because they aren’t using an LLM. I’m sharing my honest opinion that they’re not using it correctly, but that’s very different than calling them an idiot.

> if you’re unable to find utility in these tools

This is a bit lawyerly, but note my carefully generic wording here: “find utility”. If you’re a Rust developer who doesn’t like the Rust output from your LLM, sure - but that’s not 100% of the job.

You’ll also touch bash scripts, make files, YAML, JSON or TOML config, write bug reports/feature requests, discuss architectural ideas and coding patterns, look through stack traces/dumps/error logs, or whatever else.

My point is that it is exceedingly unlikely that there is nothing an LLM can do to help your work, even if it’s not good at writing code in your domain.

Hence the statement that if you cannot find utility, you’re not using it correctly. It takes time to learn how to use these tools effectively, even in domains they excel in.

Many, many people are in fact “using the code it generated on its own”. I’ve been putting LLM-assisted PRs into production for months.

With no disrespect meant, if you’re unable to find utility in these tools, then you aren’t using them correctly.

I’ve spent some time optimizing Python performance in a web app and CLI, and yeah it absolutely sucks.

Module import cost is enormous, and while you can do lots of cute tricks to defer it from startup time in a long-running app because Python is highly dynamic, for one-time CLI operations that don’t run a daemon or something there’s just nothing you can do.

I really enjoy Python as a language and an ecosystem, and feel it very much has its place…which is absolutely not anywhere that performance matters.

EDIT: and there’s a viable alternative. Python is the ML language.