We all know the shortcomings of LLMs but its been interesting to observe Agency as a system and the flaws that emerge. For example, does Claude have any reason to get from point A to B quickly and efficiently? Running into walls, running in circles, back tracking, etc. Clearly it doesn't have a sense of urgency, but does it matter as long as it eventually gets there?

But here's what strikes me as odd about the whole approach. Everyone knows it's easier for an LLM to write a program that counts the number of R's in "strawberry" than it is to count it directly, yet no one is leveraging this fact.

Instead of ever more elaborate prompts and contexts, why not ask Claude to write a program for mapping and path finding? Hell if that's too much to ask, maybe make the tools before hand and see if it's at least smart enough to use them effectively.

My personal wishlist is things like fact tables, goal graphs, and a word model - where things are and when things happened. Strategies and hints. All these things can be turned into formal systems. Something as simple as a battle calculator should be a no-brainer.

My last hair brained idea - I would like to see LLMs managing an ontology in prolog as a proxy for reasoning.

This all theory and even if implemented wouldn't solve everything, but I'm tired of watching them throw prompts at the wall in hopes that the LLM can be tricked into being smarter than it is.