vishvananda

I've been doing a lot of experimenting with this with a c++ compiler project[0]. I definitely experienced a lot of challenges, but I do believe it is possible to end-up in a consistent space by putting in some very careful guardrails. I covered some of the things i learned in the follow on blog posts. My more recent runs have been producing better code after continuing to improve harnesses, tests, and prompts. New blog post coming soon on v2 and v3 reruns. I'll be kicking off some v4 runs soon. Here is a teaser showing the later runs: https://storage.googleapis.com/ralph-run-viewer-zippy-960/in...

[0]: https://medium.com/@vishvananda/i-spent-2-billion-tokens-wri...

From harness? Because people expect a squishy set of things from a harness that is different from what I end up building. I end up with a rigid internal structure that the harness uses in-turn (tests with clear error messages, tools, etc.) and a matched rigid external structure that drives the turn tracking progress and deterministically handles the overall progress. You could call that whole thing a harness but that makes the definition muddy and hard to talk about. So scaffold or skeleton seems more appropriate. The harness constrains the agent. The matched endoskeleton and exoskeleton gives it structure.

I’m definitely on the deterministic code train as well. All of my success for long running tasks has been around wrapping the agentic harness (cc, codex-cli, etc.) in a deterministic workflow with deterministic gates. We need a name for this outer layer. In my mind that is the true harness because it constrains the agents failure mode. I think flow engineering has been proposed. Maybe it’s the agentic exoskeleton?

For me it was earlier this year when I started dusting off some old stalled projects and had an agent work on them. In a few days I:

* Built a clone of the Alpha Zero implementation[1] my team built at oracle

* Ported my hobby NES emulator from javascript to rust[2] (this actually took less than 30 minutes and worked on the first try)

* Implemented all of the lessons from the C++ Grandmasters Challenge (which eventually led to a complete c++ compiler[3])

The thing that flipped the switch was using it to build things that I actually put sweat-equity in to previously. I knew how hard these things were to build, so it landed in a way that other projects had not.

[1]: https://medium.com/oracledevs/lessons-from-implementing-alph...

[2]: https://github.com/vishvananda/popeye

[3]: https://medium.com/@vishvananda/i-spent-2-billion-tokens-wri...

I've been experimenting quite a bit with long-horizion agentic coding[1] and I have also noticed that agents seem to perform worse when forced into certain architectural patterns. I have found that is a bit better when including the constraints along the way instead of adding them after the fact. There seems to be a side-effect I have been calling "calcification", where a pattern starts appearing in the codebase and the agent follows the pattern to the point where it dominates the context and becomes self-reinforcing. This could potentially be a strength or a weakness for existing code bases depending the codebase quality. I will have more insights on this soon as more from-scratch runs conclude that include architectural guidance from the beginning.

[1]: https://medium.com/@vishvananda/i-spent-2-billion-tokens-wri...

I think there is a flow in most organizations from:

llm -> prompt -> result

llm -> prompt + prompt encoded as skill -> result

llm -> prompt + deterministic code encoded as skill -> result

I do think prompting to generate code early can shortcut that path to deterministic code, but we're still essentially embedding deterministic code in a non-deterministic wrapper. There is a missing layer of determinism in many cases that actually make long-horizon tasks successful. We need deterministic code outside the non-deterministic boundary via an agentic loop or framework. This puts us in a place where the non-deterministic decision making is sandwiched in between layers of determinism:

deterministic agentic flows -> non-deterministic decision making -> deterministic tools

This has been a very powerful pattern in my experiments and it gets even stronger when the agents are building their own determinism via tools like auto-researcher.

This is actually a very interesting moment to potentially overcome network effects, because more and more code is going to be written by agents. If a crdt approach is measurably better for merging by agent swarms then there is incentive to make the switch. It also much easier to get an agent to change its workflow than a human. The only tricky part is how much git usage is in the training set so some careful thought would need to be given to create a compatibility layer in the tooling to help agents along.

I share the concern based on the current productivity expectations but I did stumble across something recently which makes me feel a lot better. There was a change in the tax code that coincides with the beginning of the post-pandemic layoffs[1]. This was changed back last month by the BBB which likely means a bunch of new R&D spend for big tech. I think this is why we are seeing intense M&A activity and if we can keep the AI hype under control it will probably lead to new hiring as well.

[1]: https://qz.com/tech-layoffs-tax-code-trump-section-174-micro...

I am by no means an AI skeptic. It is possible to encode all sorts of things into instructions, but I don’t think the future of programming is every individual constructing and managing artisan prompts. There are surely some new paradigms to be discovered here. A code locking interface seems like an interesting one to explore. I’m sure there are others.

I'm really baffled why the coding interfaces have not implemented a locking feature for some code. It seems like an obvious feature to be able to select a section of your code and tell the agent not to modify it. This could remove a whole class of problems where the agent tries to change tests to match the code or removes key functionality.

One could even imagine going a step further and having a confidence level associated with different parts of the code, that would help the LLM concentrate changes on the areas that you're less sure about.

Heroku is turning the 12 factor manifesto into a community project and modernizing it. I posted my thoughts in a blog[1], and I'd love to hear what other people think!

[1] https://medium.com/@vishvananda/the-rebirth-of-twelve-factor...

This is something I’ve been arguing for for a while[1]. I called it a “Framework Knowledge Base”. I think it needs to go a bit further and include specific code examples, especially for newer bits that are not in the training set. Ultimately RAG or even fine tuning might be better than a system prompt. [1]: https://devops.com/the-rise-of-coding-assistants-superchargi...

Doesn’t this already exist as https://www.npmjs.com/package/dotenv-cli ?

I've followed Caleb on YouTube for a while, due to his MtG content. He is PhD in Optical Sciences that makes some very interesting AI art. He has created a new card game with hand curated AI art and some interesting rules aimed at solving a bunch of the problems around existing collectible card games. He wrote an essay[1] about his design goals which is fascinating. [1] https://calebgannon.com/2023/07/08/the-making-of-algomancy/

Just reran the benchmarks from 10 years ago, python is only 37X slower than C on the benchmark now, and the go version is running faster than the C version. Python still has big productivity wins of course...

Python is extremely slow for some tasks. I was surprised to discover how slow when I ran some benchmarks, despite having used python for many years at the time. It has been improving lately, but here is a blog post I made on the topic quite a few years ago that has some interesting comparisons: https://gist.github.com/vishvananda/7a2f1942d0e9ffff4093

This is definitely an intriguing line of thinking. I too am constantly appalled by the complexity we introduce into the things we build, but I suspect a lot of it has to do with human issues that can't be solved by better technology. That said, I'm very curious to see what tractor ends up looking like.

I'm surprised that neither the documentary nor the review gets into the legacy of OpenStack. I may be biased, but it seems to me that a huge amount of the success of kubernetes is directly attributable to OpenStack.

First, OpenStack paved the way for a bunch of companies to invest real money in working together to compete with AWS. Second, there was massive turnover in ~2013 in open source contributors from OpenStack to Kubernetes. I wouldn't be surprised if a good 50% of the kubernetes community was inherited directly from OpenStack.

In this particular case, some of the runs from iFit instructors are actually quite good, and it cool that it adjusts the speed and incline to match the instruction. Probably not worth the extra $$$$ but it is pretty cool. But now I also want to be able to watch regular videos. I usually walk outdoors for an hour a day to get my 10,000 steps in, and the Chicago winter makes that tough, so I'm thinking an hour of walking on the treadmill while i catch up on my favorite shows might be a good substitute.

Thank you for this. I was hoping someone had figured it out but I hadn't found this yet.

I hadn't been following the "Right to Repair" movement too closely until I discovered that NordicTrack recently locked their users out of Privileged Mode[0], which makes it virtually impossible to install 3rd party software like Netflix on my treadmill. I just want to watch a video while I exercise!

[0]: https://www.wired.com/story/nordictrack-ifit-treadmill-privi...

You could always use a pointer for this, but admittedly it is pretty ugly when compared to a true optional type:

    func f(s *string) string {
        ret := ""
        if s == nil {
            ret = "default"
        } else {
            ret = *s
        }
        return ret
    }

    func main() {
        s := "foo"
        fmt.Println("Hello, " + f(&s))
        fmt.Println("Hello, " + f(nil))
    }

While the particular issue of network namespaces and locking the os thread was fixed, there is still c code that must run before the go runtime starts to work around the issue that you cannot do some of the necessary nsenter calls once you have started additional threads. The c code to make runc work is encapsulated in libcontainer/nsenter[0]

[0]: https://github.com/opencontainers/runc/tree/master/libcontai...

Go is actually a really poor choice for the container runtime because much of the container setup cannot be done from multithreaded code[0], so it has to be done in C before the go runtime initializes. I do think rust is a better choice for this layer than C because there are still security risks, but getting rid of Go for this layer is a win. I'm not sure why RH chose to rewrite it in C rather than using rust[1].

[0]: https://www.weave.works/blog/linux-namespaces-and-go-don-t-m... [1]: https://github.com/drahnr/railcar

I was the main developer with a little support from the rest of my team. I moved on a couple of years ago and my team was RIFed a few months later, so likely they don't have anyone left to work on it.

Nice work on this! If you cut out a bunch of the features supported by other containerization systems, the code gets quite short. My first one was in C and only a couple hundred lines. I worked on a full-featured rust container runtime while I was at Oracle. It is compatible with the oci-runtime spec, so can be used as a runtime for docker or kubernetes. The most up to date fork is at https://github.com/drahnr/railcar if you want to take a look at it for any ideas.

Less years, but I concur.

There is also a rust implementation that I wrote in my time at Oracle. Unfortunately they no longer maintain it, but there is a fork with some more recent updates: https://github.com/drahnr/railcar

Unfortunately I don't remember the exact numbers, but I think it was a couple percentage points worse than we were able to get with the large models.

we did a lot of our early experimentation with small networks. I don't think we went any smaller than 5 layers of 64 filters as we mentioned here: https://medium.com/oracledevs/lessons-from-alpha-zero-part-5...