To be fair there's probably a considerable amount of engineering that went into evaluating those markdown files so the agent behaviour is statistically reliable. The markdown is the product, not the process
If those skills were unreadable, however, that would imply proper engineering - like e.g. the skills themselves being an output of iterative RL over set of evals.
I'd put it like this: regardless of the merit of how they're applied, it would at least demonstrate possession of the advanced skills expected of experienced software engineers.
Start from scratch, do some test runs, find the bugs, add the minimal possible text to avoid the bug, iterate
You can get 95% of my impl workflow skill by just telling Claude "split the work into slices and use ephemeral subagents" and the other 5% takes like 10x as much text to achieve
Overall, this whole approach to programming seems to align really well with the original premise of COBOL. I wonder when people will start putting
# Identification Division
into their Markdown files.
Not trying to be snarky. I genuinely don't get it
Write a new prompt, evaluate, it now succeeds 99% of the time. Measure what changes between prompt #1 and prompt #2, understand what contributed to the performance jump.
Write a third prompt, this one succeeds 100% of the time. Increase the size of your evaluation set, find a 1/5000 error-class and a 1/10000 error-class, add some explicit code to correct for this cases.
Roll out to production, collecting usage metrics. You make some tweaks to your harness, your prompts. Eventually you have confidence that your system has fewer mistakes than 1 in 100k.
Now, multiply this iteration across all your different prompts and different ways that they might interact with one another.
Typing is not a noisy source. It should be reliable and deterministic.
Protecting an agent from fairly obvious attacks should also be deterministic.
Real world is probabilistic in practical / metrological, if not fundamental sense, and separation between data and execution does not exist. Our reality does not support such separation.
> a probabilistic/stochastic word generator on top of deterministic circuits instead at 10 magnitude worse efficiency
It's 10 magnitude better efficiency end-to-end, if you factor in design time you'd have to spend to get your "deterministic circuits" (which really aren't, we just paper over it) into shape so they deterministically solve a specific problem, for each problem you want to solve - where with the "stochastic word generator", you just need to change the text prompt.
Still it can be a software engineering if the gradient candidate / measuring gradient / repeat process can be done at scale.
https://news.ycombinator.com/item?id=44978319
"Most classical engineering fields deal with probabilistic system components all of the time. In fact I'd go as far as to say that inability to deal with probabilistic components is disqualifying from many engineering endeavors."
Reading comprehension 101 is a prerequisite for doing engineering, too.
Engineering is literally the art of making educated guesses and then testing/proving/disproving/improving upon them. Nothing is exact. Everything is approximate. Iterate until the result is good enough.
Engineers don't just tie a sheet to a rock and throw it off a cliff and call themselves aerospace engineers.
They do full diligence on the theory, math, physics, material science, fluid dynamics, etc, and plan a controlled series of tests specifically designed to verify/challenge/disprove their concept and the theories behind it.
Sure, there's a team member ultimately responsible throwing half a dozen rocks off a cliff in the first test.
A technician.
The guy who throws the rock off the cliff is a technician.
Engineering often continues until the concepts and theories are developed into safe, practical methods. "If you stay within these parameters, you can confidently expect these results." The reliability can be codified and reproduced without going from first principles on every application of it.
It's not clear to me that the current AI fad is really developing such reproducible, safe methods. "If you stay within these parameters, you might get these results. Or a teapot. Or some subtly misleading fabrication."
You have to do full due diligence to validate every result. There is safe usage where the hard work was done up front so that day to day practice can skip to boring and reliable application.
People who think it's a system that thinks and reasons have confused the agentic harness, perhaps forgotten(?) layer0[0] is a seed, the inference engine sets to a concrete value when the caller leaves it as 0.
They probably work on (current) AI software by repeatedly writing prompts like "DON'T READ THE FILES IN /tmp. SOME OF THE FILES IN /tmp ARE VERY LARGE. DUE TO THEIR SIZE, YOU ARE NOT TO READ THE FILES IN /tmp." and wondering why the model becomes obsessed with files in /tmp 100k tokens into every conversation.
more like 10%.
> Write a new prompt, evaluate, it now succeeds 99% of the time.
in theory, maybe. In practice still 10%.
> Write a third prompt, this one succeeds 100% of the time
Only in Sam Altman's wet dreams.
> Consider Chanel's advice: before leaving the house, take a look in the mirror and remove one accessory. Human creatives have memory and always try to do something new, so if you have a space to quickly jot down notes about what you've tried, it can help you in future passes.
How about "canvas design"?
> THE ESSENTIAL PRINCIPLE: The topic is a subtle, niche reference embedded within the art itself - not always literal, always sophisticated. Someone familiar with the subject should feel it intuitively, while others simply experience a masterful abstract composition. The design philosophy provides the aesthetic language. The deduced topic provides the soul - the quiet conceptual DNA woven invisibly into form, color, and composition.