disconcision

i maintain a codebase that uses Bonzai (https://github.com/hazelgrove/hazel/); agent based development works just fine, though probably a bit more token churn

not exactly the same, but worth noting that in a spectacular display of being too early, microsoft shipped this 30 years ago (active desktop in 1997 merged the windows explorer with internet explorer, turning folders into web pages).

i assert that by your evidentiary standards humans don't reason.

presumably one of us is wrong.

therefore, humans don't reason.

all three are isomorphic. but in some languages if you define a function via something like `function myFun(x: Int, y: Bool) = ...` and also have some value `let a: (Int, Bool) = (1, true)` it doesn't mean you can call `myFun(a)`. because a parameter list is treated by the language as a different kind of construct than a tuple.

speed of plane is about 3% speed of satellite so i wouldn't expect handoffs to be much more frequent than with stationary receivers?

i can't speak for everyone, but in the above assertion am using 'roughly exponential' to mean that world population between say 1880 and 1960 followed a curve that is, roughly, exponential, in the technical sense. much of the discourse 50 years ago, e.g. the population bomb, was predicated on this observation.

we had nearly 2 centuries of roughly exponential growth. ended in the 60s but not everyone got the memo

> so no operator precedence seems like a massive foot-gun

how do you mean? given that spec, ambiguous code just won't compile. that could potentially be inefficient, but not a foot gun.

not obvious to me this makes things better as opposed to worse? sure, the time bound helps but in the runup to a crunch won't we get vastly more devices in causal range at an asymptotically increasing rate?

never thought about it before but after playing with it a while i notice i tend to approach from the right, which means moving out if i'm inside on the right side. i think this is because my positioning accuracy seems to be higher moving leftwards than rightwards...

for better or for worse, most specific scientific jargon is already going to be in english

I've yet to be convinced by any article, including this one, that attempts to draw boxes around what coding agents are and aren't good at in a way that is robust on a 6 to 12 month horizon.

I agree that the examples listed here are relatable, and I've seen similar in my uses of various coding harnesses, including, to some degree, ones driven by opus 4.5. But my general experience with using LLMs for development over the last few years has been that:

1. Initially models could at best assemble a simple procedural or compositional sequences of commands or functions to accomplish a basic goal, perhaps meeting tests or type checking, but with no overall coherence,

2. To being able to structure small functions reasonably,

3. To being able to structure large functions reasonably,

4. To being able to structure medium-sized files reasonably,

5. To being able to structure large files, and small multi-file subsystems, somewhat reasonably.

So the idea that they are now falling down on the multi-module or multi-file or multi-microservice level is both not particularly surprising to me and also both not particularly indicative of future performance. There is a hierarchy of scales at which abstraction can be applied, and it seems plausible to me that the march of capability improvement is a continuous push upwards in the scale at which agents can reasonably abstract code.

Alternatively, there could be that there is a legitimate discontinuity here, at which anything resembling current approaches will max out, but I don't see strong evidence for it here.

I actually did something similar recently for my website* and it was actually a Claude failure case; helped a little but ultimately cost me more time than doing it entirely myself. For my version I wanted the actual book spine images; since there's no database for these, I took pictures of my shelves (milkcrates) and wanted clickable regions. I also wanted the spines to link to a reasonable webpage for the book, falling back to goodreads if nothing else was available.

It was very surprising to me that claude wasn't very good at even the latter. It took several rounds of prompt refinement, including very detailed instructions, to get even 75% non-broken links, from ~30% on first attempt. It really liked to hallucinate goodreads URLs with the book title in them but an invalid ID (the title part of URLs is ignored by goodreads).

The former was less surprising... It attempted a very rough manual strategy for generating superimposed SVG outlines on the books, which often started okay but by the right side of the the image often didn't even intersect the actual spines. I tried to verbally coax it into using a third party segmenter, but with no luck. We eventually found a binary search style iterative cropping strategy that worked well but was taking 6 minutes and nearly a couple dollars per spine so I killed that and just did the SVG outlines in figma myself.

* https://andrewblinn.com scroll down to find the bookcase and drag down on it to zoom and unlock the books

with the exception that it doesn't seem possible to fully disable this for grok 4

you can (via the api, or to a lesser degree through the setting in the web client) determine what tools if any a model can use

in defense of my upvote i copy-pasted the title into google in a new tab and then forgot that i had done so

wait until you find out about human programmers...

this is somewhat true but i'm not sure how load bearing it is. for one, i think it's going to be a while until 'we asked the model what bob said' is as admissible as the result of a database query

OKLCH plus relative colors let me seriously reduce the amount of hardcoded colors from my style sheet:

https://github.com/hazelgrove/hazel/blob/dev/src/web/www/sty...

interesting!! is BABLR yours? is there a page that talks about your error-correcting/embedding-gaps approach? couldn't immediate find one in the docs. btw new version of tylr (hazel's core editing engine) is multi-language from the ground up, with a javascript grammar underway; new version is not yet online though

there are approaches to ensure you always or almost always have syntactic validity, e.g. structured editing or error correcting parsing, though of course it's true that the more permissive the system is the more tortured some of the syntactic 'corrections' must be in extreme cases. the approach we're taking with http://hazel.org (which we approximate but don't fully succeed at yet) is: allow normal-ish typing, always correct the code enough to be ran/typechecked, but insert placeholders/decorations to telegraph the parse structure that we're using in case of errors or incompleteness.

i feel it's not quite that simple. certainly the changes you suggest make the paper more straightforwardly defensible. i imagine the reason they included the problematic assertion is that they (correctly) understood the question would arise. while inserting the assertion unsupported is probably the worst of both worlds, i really do think it is worthwhile to address.

while it is not realistic to insist every study account for every possible objection, i would argue that for this kind of capability work, it is in general worth at least modest effort to establish a human baseline.

i can understand why people might not care about this, for example if their only goal is assessing whether or not an llm-based component can achieve a certain level of reliability as part of a larger system. but i also think that there is similar, and perhaps even more pressing broad applicability for considering the degree to which llm failure patterns approximate human ones. this is because at this point, human are essentially the generic all-purpose subsystem used to fill gaps in larger systems which cannot be filled (practically, or in principle) by simpler deterministic systems. so when it comes to a problem domain like this one, it is hard to avoid the conclusion that humans provide a convenient universal benchmark to which comparison is strongly worth considering.

(that said, i acknowledge that authors probably cannot win here. if they provided even a modest-scale human study, i am confident commenters would criticize their sample size)

it brought my 16-core desktop with 24gb GPU to a crawl, perhaps the only time a website has done that. impressive tbh

this might work https://nitter.space/disconcision/search?q=from%3Adisconcisi...

unfortunately the best source for examples and screenshots at the moment might be a search of my twitter feed: https://x.com/search?q=from%3Adisconcision%20hazel ... we need to update the website

llm hole filling ala the paper is actually live in the dev version right now (if you enter an openrouter API key in the second sidebar tab). it's slow and buggy at the moment though, it's only been running at all for the last few days

i'm not convinced takeover is a fair description. its true that functional languages are over-represented in the more theory-laden subfields, but this is in on small part because functional languages are easier to reason about. 'reason' here is not intended in any prescriptive sense, just in the sense that they are more mathematically circumscribed. proving anything in an imperative language is going to mean proving it in the purely functional fragment of that language first and then drawing the rest of the owl, so when someone wants to try something new they're probably going to try it in mice first.

> used to remark that the the Matrix is the kind of movie the matrix would make to think you've won. The Wachowskis who are very American did not understand S&S

to be fair this is explicitly a theme in the (imo unjustly maligned) sequels

this is not quite right. haskell and scheme have if expressions, not if statements. that's not the same as if being a function. if is not, and cannot be, a function in scheme, as it does not have scheme function semantics. specifically, it is not strict, as it does not evaluate all its subexpressions before executing. since haskell is non-strict, if can be implemented as a function, and iirc it is

Here's a similar example using lean widgets (in the live programming part of the talk): https://www.youtube.com/watch?v=c5LOYzZx-0c