506 karma · joined May 31, 2016
Not for any neutrino experiment I know of. There are paper committees that govern publication but the actual preprint submission to arXiv is always done by one of the authors.
> I would think arXiv can work something out for these cases.
Yes, I also expect special cases would be defined.
- Keep diagram source (eg Graphviz dot) with the document.
- Write software developer manuals that can reference code robustly as the code changes (eg include code snippets from source based on regex).
- Literate Coding / Reproducible Research pattern. Eg, org file explains and generates/builds/runs code and graphs/figures which then get included back into the document.
Some of the problems
- At some scale, one needs a DAG (eg make) or to make every command idempotent with fast no-op. Otherwise, each little change to a source block takes too long and there is a worry that something didn't update which should.
- At some scale, the document becomes the size of a library/package and it does not compose and/or I want to run the code outside of the document.
- The unenlightened around me do not use Emacs so an org file is a "me" file. In some ways this is a plus as it keeps others fingers out of my pie but it also means no way to share the baking.
That was in the days before LLMs. As ellieh's #3 points out, things are different now (we all know that). I now have an LLM externalize org-babel. The LLM maintains an org or LaTeX document describing bits of work, an external library/CLI which runs to produce content including putting numerical results into LaTeX macros, a Makefile or Snakemake to regenerate content and figures. When things are found to change prior understanding the LLM remakes a section of prose in the LaTeX document. I then write my own notes or another LaTeX document so that the trip through eyeballs to fingers on the keyboard assures I keep some level of understanding.
Likewise, in the software documentation goal, it's far better to give a good LLM access to the source and have it generate documentation targeting some learning goal with follow exploration via Q&A than it is to read some prepared document that assumes my goal. Software documentation is kind of a relic useful only for those people that have not yet taken up LLM tools.
Posted. Having a preprint on arxiv is not publishing.
Many of us solve our UI problems and we land into our own happy minima.
Many of the UX "solutions" are either invented problems or not my problems or worse, grinding some anti-me agenda.
Like many, a lot of my UX improvements have been to simplify the UI. Tiling, with just a few windows per virtual desktop in herbstluftwm: kitty, emacs, browser and some rare transient application (usually PDF or image viewer).
The idea in the screen shot horrifies me. Why would I buy an expensive high pixel monitor just to use only 10% of it to display a single window? Why would I want to waste CPU to animate the migration of a full window to a single button icon (another example given)?
These UI "innovations" are great for movie props but do they actually help in reality? Not that I see. The "stagnation" of UI to me is more a plateau. Anything truly new must be driven by new "I", no input/output channels between user and computer.
The one interesting idea I saw in the article was Lifestreams. I think storage and recall is something that needs improvement and new ideas because for me that is a problem. I produce and consume too much info while recalling it is hard and while also I often do not produce or consume info that I wish I had.
Again, it's the I/O that drives the progress.
And when there are more than one transmitters in real world conditions instead of anechoic chamber?
It's cool and all and if this work is for it's own sake, for the sake of research, no issues. But otherwise, I struggle to think of practical use cases. I grant that my imagination may be deficient.
> the NVIDIA B200 achieves 1.6× to 2.3× higher intelligence per joule than the APPLE M4 MAX across QWEN 3 and GPT-OSS model variants
The B200 = "cloud", M4 = "local".
So "cloud" does even better in energy than it does in power compared to "local". Or, to flip it, "local" is both slower and more expensive than "cloud".
You misread it. From the abstract:
> local accelerators achieve at least 1.4× lower IPW than cloud accelerators running identical models
That's "intelligence per watt". They also have IPJ, per Joule.
So, they find local is 40% "dumber" than cloud for the same power or 40% more power for the same "intelligence".
Tables 13 and 14 summarize their IPW and IPJ metrics.
But, to your actual point, I think the "local is 40% dumber per watt than cloud" message is still an understatement. And maybe this is something I failed to find in the paper but they seem to ignore the "idle baseline" costs and talks about explicitly focusing on the power consumption of just the accelerator under load.
There is a large baseline power consumption just to support the accelerator. CPUs, memory, PS losses, network, fans, general environment cooling. This "cost floor" is different for data centers and a "random local computer" and I think must be in favor of data centers which are designed and built with efficiency in mind.
Idleness should also be considered. My local GPUs at $WORK and home are idle more than they are used. Idle time energy in real world scenarios should be somehow attributed to those brief, punctuated times when LLM functions are actually active on the accelerator. Actual, local LLM usage of a GPU is brief (assuming one user per PC). Even with my heavy usage developing s/w I'd guess I heat up a GPU about one hour per day total, sometimes much less. If that is local then one must pay 23 hours of idleness for that 1 hour of "intelligence". Of course a local PC is used for other things and the idleness penalty must somehow account for that. OTOH, data centers try to maximize utilization so their idle time penalty would be much less, perhaps close to zero, by construction.
- "no idea what you’re talking" = "you have outsourced expertise to an LLM"
- "solution that doesn’t actually work" = "piecemeal blocking leaves many problems still active"
- "used to be free" = "you / others in general now pay LLM services for the outsourcing leading to poor solutions while in the past poor solutions came for free"
Your observation on the readership bias of HN is a good one for people to add to their HUMANS.md before reading and commenting. :)
I agree. Other reasoning traces simonw quoted in his blog post showed that the model made changes to consider realistic fork rake. I think this may also be the first case where the chain went inside the seat stays. The overall bike geometry is still comical but these bits show improvement in this model over prior ones.
On the other hand, given the absurdity of the original prompt, I should not necessarily expect realistic bike geometry.
The article touches on Datalog statements changing over time. One ingredient I think would be good to add to the system is to make every statement carry "providence" metadata. The providence should be sufficient to enable later confirmation that a statement is still valid or if the statement needs to be reformed without the need to remake the entire graph from scratch.
I would make at least some forms of providence follow a strict schema that is defined for the subject matter that is being captured. For example, statements about a code base should refer to the source files and their version (file modification date, content hash) from which the statements were concluded. When a source file is modified we may then find all statements made from them and reevaluate just those statements.
The next level would be to keep statements even if reevaluation breaks them and add a method to derive a subgraph for a given state of the subject. For example, over many releases of a code base, a lot of statements would not change, some would. Having a graph that spans all conclusions about all releases of a code base and a way to form the subgraph for a specific release would allow the system to efficiently target queries for a particular release.
...[finishing with], though you *can* also display images in a terminal, including in Emacs running in a terminal.
Relative to other results I agree. But on an absolute measure, there is not a single element in the current bicycle that is real-world accurate and many elements are omitted or non-physical (eg, the transparent seat tube top, entire lack of a head tube).
Consider a series of followup benchmarks.
With a fresh context of the LLM under test, ask it to generate a list of findings for how the pelican-on-a-bicycle SVG that was produced is inaccurate compared what the real world scene might appear, accepting for the limitations of SVG as a medium. Then, feed back the list of findings to the original context for a second try. The benchmark can stop here by humans looking at the result and forming their own conclusion.
Next phase is to repeat the analysis phase using the 2nd context to determine what findings were satisfied and what new inaccuracies are found. These two differences can form a second benchmark.
Last phase is to iterate with the goal to drive the number of findings to zero.
I wrap bubblewrap in a script that supports config files to allow different "profiles" of use (analogous to eg firefox profiles). The bwrap starts with the whole filesystem mounted read-only, then mounts the current directory read-write and then applies further bind mounts for devices, special case other read-write (eg, ~/.cache/) and to mount empties to cover sensitive directories (eg, ~/.ssh/). The profile also specifies the default command to run and for claude, it gets yolo mode.
I thought maybe NERSC doesn't accept jobs from private corporations but I checked and that's not true, as long as results are not held proprietary.
Perhaps ANL was used as they have a lot of compute and they lead and host the Genesis Open Models Initiative?
From your link, 2048 B300 GPUs were used for 6 months. If google search is right, NERSC has 7168 A100. B300's are way more capable than A100. To do this training in 6 months, "3 NERSCs" would be needed.
Between the political angle and the technical, I'd guess these two make up a big chunk of the answer.
US national lab scientists are not even civil servants. The labs themselves are run by a corporation under contract to the DOE and the scientists work for that corp. The managing corporation changes from time to time and the scientists transparently start working for whatever assumes the replacement. The land, the hardware, the buildings and any physical products are owned by the US gov't. To a very large extent, the intellectual output is set free to the world in the form of papers, presentations and to some small extent (eg compared to CERN) in the form of software.
I find this depends on the LLMs being used and the person using them and the problems being solved.
Like, Fable can be sent off to do some big thing for a long while and I find myself getting bored and move over to push on some other thing. Or, while reading a paper I have a string of follow up questions and ideas and launch them via web or CLI agent. Or I bounce between multiple chores in different packages, each of which is fast for an LLM and a low cognitive burden for me. Then other times, something needs my full, ongoing and serial attention where the LLM turn is only a minor element. I'll use the brief LLM interludes to get up, walk around a bit, stretch and think.
As for Herdr, it seems nice when I tried it a few weeks ago. But, I'm surprised it is getting so much attention (kudos). Like many people, I've developed more than one work-alike before Herdr hit the scene. They were based on tmux which I have concluded I simply detest. In the end, I made a little agent hook script that speaks to Kitty terminal and that plus Kitty and a stable SSH persistent connection gives 80% of what I was looking for. I do still like the "control panel" aspect of my past and herdr's tries. Adding that would get me another 10%.
:)
The OP is not sitting in a one-way flow of info:
LLM->human->code
LLM<-->human<-->code.
I'd argue that anytime we drive an LLM through more than one turn (and/or more than one session) we are really doing a human-is-the-loop thing. The OP's extreme case of begin the only thing editing code is on a spectrum with the other extreme being vibe coding (never looking at output code, but still interacting with that output in some way).
Off the spectrum is what I call LLM-vomit. A human one-shots something and puts it out for others to see, suffer and clean up or ignore. This is code that is encountered literally out of context and can only be further improved (if that is even attempted) by approaching it from first principles. Such code is akin to people using LLM to generate an answer delivered to another human. Both flavors (sorry) of LLM-vomit are bad. I can prompt my own LLM to do that, don't do it for me.