jamiejquinn

Appreciate the looking good aspect but surely the graphics community also appreciates some level of consistency...?

Damn, coming from the CUDA world, this is quite unexpected. Do you know of any resources on FP in the graphics hardware? I mean do they fit A standard (nvfp, bf, etc?)?

Applied mathematician here.

Where I see models having a huge impact is in simulation code development.

One blocker for years now has been the adoption of GPUs. LLMs can fairly successfully and very quickly port to GPU and suggest/implement useful optimisations. Once it's verified, a code can go from anywhere between 2x to 1000x faster (mainly because CPU codes are so poorly optimised). Some science can reach much greater problem sizes, while some can run the same problems in hours rather than months and both can be revolutionary. Even more than that, LLMs seem to be finding fundamental performance bugs in both open and closed source core libraries so there's a bit of a whole-ecosystem uplift.

Can't comment on the more theoretical, less computational applied maths impacts!

Goddamn Gauss, man.

Agreed. For a long time it felt like most governments/funders accepted that pure research is a necessary part of feeding the applied research machine. And now the funders seem particularly short-sighted.

In saying that, in the spirit of a good faith argument, is there value in temporarily deprioritising pure research in (current) times of funding drought?

"Those rethinks are load-bearing parts of your voice."

Loved the (possibly not deliberately ironic) use of the LLM'S favourite analogy, load-bearing.

Nice wee article though. I particularly agree with "Don’t take all of the model’s copyediting advice." I've been using LLMs to review teaching material and often it just doesn't get the point of teaching. It's good for prompting reflection on your writing but it's still an idiot savant.

Ahh I only do this when reinstalling or moving to a new laptop but it's good to know about Atuin for the future!

Ahh I see you've never been to a physics conference!

I've relied on this so much that I copy my entire history between machines...

Generally agree. As with many optimisations, branchless code can easily be less obvious than the branchy equivalent.

As far as I can tell (and my research was on the simulation side of Navier Stokes) the key AI output was a specific counter-example solution, generated with a method suspiciously close to that developed by the research duo involved in the controversy, a method that was discussed with Codex. So to me that insight is as insightful as the next undiscovered prime.

Why?

Plenty of scientific simulations end up being IO or communications bound when at scale (e.g. 30k CPU cores). Can hide latency to a certain degree but basically any algorithm that uses timestepping must halt at some point to allow data to flow around the nodes, or to dump data to disk for visualisation or checkpointing.

In saying that there are some novel and very clever algorithms that continue on without seemingly necessary boundary data, that then self correct when the data comes through, thus completely hiding the latency at the cost (in both accuracy and time) of running a correction process.

The broad swathe of analyses done to answer the question of "how did the economy change due to brexit?" consistently suggest brexit shrunk the economy. All analyses are biased or flawed in some way because they rely on making a prediction of what the economy would have been if brexit hadn't happened, but that's only a prediction. Since they are flawed, but they are many, the likely truth is in the aggragate of many analyses, just like when we take multiple measurements of a noisy system and take the mean to get a more certain result than any one analysis. If what you're saying is true I would expect there to be some predictions that back up what you're saying exactly, along with a number of predictions that the economy increased in size. That's why I'm asking for evidence of predictions of a larger economy, because it makes your statement far more plausible.

Your point on goods isn't... Good. Services is the majority but goods still make up ~1/3rd of exports, and don't forget we still have to import goods. Brexit affected that too.

truth of this unfortunately depends on the analysis of others, thus when those analyses . The data cannot itself show whether the

I don't think that's an honest reading. If there were a wealth of data and analyses that suggested the UK's economy is larger because of brexit I could agree but I certainly can't find any. You're right to be skeptical of the doomsayers that predicted (and continue to predict) disaster, but it seems incredibly unlikely (and unsupported by data) that Brexit did nothing.

When economists struggle disentangling the effects of brexit from the pandemic, an actual disaster, and so can't really determine if brexit was a disaster, I feel like you can't honestly say it wasn't. But still, while it's hard to gauge how bad the effect was, few experts seems to think it was good.

Ding ding! We've found our best candidate for ambassador to a newly discovered alien species!

I agree with you but I believe the definition of handcrafted will shift to include software produced partly by AI, just as most handcrafted things involve industrial machinary at some point.

But then, I can also imagine a world where nobody cares enough about writing code and instead we have "human designed software" just as we currently have human designed cars and fridges but few made by hand (and no market for them). Then, handwritten software becomes an impressive curiosity like Chris Sawyer's original Rollercoaster Tycoon written entirely in ASM.

I find your take a little lacking in nuance...

It's not the only option but jupyter notebooks are excellent for (at least) quick prototyping, data visualisation, literate programming and exploring ideas. I write plenty of Fortran and C++ and I still reach for notebooks when appropriate. In discussing with a non-technical, scientist friend, she pointed out that existing notebooks can be used and edited by folk even if they don't know python that well.

At the very least I see this particular tech as enabling more science by removing the operational challenge of running a jupyter server.

These tools take nothing from those who want to write fortran to do their science but make computation more accessible to those who don't.

I laughed so hard at that, thanks

Hard agree. AFAIK Jon Blow's Braid couldn't be made with an existing engine (at the time) due to its time warping mechanics.

That's a big reason I like developing games with frameworks. It allows reusibility and access to clever code, while still remaining flexible enough to support weird mechanics.

That's interesting. Coming from C++ and Zig, the massive time "wasters" are metaprogramming features, i.e. Templates and comptime. Are Rust's macros the compile-time culprits?

I'm really sorry to be harsh but if you don't use programming languages and have never written a transpiler, why should anyone read your comment?

You critique the approach as brittle and yet the approach you propose doesn't solve the thorny problem gp described of a growing ball of reserved keywords.

Right, that makes a lot of sense. Seems like reviewers can easily see and check those asserts to understand if the LLM has written something sensible (and I presume those asserts are strongly declared in the prompts).

I'd love to read a blog post or something on what you've discovered on good software patterns for AI. I'm only now getting into using LLMs and beyond the purely technical aspect of what LLM is "best" it seems like there's a wealth of learned experience in how to structure and write code to work well with them.

I've heard good arguments for using Rust with LLMs because the compiler keeps the LLM from making silly mistakes. What languages have you been developing in?

This is an interesting approach. When you say no abstractions though, what do you actually mean? I would count functions as abstractions but do you mean classes/structs representing an object? Effectively no new types?

I disagree with your definition of meaningful here. Society's willingness to pay is certainly a signal for meaning in an output but it seems quite inaccurate. Think of the number of artists and thinkers that weren't recognised in their lifetimes, their work was still meaningful but society hadn't discovered it yet.

Similarly, there are a number of things that would be incredibly meaningful to all of society (eradication of disease, nuclear fusion, etc) that we choose to deprioritise to instead eat fois gras and fight.

Sorry to jump down your throat on this, we're on the same side, I think BI is inevitable and worthwhile. But it's worth pointing out that BI enables more than just the meaningless things.

I was surprised to find out that material science uses up a significant amount of the compute time on the big research machines, at least here in the UK. It's both potentially lucrative research and computationally intensive. Combustion is the other big one that hasn't been mentioned but that's... Controversial.

I've found trying to find a distraction-free editor a distraction in itself. I'd bet money that most authors using an old word processor had some external reason to use it (cost, availability, editor compatability) and just stuck with what they know.

> it'd be a bug if they diverge

That's a very nice rule of thumb. I've often overabstracted when two pieces of code look similar at one point in time and then they diverge.

That's typical of models that use the anelastic approximation, where it's useful for a number of reasons to rewrite the equations replacing the true vertical with a vertically stratified variable. I've seen density, pressure and temperature used.

That's less of a different model and more a different way to rewrite the equations to make them easier to analyse or simulate.

We might be talking at slightly different angles here. There's a strong difference in the equations between compressibity of the fluid due to compression and changes in density due to temperature, chemical concentration, etc. The term compressibility usually refers to the first usage, and modelling it leads to sound waves in the system and has major implications for how the system is simulated, I mean it's an entirely different class of algorithms. The second, where density still changes but not due to compression, so no sound waves, that can be easily modelled without including full compressibility. This allows (generally simpler) incompressible models to still incorporate useful thermal physics where important, like in climate and weather. Also, the smaller the scale of the system the more compressibility matters so I wouldn't be surprised if compressibility starts to matter for e.g. Tornados. But I'm not certain on that...