touisteur

The focus of their work seems to be low-level firmware, and not CUDA or any kind of compute. It makes sense a lot there because you won't need as many libraries.

The syntax is really a red-herring especially when you get contracts, proof of absence of runtime errors and higher-lever functional proof, mostly automated, or assisted with annotations. If you're actually going with this effort, the lack of curly braces, ampersands shouldn't be a main concern.

I don't think there's anything preventing anyone to sell an Ada compiler, tooling. It's an ISO standard and you can read its spec (the holy Ada Reference Manual) and rationale for free.

I'd say the biggest hurdle is that Ada is a very complex language to implement. I've been writing Ada for more than 20 years, and writing static analysis tools for 5 years with libadalang (an amazing toolbox...) now and I still find it hard to grasp the whole combinatrics of features.

Easy to use, but very hard to implement.

Why would you assume that ?

PTC still maintain PERC (for some definition of 'maintain'). At least I hope so.

Haven't seen uncheched_deallocation in modern codebases in a long time... The secondary stack removes many/most use cases of short-lived dynamic allocation and pointer-passing. Controlled types, containers, alleviate most other use-cases. I'd frown at code review if I saw raw memory allocation. Memory safety issues I've seen (very rarely in 15 years of dev) hinge on aliasing, stack overflow (which you can mostly catch now) and tumbling down in C bindings.

I'll praise the language I use daily as far more memory-safe than the others I use daily (bar managed-memory Java-like languages).

People sleep on /dev/rtc and PIE !

And it really isn't so surprising to have to go 'down' to PTX for such a low-level optimisation. For all the love AVX512 articles get here, I, for one, am glad some people talk about their PTX secret sauce.

I wish Intel didn't kill the maxas effort back then (buying the team out and ... what ?) as they were going even lower down the stack.

I've always found xkcd's radiation chart interesting: https://xkcd.com/radiation/ where it talks about coal vs nuclear plant at the beginning. Maybe his sources would help. ISTR that lignite would often be a bit radioactive and burning so much of it would spread it through leftover emissions ? If you find a proper source, I'm interested too.

I thought that was what HDR was at first in my young naive days: for the same sensor put several ADCs in parallel with overlapping ranges and auto-switch to the non-saturated one...

The absence of anything >200G isn't reassuring there...

Maybe a sign of NVIDIA divesting low-margin SoCs or finding a way to be in nextgen phone chips ? (Edit:) solidifying the cuda moat in phone SoCs too ?

Which is sad (array size limited to an int) and has always annoyed me, coming back from Ada where the index of an array is just another discrete type (including boolean, enum type, and ranged integers).

I think the pattern in question might be (for example) the way some people (like me) sometimes write JSON as a trace of execution, sometimes directly to stdout (so, no going back in the stream). You're not serializing a structure but directly writing it as you go. So you don't know in advance how many objects you'll have in an array.

Another way of seeing this is: whatever way people are using JSON, over a fleet of all the machines running the parser (or serializer), reducing the execution time (a good enough proxy to energy consumption) on the whole fleet is worthwile. Especially if it has zero impact on the code calling this parser/serializer (so, no second-order energy spent because of the change breaking some API or contract).

Sometimes you have a scalar instruction but not a vectorized one, or it doesn't match the 'lanes' you want to operate (ISTR weird holes in AVX where I could find a specific instruction for 8, 32, 64-bits lanes but not for 16). Always good to have an escape hatch there, especially a highly pipelined (or pipeline-able) one.

SPARK allows you to statically prove properties about Ada code. Proving a sort implementation is a classic example : https://blog.adacore.com/i-cant-believe-that-i-can-prove-tha...

Ranges are awfully useful in Ada, but a very small help in dimensionality/units analyis.

If you mean to go all the way on units, dimensions and typing, there's a bestiary there, quited maintained (with several Ada 'takes' too) https://www.gmpreussner.com/research/dimensional-analysis-in...

Cunningham's Law states "the best way to get the right answer on the internet is not to ask a question; it's to post the wrong answer."

From https://meta.m.wikimedia.org/wiki/Cunningham%27s_Law

Later on GP talks about radar, so I'd guess probably at least the basics, convolution, matrix multiplication, and some inverse methods (evd, svd, cholesky...). Not just implementing but understanding what it does, a bit ?

Might depend on the quality of crashes. Losing hours or days of work would quickly sour me. HP RGS crashes, twice or thrice daily are just 'meh' - reconnect and nothing - but 15 seconds and typing auth tokens - is lost - maybe 'flow' but I've become resilient there.

To get a Doppler-shifted reflection you can also... move. Ground-based surface radars can work (somehow) with just no Doppler info, cataloguing, mapping fixed reflectors and very often just building ground maps. If your radar has multiple vertically and horizontally spaced receptors (eg. phased-array) or you can aim your beams, you can also somehow build a 3D map or at least map surface features/masks. This is before anything moves.

Maybe Frank Herbert got you covered, with the Bureau of Sabotage https://en.m.wikipedia.org/wiki/The_Tactful_Saboteur :-)

You ask how large and citation, and given one with sizes, you already have a billion ones. The article is about long convolutions (winograd being still a matmul implementation for square matrices and a pain to adapt to long convolutions and still N^2.3), and given comparisons with 10-years-optimized-(as you say) cuDNN and very naive barely running FFT-based long convolutions, showing actual O complexity matters at some point - even with the best hardware tricks and implementations.

I don't know what more to say ? Don't use FFT-based convolution for long convolutions if it doesn't work for your use cases or if you don't believe it would or should. And those of us who have benchmarked against SOTA direct convolution and found out FFT-based convolution worked better for our use cases will keep using it and talk about it when people ask on forums ?

Well let me tell you about the pretty high-end and expensive L40 that shipped with PCIe-4.0 to my utter dismay and disgust. Only the H100 had 5.0 although I could already saturate 4.0 (and 5.0) with Mellanox NICs and GPU/StorageDirect. Waiting for the next one to maybe get 5.0.

You can mostly compute it for yourself or get an asymptotic feeling. Matmul is N^3 even with tensor cores eating a large part of it (with diminished precision, but ok) and FFT-based convolution is NlogN mostly. At some point - not that high - even the available TFLOPS available on tensor cores can't keep up. I was surprised by how far cuDNN will take you, but at some point the curves cross and matmul-based convolution time keeps growing at polynomial rate and FFT-based mostly linear (until it gets memory bound) and even that can be slashed down with block-based FFT schemes. Look up the thesis I posted earlier there's an introduction in it.

The convolution by FFT overlap and save can have very low intermediate storage (none on GPU with cuFFTDx for example). And most of the time, the IFFT doesn't have to happen right away, lots processing can still be performed in the frequential domain.

Having each of 18k CUDA cores of a L40s perform small 128-points FFTs and with very little sync or overlap manage long filters... is pretty efficient by itself.

There's a lot happening in the HPC world on low-rank (what you're intuiting with PCA), sparse and tiled operations. I have a hard time applying all this to 'simple' signal processing and most of it lacks nicer APIs.

I've seen lots of interesting things with 'irregular FFT' codes and working on reducing either the storage space necessary for FFT intermediate results, sometimes through multi-resolution tricks.

Look up Capon filters and adaptative filtering in general, there's a whole world of tricks there too. You might need a whole lot of SVDs and matrix inversions there...

Bust mostly if you're on a GPU there's a wealth of parallelism to exploit and work-around the 'memory-bound' limits of FFT-based convolution. This thesis https://theses.hal.science/tel-04542844 had some discussion and numbers on the topic. Not complete but inspiring.

Exactly this. I was thinking exactly like GP but I've been doing a large amount of benchmarking on this and the FFT rapidly overcomes direct convolution with cudnn, cublas, cutlass... I think I'd seen a recent Phd thesis exploring and confirming this recently. NlogN beats N^2 or N^3 quickly, even with tensor cores. At some point complexity overcomes even the bestest hardware optimizations.

And the longer the convolution the more matrices look tall-skinny (less optimized). Also you have to duplicate a lot of data to make your matrix toeplitz/circulant and fit into matmul kernels which convolution is a special case...

To me this 'problem' of reproducibility from time-outs had been 'solved' in many static analysis tools or solvers, with an actual 'steps' counter or some form of tuneable constant-across-runs metric... one way to do it anyway.

Talking long convolutions, on parallel architectures, through FFT there's a lot of performance to gain from Overlap-and-Add or Overlap-and-Save schemes, especially on GPUs with cuFFTDx which brings a whole new set of device primitives to the table and looking at (or getting inspiration from) tcFFT which allows using Tensor Cores for FFT and actually increases throughput on lots of convolution workloads.

Same for AVX instrinsics. Very fun to use (for some of us), powerful when correctly used, but so hard to actually get right and to get the expected performance. I wish Intel had put more into ISPC back then.