Hacker News Reader

Comment by boulos | Hacker News Reader

Off-topic: it's too bad that The New Stack reproduced verbatim the term "Memory Controller" instead of "Memory Allocator" (as mm/slab.h agrees). I was really confused as to how there was going to be a Linux Memory Controller, and thought some new-fangled AMD/Intel memory controller thing had moved into software...

View on HN

My first mental image was of an MMU that was running Linux internally—like the Xeon Phi, but even more ridiculous.

Yes, though Xeon Phi had a BSD :).

For those of us that don't know what the above is about: https://news.ycombinator.com/item?id=12293771

I thought it was an MMU designed to work with a specific Linux memory feature.

My first jump was some major change to the MM subsystem

I've used a couple DMA controllers that are Turing complete-- it's pretty cool what kinds of things you can offload.

Turing completeness means undecidability of termination, not something you'd enjoy in DMA operations!

Turing completeness does not mean you cannot reason about the termination (or performance) of any particular program.

It's not something you'd enjoy in most computer programs, but we still use Turing complete CPUs and Turing complete programming languages, because they're more flexible. No reason DMA needs to be different.

Most hardware will hang if you program it incorrectly, even if it isn't advertised as being turing-complete.

Turing completeness is a statement about the nature of infinity. It is always decidable if a Turing machine with a fixed upper bound on it's tape length will halt.

There was a post (this year?) mapping Godel's incompleteness theorem to the halting problem. I found it inspiring, not in the least because the link revealed they were both in fact statements about the properties of infinity - something that was not obvious by just looking at either problem on its own. It is of those few cases where the whole was obviously bigger than the sum of the parts, for me at least.

There’s a semi-wellknown trick on the raspberry pi for performing high-bandwidth and high-precision (in time) operations on GPIO pins from userland, using the DMA engine. You keep an in-memory buffer of the next few milliseconds of bit patterns you want to write to the memory-mapped GPIOs and then just leave the DMA engine endlessly copying data from this (circular) buffer into the GPIOs. But crucially, you “pace” the DMA transfer by configuring it to alternate between copying a word to the GPIOs and copying a word to some other peripheral that you’ve clocked at a specific rate (e.g. the audio DAC, which won’t acknowledge each write until it’s converted the previous one). This lets you define the state of the GPIOs as an arbitrary waveform sampled at some frequency of your choosing, which is a crazy powerful tool for e.g. motor control (that’s what the Printipi 3D printer does) or bitbanging communication protocols, etc.

You can also flip it around and sample GPIOs at close to 1 MHz. Piscope is an example of that. Once you start pushing 1 MHz, jitter starts to become nontrivial as you’re saturating the bus or competing with other memory transfers.

So yes: a capable DMA controller can do some pretty mind-blowing things!

Fyi, a simple microcontroller like stm32 which is like 3$ can push gpio at 25mhz reliably.

It’s true. And with the newer multicore Pi’s, you could probably get similar results just by pinning a thread to one core and dedicating that core to managing your IO. There are a million ways to solve any problem.

Perhaps the single biggest advantage of the DMA solution for something like pi-scope is that it means people who want to use the pi as a scope need only download a piece of software: no soldering or flashing of MCUs required. The initial setup is more beginner-friendly, with the drawback that contributing might be more intimidating because the software is doing more complex work.

That sounds like an excellent start for a wavetable based synthesizer.

Reminds me of the ASICs in the Amiga.

Perhaps Memory Resource Controller is the best term? From the memory cgroup documentation [0]:

> NOTE: The Memory Resource Controller has generically been referred to as the memory controller in this document. Do not confuse memory controller used here with the memory controller that is used in hardware.

[0] https://www.kernel.org/doc/Documentation/cgroup-v1/memory.tx...

Programmable memory controllers might be really interesting, even if it was limited kind of programmability. You could have process specific policies like "don't pollute cache" or "real-time" or per-app tuned prefetch rules. Someone must have done this in academia...

I've been hoping for some time that someone could make an in-hardware (or FPGA) compressing memory controller or MMU, with low enough latency to be comparable to memory access, possibly even higher bandwidth (being compressed), and transparent. Would increase the effective memory of SoCs, but also in general. Thinking like zram but realtime. Another one for academia I guess.

Basically a hardware lookup table, right? Potentially fast to read, but impossibly slow to write as you’d need to reprogram an FPGA. Still a cool concept.

I had in mind something like several parallel hardware LZ codecs sitting at the border between the CPU caches and memory, along with some VM driver software. Given CPU writes would go to caches first (as managed by cache-coherency logic), they would remain fast to read, only at the point a flush is triggered would a codec have to write a compressed page to memory - which may be slower than uncompressed. I'd imagine there would need to be a bunch of experimental work to determine page size and other parameters, which may even be application-specific tunables.

There is one specialized area this is widely done in: hw texture compression in GPUs. Buts it's read only and not automatic.

GPUs also do lossless transparent compression in some places for bandwidth reduction, at the cost of allocating some additional metadata. It's commonly called delta color compression. (And it doesn't reduce memory allocations, but it fits the theme here.)

You might be interested in the programmable prefetcher work from Sam Ainsworth at Cambridge University https://www.cl.cam.ac.uk/~sa614/papers/programmableprefetche...

Reply on news.ycombinator.com