Comment by gumby | Hacker News Reader

Comment by gumby | Hacker News Reader

Parent Full thread

The name and the format are distinct. The linker output (the a.out file) will be in whatever is considered native by the compiler, which for most people is the format of the system they are running the compiler on.

“a.out” is barely a format: just a block of executable, a block of data initialization and an integer to say how large the (uninitialized) heap should be. Later some symbols and a bit of debugging info was added. But remember back then a lot of stuff was still written in assembler and machines were not that powerful, so something simple was not only sufficient but better.

Separating code and data wasn’t even really needed but back then it wasn’t clear that (almost) every machine would be a von Neumann architecture. There were still plenty of Harvard architecture machines (split I/ D) space. Of course these days we’ve partially circled back with write protected executable blocks and execute permission forbidden to data and stack.

When I started designing bfd in 89 or 90 I started with a.out. There were already other formats (predominantly forms of COFF) because the limitations of a.out had started to be a problem.

An aside on BFD, which doesn't seem to have come up, and I suspect relatively few people have read the History node of the ‘Binary File Descriptor’ Library manual on the back-constructed name: “The name came from a conversation David Wallace [gumby] was having with Richard Stallman about the library: RMS said that it would be quite hard--David said "BFD". Stallman was right, but the name stuck.”. [I gather that's ‘Big’, ‘Deal’, and something like an adjective.]

That’s basically correct. The first time I gave a talk on the subject, I was asked what the acronym stood for. I had to come up with an explanation on the spot and casually said “Oh, it’s Binary File Descriptors” as if that had always been the name. And that’s how it got an official name.

Both of these anecdotes just made my day :)

same!

I was just thinking about this in the shower. The death of the Harvard architecture combined with the slow adoption of w^x seems to vindicate the problem, but I think there's a deeper truth to it: Turing complete machines can emulate any other Turing complete machine. Harvard architecture is immune to stack-smashing, but it's not immune to privilege escalations - Fundamentally, there's a usability tradeoff of "Loading new executable code from the data segment" that makes true Harvard architectures inoperable. We've settled on a happy if imperfect medium.

Van neumann is more general and provides more flexibility (what if you need more data space and don't need all that code space?).

There are still new harvard architecture microcontrollers being designed. It provides some security in the field as well as simplifying the chip layout.

It'd be pretty annoying for a JIT compiler to have to manage too. Java can do a garbage collection with barely any memory left - cool! But, it'd be a shame if similar tricky stuff was written just to e.g. decompile code just to make way for more code space. Pages with only one of 'write' or 'execute' privileges work well enough

Harvard architecture is pretty common in MCU and ASIC space, and especially in tiny little low-powered processors that might have a tiny bit of flash that instructions are read out of and an even smaller amount of SRAM for data. The ARM9 architecture is effectively Harvard for example.

These use cases could be served by asking the operating system or the hardware to copy data to the code memory. Weakens absolute protection, but it would be an escape hatch to enable JIT compilers.

That's half the definitional difference between Harvard and Von Neumann architectures, though - That never the twain shall meet.

There's a second insight on the tip of my tongue, here: Harvard architectures can emulate Von Neumann ones, and vice versa. And problems, specific problems, can be solved by either. But it seems to me that there's something interesting about emulation too - That Harvard architectures can open themselves up to precisely the same vectors of attacks if you enable "Copy this data block to code memory".

But not if instead of copying, they support a "Run this data as a program in virtual machine with access to these segments of memory" operation. The behavior would be functionally identical to emulating the architecture and interpreting the data, except that it wouldn't be slow and could be done recursively with no cumulative overhead.

If you couldn't transfer data memory to code memory at all, how would it be possible to write an ordinary compiler? If it's possible to run a compiler on the system, just have your JIT do whatever the compiler does.

Write it to the filesystem (ELF format presumably) and execute the file.

Some Harvard architecture implementations provide ways of shoving data into executable space and back (like special copy instructions) but it's not general purpose. Generally you compile on Von Neumann and flash to Harvard.

Well you write a bytecode compiler and run an interpreter for it out of code space. It could be a JIT compiler, just not one that generated native code.

This approach was pretty common in the 80s Lisp days IIRC.

Java already compiles to bytecode, which executes on the JVM. But this is too slow for most practical applications. JIT compilation to native code is crucial for execution of Java programs with competitive speed.

It would be possible to compile everything to machine code ahead of time (pretty much what GraalVM does), but that would compile everything even if it would never end up being executed*. Without JIT compiler, it would not be possible to recompile code to take advantage of tracing information gathered at runtime. Also, newly loaded Java code would not be able to be optimized at all.

There are workarounds for all of these. For example, the JIT compiler could generate a new executable with the optimized code and migrate execution state to a new process. But it seems very clunky.

*: Not really an issue with microservices and stripped-down binaries. A huge deal for IDEs and environments with plugin architecture though.

In a standard harvard architecture that isn't possible.

WASM is a Harvard architecture, so it's not as dead as one might think. :)

I believe this is for the same reason some microcontrollers do it, to whit there is no page table and much less page access bits. These days the browser is a greater thread vector than ordinary application code.

Wasm is a Harvard architecture mostly because it wants to support implementations based on static recompilation. Obviously, self-modifying code is a problem for static recompilation, and the easiest way to prevent self-modifying code is just to not expose code in the address space at all. There's also the historical fact that wasm is a successor to asm.js, and asm.js is a Harvard architecture by virtue of the fact that JS Typed Arrays don't allow you to execute code stored in them.

Wouldn't GPU somewhat be Harvard architecture - e.g. code + data is split (then again, it's not general purpose CPU so this is way off, but it's coming closer every year)...

Reply on news.ycombinator.com