Comment by nostrademons | Hacker News Reader

Comment by nostrademons | Hacker News Reader

Parent Full thread

I'm curious whether you think the ideal boundary between SW/HW might've shifted in the last ~40 years, as the things we use computers for have drastically changed?

I know basically nothing about hardware, but I know the software layer from the OS/compiler up through the UI. There's a fair bit of evidence that things we've traditionally assumed belong in the kernel actually belong in userspace, and they're being reinvented in userspace as a result. For example, most modern languages & frameworks put some form of scheduler in the standard libs - we're reimplementing the abstraction of a thread as promises or fibers or async/await or callbacks. Many big Internet companies disable virtual memory in their production servers, because once the box begins swapping you might as well count it as down. Many common business apps program to a database, not a filesystem, and then the database uses block-based data structures like B-trees and SSTables but then has to implement them on top of filesystems.

At the same time, the classic OS protection boundary is the process, but the unit of code-sharing in the open-source world is the library. As a result, the protection mechanisms that OSes have gotten very good at are largely useless at preventing huge security violations from careless coding in a library dependency.

Most of these came from computers being used outside of the original domains that the system software developers assumed, eg. nobody in the 1970s would've imagined 10 million GitHub users of widely different skill levels all swapping code. Knowing what we do now about the big markets for computation, are there additional operations we'd want to put in hardware, or things currently done in hardware that should be moved to software?

There is a fascinating talk by Cliff Click "A JVM Does That?" At the end he shares some opinion about what should be done by JVM or OS and what should change.

video: https://youtu.be/uL2D3qzHtqY

slides: http://www.azulsystems.com/blog/wp-content/uploads/2011/03/2...

I remember a talk, where he was also talking about hardware. It does not seem to be this one. For example, a time register would be useful. Syscalls like clock_gettime are too slow. CPU info like cycle counts fail with dynamic frequency scaling.

Nit: disabling swapping is not the same this as disabling virtual memory. Virtual memory is just something that allows swapping, but does not require swapping.

> There's a fair bit of evidence that things we've traditionally assumed belong in the kernel actually belong in userspace, and they're being reinvented in userspace as a result.

That's basically the pitch given by unikernels like Mirage: functionality which was traditionally implemented by an OS, like storage, is turned into a library that gets compiled into the application like any other. If an application wants to access a filesystem on the disk it can use an appropriate library; if instead it wants to manage the data being stored on (some section of) the disk directly, it just needs a different library. That way, applications like relational databases can claim their own section of the disk and read/write it directly, to avoid the performance and reliability (e.g. caching/flushing) penalties of going via a filesystem.

I think all of your points are very valid, but they're focusing on one part - virtual memory - which you suggest to remove (and this can be done today by not using that part of the hardware, and in this case the penalty of having that unused hardware is AFAIK fairly small.) My original point was that adding (or changing) hardware to accommodate HLLs is not going to buy you as much performance as people think, and this is focusing on a different part sw/hw boundary (basically what should compilers/interpreters/runtimes be doing vs what should be handled at the ISA level, versus your points which talk about what protection mechanisms we want and who among hardware, software and OS should do what here. I guess I should have said C/RISC and keep Unix/MMU out of it as I did in TFA.)

What will actually happen with protection mechanisms I don't know; certainly Unix-style mechanisms are used to ever more places with say HSA's idea of accelerators and CPUs being aware of the same virtual memory maps. Compatibility is a very strong force here. On the other hand there's a lot of stuff happening with the memory protection disabled as you described. My predictions here are going to be less educated than many others', to be honest, because I deal with embedded systems whereas most of the exciting stuff here happens in servers, I'd guess (but I can tell that in automotive embedded systems of all places not only do Unix-style processes gain traction right now but so do hypervisors with actual multiple OSes, some of them POSIXy, sharing chips. So this is a data point showing a trend in the "more of the same" direction.)

Reply on news.ycombinator.com