Ask HN: Why aren't there DSLs for Operating Systems? (Or are there?) | Hacker News Reader

Ask HN: Why aren't there DSLs for Operating Systems? (Or are there?)

34 pointsabrax314171 comments

There are DSLs for nearly eveything we do with computers. Indeed, even "general purpose" programming languages are, in a sense, DSLs for ... math (Fortran), gofai (Lisp), etc. And of course there are zillions of more specific DSLs. The purpose of a DSL is to reduce the complexity of writing code for the target domain, which usually involves specialized vocabulary, functionality, and syntax. But the most complex programs that exist (well, at least among them) are operating systems. Yet we still build them in general purposes languages, like C. Of course, we almost immediately build a layer of functionality on the hardware and system services, but even when you get to the top, like programming bash commands, it's largely done in C (for Unix-likes). I'm wondering whether/why there isn't a DSL for operating systems. (Or maybe there is, but it's hard to find if so; I've never seen one, and it's a difficult set of phrases to search the web for, so I might be just missing the obvious, for which I apologize in advance, and appreciate the pointers.)

View on HN

If you look at C and UNIX, you will see that there is a very intimate relation between those two. If you write an operating system, what is the easiest language to use if you want to build on top of that operating system? The answer is almost certainly the language you wrote that operating system in.

In some sense C was the DSL to build UNIX. It becomming a general purpose language happened after that and because of the success of UNIX.

Also in general operating systems are extremely complex and need a wide range of functionality and extensibility while being able to easily interface with assembly. That makes a DSL tend towards a general purpose language, as has happened with C.

Interestingly, the same is true for Lisp -- it was not initially a GPL. Perhaps the same is true for every GPL? (We may just be playing semantic games here because, for example, Fortran being a "math-oriented GPL" is the same as it being a "DSL for math."...hmmm)

To me the distinction is largely in the eyes of the user. Most modern programming languages have facilities to achieve a wide range of goals.

I think the question "is there a programming language made specifically for X?", is far less important than the question "is there a language in which I can do X, efficiently/securely/scalable/...?"

There are some languages which are clearly domain specific. Markdown is not a general computing language.

Original Lisp was invisioned as a way to not have to write assembly.

Note: That's avoid having to write os, interpreter, compiler, shell, etc 1st before writting application.

I don't think that this is true, except perhaps indirectly. According to Duncan McGreggor's wonderful (and very well researched) "Hidden History of Lisp"

https://blog.lfe.io/excerpts/2015/03/27/1101-the-hidden-orig...

McCarthy was interested in computer intelligence, logic, and mathematics. Shaw, Newell, and Simon, at RAND and CIT, were already working on exactly these topics in their Logic Theorist project, using IPL, an ugly language that they designed, and ran on one the first digital computers, called JOHNNIAC - after John Von Neumann.

I you are interested in just how ugly IPL(V) was, you can see more of it than you'll ever want to here:

https://exhibits.stanford.edu/feigenbaum/catalog/ff363pt6522

Ugly as it may have been, IPL language did exactly what McCarthy wanted, and introduced the concepts of list processing, symbols, and recursion. But, a bit to your point, it's ugliness, being akin to assembly, was exceedingly off-putting.

A couple years later, at MIT, McCarthy (who was simultaneously and coincidentally playing around with lambda calculus) and his students (most centrally, Steve Russell) figured out how to do list processing etc in a much simpler syntax.

Still, the "modern" homoiconic Lisp of s-expressions was a bit farther down the line, as it was originally conceived with M-expressions, which were a bit more algol-like. S-expressions were initially just the data representation. Russell realized that the s-expression syntax could be used for both.

Thus the birth of Lisp, probably the world's most influential and enduring programming language. Sorry, this got a bit off topic.

Not if contrast with MUMPS language (pre-algo) as viewed from issues with today's lisp & multiprocessing. But that's a discussion more for a separate post.

This was exactly my thought too, that C is pretty much the DSL for (many) OSes.

DSL for assembler might be more in-line with original C language intent.

You may be very interested in the STEPS project from Alan Kay’s Viewpoints Research Institute. STEPS was an attempt to build an entire desktop operating system environment with only 20,000 lines of code. In order to accomplish this, the design makes extensive use of DSLs defined using OMeta. There are examples of TCP and the 2D graphics framework being implemented in an impressively small amount of code.

http://www.vpri.org/pdf/tr2012001_steps.pdf

Adding on: Alan Kay gave several fascinating talks about the project. Such as https://youtube.com/watch?v=ubaX1Smg6pY

Did you go down the rabbit hole from yesterday too?

I didn’t see any posts related to STEPS yesterday, but I’ve been aware of STEPS for a few years.

It was this link [1] from a couple days ago. There was a fair bit of discussion about it on HN [2].

[1] https://chreke.com/little-languages.html

[2] https://news.ycombinator.com/item?id=33693472

Thats it.

sh IS the DSL*, otherwise, I'm not sure what you mean by "for operating systems".

The question "What is the domain specific language for operating sysyems?" is meaningless and unanswerable, because "operating systems" does not define the D in DSL in any meaningful useful way.

* For instance, when you try to replace all the sh scripts in a distro with say python (as some distros have tried to do) you immediately discover several ways that python is far less convenient for executing external executables and feeding, collecting, piping, their inputs and outputs. Then for all of the arcane annoyances with bash/ksh syntax, it's basic shell / batch / glue functionality is more important than all of it's "not like a better designed general purpose language" annoyances. In other words, exactly, it's not a general purpose language, it's a domain-specific language.

There are a number of DSLs for specific tasks, such as eBPF or Dtrace filters, Julia Lawall’s semantic patches, DSLs for device driver development or writing system call emulators and many more.

A DSL covering more and different parts of OS development would probably have to be so general that it doesn’t make sense not to use a general purpose language…

If macOS has a DSL, it's AppleScript.

AppleScript is a scripting language created by Apple. It allows users to directly control scriptable Macintosh applications, as well as parts of macOS itself. You can create scripts—sets of written instructions—to automate repetitive tasks, combine features from multiple scriptable applications, and create complex workflows.

  set targetFolder to POSIX file "/Users/Me/Desktop/MyFolder"

  tell application "Finder"
      set finderFolder to item targetFolder
      set lstTargets to target of every Finder window
      if {finderFolder} is in lstTargets then
          set target of (make new Finder window) to targetFolder
      end if
  end tell

That said, AppleScript is probably not long for this world. The future looks more like visual programming via Shortcuts.

I'm not sure I understand the question, but I interpret it as being about one of two things: Importing code written in a DSL into the OS at run-time, and creating a custom OS kernel specified in a DSL at compile-time. There do exist examples of both, and there is some overlap between projects.

Widespread examples include firewalls and routing rules written in DSLs, which get compiled into a form that gets run inside the kernel. Example: nftables in Linux.

Some research OSes allow drivers to be written in a "safe" language to run inside the kernel. Some perform JIT-compilation, others create a custom kernel at compiled-time. Go search for Scout, Termite2, and Synthesis Kernel.

Windows NT ran dos, windows as separate VM instances

Devil is a DSL for device drivers: http://phoenix.inria.fr/software/past-projects/devil.html

The definition of DSL is vague, but at least in some sense, some operating systems do implement such a DSL.

For Windows between 2000 and 7, that would be VBScript. The language itself is primitive, but through various COM APIs (WMI, ADSI, ADO, MSXML, WScript.Shell, etc.) one can do borderline anything with the OS, or even with the whole Windows networks.

Modern Windows, while still supporting VBScript and most of these COM things, has also added PowerShell as another option.

Is the OS itself written in VBScript? (You're going to make the fair complaint that it's hard to tell where the OS stops and something "non-OS" starts. And I get that. I'm guessing (because I don't work on Windows) that the folks who support the OS don't code, for the most part, in VB.)

In 2000, the performance overhead was too large. PowerShell, and most of PS applets are written in C#. PowerShell is a lightweight scripting layer on top of the same .NET runtime.

BTW, last couple times I dealt with Linux, I remember essential userland OS components implemented with bash scripts.

The closest thing to a DSL for writing the OS was C for Unix. They were created alongside each other. There are more niche examples too like Oberon.

C was orignally a macros for assembly setup.

It wasn't just VBScript - one could use any language that registered its interpreter with Windows Scripting Host, though VBScript was the most popular due to being one of the two options shipped with the OS (the other was JScript, a javascript implementation).

I have at the very least seen WSH support in glory days of ActivePerl, I think ActivePython and ActiveTCL included that as well. There was also I think a REXX implementation available.

Also, the same approach of using COM for everything is continued in PowerShell and generally all new Windows APIs - a lot of stuff in PowerShell is thin wrappers (if at all) over COM APIs, and you can definitely call custom COM objects as well.

Windows post GUI on top of dos, was based on ported OS concept of vax/vms micocode. so dos & 'MS windows' each became a 'dsl' vm.

Defining a DSL for writing operating systems would essentially be the same as writing an operating system (a huge amount of work!). The range of what an OS does is wide enough that your DSL will probably be pretty general purpose anyway, though.

DSLs for interfacing with OSes exist in the form of command shells, syscalls, and ABIs.

originally, the hardware bios & hardware drivers was the dsl stuff OS'es used to make things happen at hardware level.

EBNF is/was one way used to define how to write an OS.

For many years, IBM has developed its mainframe operating systems in a series of proprietary languages, such as PL/S and PL/X, designed specifically for systems programming.

Compilers and manuals for these aren’t generally available, but here are published descriptions of two older variants, for use by customers who received source listings on microfiche:

http://www.bitsavers.org/pdf/ibm/360/pls/GC28-0786-0_Guide_t...

http://www.bitsavers.org/pdf/ibm/360/pls/GC28-6794-0_PLSIIgu...

Your question isn't completely clear. What is it that you want this DSL to do exactly? Windows has PowerShell, which is precisely a language to reduce the complexity of managing systems. AppleScript is a DSL for controlling your macOS desktop (and more). The various shells available on Unix systems works as glue, but perhaps not exactly a DSL in the same sense that I feel PowerShell is.

Ansible, Puppet or Nix could also be viewed as operating system DSL. Again depending on what your goal is.

Depends on how define OS 'DSL', 'os' just metadata bindings for hardware / software

* embedded os : https://blog.felgo.com/embedded/embedded-operating-systems

* OS modules / daemons : ebpf / systemd

* ebpf : https://news.ycombinator.com/item?id=33489935 ; https://news.ycombinator.com/item?id=33462031

* tiny core : https://en.wikipedia.org/wiki/Tiny_Core_Linux

* yocto : https://www.yoctoproject.org/

* 'bootloaders' don't have to just 'load an os' https://wiki.archlinux.org/title/syslinux

* uefi : https://askubuntu.com/questions/651902/what-is-the-differenc...

* OS device drivers for external devices such as printers (post script printers)

* OS api's

I like your question.

I am currently interested in this problem and more specifically the expression problem.

Also the problem of how do you tell a computer to do X operation when it encounters Y.

Computers are often turning machines and state machines.

I recently implemented async/await with a giant switch statement.

Async await is far more powerful than bash jobs.

I think the scheduling of processes on a computer needs a powerful DSL.

Kubernetes is interesting since it scheduled processes on a cluster of nodes. But it doesn't define pipelines.

Kafka defines a pipeline for processing.

Bash defined pipelines of operations and processes text.

Java streams define a pipeline.

JavaScripts promises and C# async/await are interesting for scheduling tasks.

ETL workflows Extract Transform Load and Enterprise service buses often have visual DSL/GUIs with eclipse tools for defining mappings.

In Python I often chain together processes with POpen. But it's very low level

I think all these topics are linked.

I want a DSL for defining operations and chaining them together similar to a pipeline. I also want when_any and when_all support and multithreading support.

I didn't read a lot about it, but I recall that Microsoft created a language called P that was a DSL for writing device drivers (more generally, state machines, but I think the point was to use it for device drivers). I also recall they (allegedly, at least) used it to rewrite the USB stack for Windows 8.

Looks like they released the compiler under the MIT license and put it on Github, so you may want to look into it.

https://en.wikipedia.org/wiki/P_(programming_language)

https://p-org.github.io/P/

https://www.microsoft.com/en-us/research/project/safe-asynch...

Programming probably has a better shot at this happening if programming can be automated more.

DSLs for operating systems might be easier to write with programming languages that have higher-order unification – meaning that they can automatically piece together programs to fill out the meaning of one smaller word that has a complex, concise meaning – and totality checking, which means that given subprograms can be known to run to completion. And first-class types. (Dependently typed languages.)

This combination allows for tightly controlled and deterministic automation of programming, which is especially useful for getting rid of boilerplate code, conversion between formats, serialization and deserialization – and the maintenance of this kind of code… when all targets are moving targets due to many pieces of software that change from day to day. As found in things as complex and arbitrary as operating systems.

Original LISP was pretty much a DSL for symbolic computation, something that wasn't exactly catered to at the time (it wasn't first, but it was so early in computing you could write a Master's Thesis about IF-THEN-ELSE construct).

That said, various Lisp Machine systems as well as internals of Lisp implementations often contain DSL for implementing the OS/language, sometimes with many smaller DSLs (while the ur-example is "LOOP" macro from MIT, my favourite is a file from Genera bearing a comment - paraphrased - "behold, I implement half of PL/I to implement structure packing into byte vectors").

Of course, in languages like Lisp or TCL the separation between DSL and main language is not so stark as in case of DSLs implemented in let's say C or Java.

This smacks of forgetfulness. I wonder what the age is of the OP.

As others have pointed out, *nix systems (Linux in particular) are rife with DSLs. Not just C but (c|ba|k|tc|z)sh, sed, awk, grep, to name a few. Each of these tools has it's own mini DSL, some more complicated than others, that can than be composed to with other tools, using their own DSLs to manage tasks (e.g. operating system tasks).

Even the task of creating DSLs has many DSLs attached to it (lex, yacc, etc.).

Programs that get used and become popular introduce their own DSLs, often times. Apache, NGinx, etc. all have config files that each have their own special sauce of DSL for their own tasks.

Even many of the newer systems like Docker, Kubernetes, etc. have config files/DSLs to manage them.

I'm sure I'm forgetting even more.

Is your hypothesis that the OP has already forgotten more than you ever knew, or that they are too young to know much? Regardless, I can tell you that the OP has worked on (meaning deep in the guts of the OS, but not for the 'manufacturer'), a dozen OSs that they can recall off the top of their head. They were written in one of several different BAL-like assemblers and assembler macro languages, MACRO-10, PDP8, 11, and VAX assemblers, two different Lisps, several 68* assemblers, and C. So, probably the former interpretation is more likely correct.

My hypothesis is that we have a collective knowledge that we tend to forget and that the new generation often doesn't understand, or takes for granted, the foundations of the infrastructure we use and sometimes tries to apply concepts they've just learned without understanding the subtlety and complexity of the ecosystem as it exists. This tends to happen with younger people which is why I was asking the age of the OP.

You being the OP that you're referring to yourself in the third person? And you're trying to insinuate that I'm an idiot criticizing an expert? It'd be nice to say so instead of saying this so obliquely. I've insulted you. Sorry. I was and am arguing in good faith. Please show me the same courtesy.

It's great that you worked on all these assemblers, macro languages and operating systems but I stand by my point. There are many DSLs in and around "operating systems" that you've failed to identify.

Maybe the crux of disagreement is that you have in mind a particular type of DSL for a tranche of operating system engineering that you haven't properly articulated. Considering your oblique insult and response, I would guess this is the case.

I’m sorry. I was not trying to insinuate that you are an idiot. I was trying (and failing) to be funny.

We need less DSL, IMO, more constrained code generation from data models.

There is so much bloat due to everyone making a DSL to learn distributed systems with, or every little bit of UNIX coming with its own text config format.

I’d like to get away from generating yaml for Docker and k8s and import balancing forks of microservice logic v8 style; read kernel telemetry from eBPF in the app to determine how many forks the code can add or needs to subtract, and terraform because it’s still just a Linux box under there.

But the cottage industry that profits from these ideas is threatened by these ideas and sticks with showing off a new arbitrary string parser that ALSO sends the strings to an API for you!

The software industry has turned into a jobs program and has little to do with novel engineering.

Although I get where you're coming from, it seems that this just renames a DSL as a model specification language.

That’s an overly reductive, thought ending, semantic position that sticks to status quo; I have been getting that a lot from the work bee office crowd. I freelance so maybe I am capable of a different perspective? It’s relative.

It would also eliminate who knows how many lines of code in the form of parsers/lexers for dozens of random static formats, and embedded HTTP clients that reach out to APIs being copy-pasted all over to handle ops.

It would bin a whole lot of cognitive gibberish that comes with Docker and k8s, and move SRE/ops thinking to leveraging the same language ecosystems as app developers, normalizing jargon.

But yeah let’s stick with your simple reduction. Apathy and lethargy are the zeitgeist of the day. The “disrupt!” industry has become a “wait not me though!” jobs program.

Effectively a government jobs program as the rate hikes forcing layoffs show. As soon as the Fed shut off the money printer, big techs house of cards started wobbling. Truly innovative measure like layoffs, a form of politically correct economic insecurity[1], not novel new ideas, was big techs reaction to government policy shift.

[1] https://www.nytimes.com/1997/02/27/business/job-insecurity-o...

I'm curious about the v8 thing (Google's V8?) — can you tell more?

Aside from automatic memory management, there is very little evidence of productivity gains from using higher-level languages. $hypedlanguage fanboys can talk until your ears bleed about why $hypedlanguage is better than the competition. But invariably they can not cite empirical evidence that support their claims. Furthermore, productivity is much more than just programming language (e.g C sucks, but I got Valgrind now so it sucks less). Those who write OS'es knows this and stick with what works.

Ummm... using compiler to dump the assembly of basic c program:

   #include <stdio.h>
   main ()
      {
    }

results in around 800k - 1.5mill individual lines of assemble necessary to do bookkeeping before/after the program runs.

Writing this for each specific os / hardware situation (even in general case), would seem like a big waste of time vs. having the compiler generate it.

Try adding -nostdlib, right?

You can tell the compiler to do none of the standard bookkeeping (someone who actually writes OSes will probably correct me)

yeah. :-)

check out MUMPS!

Is this an arguement that writting c to impliment awk, SQL, html/java script using raw X11 to input/output data is easier to do correctly / easier to understand / less buggy than just webassembly compiling gawk and use with standard web browswer features such as html/javascript etc.? Former C stuff is few million lines of code, later is less than a 1000. (including sql & database features in awk). ARC & icon language ports (5th generation programming languages), under a few hundred.

The arc & icon language ports would be smaller, but smaller code base means easier to add/impliment user requested features and still keep things 'small/understandable'.

Would much rather deal with under a few hundred than the equivalent few million lines of code.

If you rely on a web browser, aren't you still relying on millions of lines of code you're just not counting it?

Yes, but where is the focus on effort?

Compiling for each possible OS / OS variation in order to run things to support the few hundred lines that actual do what interested in

or just the few hundred lines of code that actually do what interested in.

"Aside from automatic memory management" C doesn't have automatic memory management.

user/application level or os/runtime level? C doesn't have automatic granular memory management.

sbrk, registers and runtime stack are forms of memory management. (unix inodes via pipes, OS envirnment, and files)

I think a DSL is a more often a type of “user interface” for a large and powerful underlying software system, than a way of writing native code for particular specializations. Operating systems offer many DSLs as a way for programmers to interact with them: Linux has bash and eBPF, Windows has VBScript and Powershell, etc.

If something were to offer a DSL for writing operating systems with, I think it would be either an embedded platform or a hypervisor.

Windows also has FOSS Autohotkey which is very good DSL for automation and general programming. Not OTB though, but its less then 1MB and portable.

Wasn’t Python originally conceived as a systems configuration language?

You may be thinking of PERL (more precisely "report generation", I think). AFAIK, Python was conceived as a sort of modern BASIC -- general purpose, easy to use language.

Python was conceived as teaching language, continuation of the ideas behind ABC. I recall at least one public row between BDFL and the rest due to Guido calling on that legacy as reasoning for why negative hex numbers should be written like -0x1 instead of one's complement that a lot of people seemed to prefer for technical reasons.

ACPI is pretty much a DSL, if I'm not mistaken

What you're referring to is AML, which is part of ACPI. It defines a bytecode for short programs that are provided to the operating system in the ACPI tables for performing certain operations on devices. The operating system can then 'call' these functions by executing the bytecode.

This way different devices can use different procedures to, for example, put the device into sleep mode.

https://wiki.osdev.org/AML

Domain specific. The entire OS is domain less, it's a host to run what is to be your domain. The OS has APIs to get anything done. And there are libraries to access those APIs from many languages, some that can be somewhat labeled a DSL supporting language. But until you have a domain, it's just the OS. The OS the non-domain or everything domain.

I'll just state for all us dummies, I suppose, that your casual use of DSL as an initialism without writing what it stands for, is not cool.

For folks who evidently disagree with this, consider that DSL has another common tech meaning. More common, for quite a lot of people, who may even be programmers. Defining initialisms and abbreviations upon first use used to be a basic assumption in most writing, and in most teaching of writing. It’s probably fine to assume some knowledge based on your expected audience, but when your audience tells you it doesn’t have the knowledge you assume, it’s okay and even good to have a little humility about your mistaken assumption.

levels of hardware abstraction & DSL's that support the given hardware abstraction might be better approach.

aka. typical DSL's at the pre-OS boot / bare metal level -> BIOS; communications protols - USB, epci, scsi; DSL on epci card, harddrive bios

DSL's used by OS -> ebpf, tcp, serial, printer drivers, display drivers (hdmi, displayport, vga)

DSL's allowing user to interact with OS -> shell, systemd

> consider that DSL has > another common tech meaning

Which, in an elegant self-own, you’re not going to tell us.

A self own indeed! My apologies. The other common meaning I referenced is digital subscriber line.

Sorry. I meant “eloquent”. But thank you!

In this case it means "Domain-specific language". SQL is a great example of a DSL. https://en.wikipedia.org/wiki/Domain-specific_language

still need a database engine, device driver(s) for database engine (storage & display) & for storage device for SQL to work.

I’m presuming Domain Specific Language, as the context refers to a “target domain.”

https://en.wikipedia.org/wiki/Domain-specific_language

Reply on news.ycombinator.com