Comment by vezzy-fnord | Hacker News Reader

Comment by vezzy-fnord | Hacker News Reader

Parent Full thread

As tired as it may sound, I think there just might be a convenient time for the Hurd to rise, albeit in an unconventional way.

You already have a de facto GNU OS at the moment in the form of the Guix System Distribution (GuixSD), what with it letting you configure the entire system in Scheme, perform transactional upgrades, system state rollbacks, it has its own init daemon and service scripts in Scheme, etc.

At the same time, we've been seeing some efforts in creating tiny builds of the Linux kernel [1] that go as far as to reduce syscalls, VMM algorithms, capabilities, character devices and so forth. There's also the recent Linux libOS effort to create a network stack in userspace as a shared library, though that seems about the extent of it. NetBSD then gives you a userland driver framework with rump kernels.

So what one can do is build a really stripped down Linux kernel, write a Mach IPC compatibility layer, run the Hurd servers on top of it and plug in the resulting product with GuixSD. And you now have the complete GNU system. As far as you'll ever get, anyway.

This has already been discussed before [2], but it's not something that has ever been given any priority. With the current climate, it might be worth a shot.

As to why anyone would want to do this... well, for one you get a full OS that is entirely configurable in Scheme with all of its services running as userland file servers (translators) in a sort of Plan 9-ish way, all the while supporting the Linux API. That's still pretty ahead of GNU/Linux. The Hurd also has features like running multiple instances of itself in user mode (subhurds) that can serve a similar purpose to containers.

It's obvious no one's going to port the Hurd servers away from GNU Mach, so this is the best shot anyone has. Any Hurd or Guix devs in here to comment on this?

[1] https://tiny.wiki.kernel.org/

[2] https://www.gnu.org/software/hurd/open_issues/linux_as_the_k...

Hurd developer here.

I also have high hopes for Guix, and in fact there is a gsoc project to port Guix to the Hurd. Guix tries to empower the user, a goal shared by the Hurd. They are a perfect match.

I do not believe that it is easy to provide the Mach IPC interface in the Linux kernel. The IPC mechanism is tightly integrated into the thread switching and scheduling systems.

Furthermore, I do not see much benefits in doing that, the main point would be drivers, which we want to run as a userspace process anyway. And we are doing that using the DDE framework. Rump is interesting, and it wouldn't even be that hard. It just lacks someone actually doing it.

What people don't realize is how compatible the Hurd is. We are shipping the glibc. Debian/Hurd contains ~80% of the software that is found in Debian/Linux. We have Firefox. The other day I deployed two Django sites on my Hurd development server to work out some issues I had with the Apache configuration I had on the production system. I was curious how it would go. All the Python stuff worked out of the box, I merely had to select a different Apache mpm module, because the default didn't work. Need to rescue a box? Append "init=/bin/bash" to the kernel command line to circumvent sysvinit.

Other microkernel based systems might have more ambitious goals, but many of them sacrifice compatibility to reach them. Some aspects in POSIX are hard to implement in a distributed fashion, for example fork(2). But what good is a fancy system if it requires a huge effort to port applications to it?

Right, the hypothetical tiny Linux kernel would probably be built without driver support and have NetBSD's rump kernel drivers in place of both Hurd's DDE and the monolithic Linux drivers, which keeps the userspace advantage.

It's true that a Mach port is quite tricky (particularly the virtual memory semantics).

Apparently there was an old abandoned attempt to port Mach on top of POSIX, which if someone obtains the source, might be a useful reference to see what went wrong or is incomplete [1].

It's worth noting that the reverse (Linux on top of Mach) was already done by Apple nearly a couple of decades ago as MkLinux, but they ultimately settled on XNU.

[1] https://www.gnu.org/software/hurd/open_issues/mach_on_top_of...

I'm not Hurd supporter but that's an interesting idea. I'd say replace Scheme with Python or something mainstream that's similarly powerful. Adding LISP to stuff usually kills it off unfortunately, with Clojure a freak occurrence. The transactional upgrades and rollbacks feature by itself would be compelling to some audiences.

"Adding LISP to stuff usually kills it off"

What? Oh, you mean like Emacs. Scheme is a beautiful language; it's flexible and trivial to learn. Replacing it with Python would be silly. Besides, Guile Scheme is the designated extension language of the GNU system. Gnucash, dmd (the init system), the Gimp, Guix, gEDA and many more GNU applications all can be hacked in or extended with Guile. In the GNU system Guile Scheme is "mainstream".

"Oh, you mean like Emacs."

No I mean like... surveys the market... nearly everything else. The LISP field is almost non-existent outside of academia and a few companies. Almost nothing mainstream in proprietary or FOSS is built with it. Plus, most programmers hate it. So, using something like in a FOSS app intended to go mainstream is quite a risk.

"Besides, Guile Scheme is the designated extension language of the GNU system."

Oh Ok. That they're pushing people to use an unpopular language at least explains why the apps you cited used it.

" In the GNU system Guile Scheme is "mainstream"."

Nah, most programmers haven't heard of it. Of those that have, I'd bank on only a subset even using it. I'd say it's used by a tiny set of developers, esp extension builders. And for tools who mostly each represent a small amount of developers or users.

You're really making my point for me by redefining the word mainstream to be "barely known software barely making it with a few exceptions." They should consider something that actually is mainstream as an experiment in increasing adoption. Probably some other things to reconsider while they're at it.

You mean like AutoCAD was killed by all the other CAD packages with a more conventional language? I'd like to see such a counterexample. Lisp is only killed by managers who are afraid to hire cheap labour. Like yahoo and their store.

What language is AutoCAD written in? And what percentage of successful products and projects use LISP? That's what I mean. It's the status quo of where LISP is in the marketplace. It's you that have to provide dozens to thousands of counterexampes to even illustrate a trend. They're not there.

Whereas companies adopting PHP, Python, Ruby, Lua, and so on have no time finding help, libraries, or customers. Because they're mainstream and attract such people. See the difference now?

> I'd say replace Scheme with Python or something mainstream that's similarly powerful.

I use Nix, which is a non-Lispy language which Guix is based on, and I've found myself wanting Lisp-like metaprogramming on several occasions; which Python isn't suited for. I might switch to Guix precisely for that.

> Adding LISP to stuff usually kills it off unfortunately

Do you have any examples? Lisp has been around since the 1950s, and Scheme since the 1970s; they seem to be pretty death-proof. Python's only been around since the 1990s, so it's got a shorter track record.

It's a great series of languages with plenty of advantages. There's still a small niche that uses it and one variant that's mainstreaming. Python is a knockoff of it with a decent standard library that grew wildly successful. It's why I recommend it as next best thing.

"Do you have any examples? "

Most articles written on history of LISP and its advantages come with examples of it going away in various companies or not getting adopted in general. Just try to recall how many of the following were done in (and still run on) LISP to see what I mean: processor support; operating systems; compilers; business apps; productivity tools; widely-used libraries; web services; popular FOSS apps or services; mobile apps or services; embedded systems. I'm sure your list in each is going to be tiny compared to the one's that chose alternative platforms.

That's the LISP effect. Both commercial products and FLOSS have continuously fleed from the LISP's. Those that used mainstream alternatives got plenty of success. Just compare... pick a variant... a LISP's success against C/C++, C#, Java, PHP, Python and so on. You can have a great idea but hardly anyone will contribute to it if it's LISP. Clojure's been the only exception, it's tied to JVM, and still is barely a blip [1] on the overall radar.

So, that's what I mean. You can use what increases odds of extension or adoption. Or you can use LISP with difficulty getting commercial or FOSS help. Now, the types of people that can churn out great LISP might be worth the extra effort and money. Jane St has a similar philosophy for Ocaml. Just worth knowing it's going to be an uphill battle compared to the alternatives.

[1] http://langpop.com/

I don't think you are far off. Lisp provides an added complexity with its code is data / data is code philosophy, that makes it harder to use compared to primitive scripting languages like PHP (which will have a hard time to leave its domain).

At the same time Lisp attracts a bunch of people because it is different. Which at the same time hinders its mass adoption, also secures its survival.

> tiny compared to the one's that chose alternative platforms

There are also a few domains where there are several Lisp applications or some which won't go away, even though the domain might be very specialized. For example in Computer Algebra: Maxima, Reduce and Axiom are all still maintained. In music composition there is a bunch of applications. Some new (OpusModus) and some old (PWGL, OpenMusic, Symbolic Composer). There are a few theorem provers used like ACL2 and PVS (Nasa). In CAD several Lisp system were bought off the market (for example iCAD by Dassault), but others still exist (like Genworks, or PTC's CREO elements). There are some of these clusters, small but interesting.

Shortly after the AI winter Lisp was mostly dead and only a few barely maintained implementations have remained. Now more than ten Lisp implementations are being maintained, two large commercial implementations (Allegro CL and LispWorks) are still under further development and new implementations still appear from time to time (like CLASP on top of the LLVM or mocl, a full program to C translator for mobile applications). Thus there are people who can maintain and develop complex implementations and there are some users out there.

It's small compared to PHP or Python, but it seems that there is a sustainable 'market'.

Your post is mostly accurate. Yet, the number of maintained implementations doesn't tell me anything about whether it's mainstream or worth using. Brainfuck language has a whole page worth of variants, implementations, uses, IDE's and so on. I doubt anyone will argue that we should use it for commercial projects. ;) A maintained, active implementation is only a start for a language.

For LISP, the market is small and fragmented. They've always had the problem of siloing off from each other while mostly not building enough standardized functionality to get momentum. The language also has lots of warts from the old days that it should eliminate (Clojure does a bit). They've mostly ignored all this and that's why the language doesn't go anywhere. That Python, inspired by LISP, lacks every one of these problems is probably a reason for its success.

Far as LISP's, I thought about relearning it and taking another go at it. Racket is the most interesting to me in terms of power, available libraries, community, and IDE. My old trick was doing imperative programming in LISP for rapid iteration and then auto-generating equivalent code to be handed off to a C or C++ compiler. Let me macro out the hacks needed for portability, performance, and security in many cases. Plus, interactive development + incremental function compilation is AWESOME.

Thinking on two lines this time. One is to redo my old approach with C and Go as the new targets. That lets me piggy-back on all their momentum with rapid prototyping and zero-to-low-overhead abstractions. Another is to reimplement Python & its standard lib in LISP in a way that integrates code in both. That should give a performance boost, a way into LISP for Python developers, and let me use Python code/libraries. Ton of work going into the latter.

Really busy so not sure if I can do either. What do you think about the options?

> Brainfuck language has a whole page worth of variants, implementations, uses, IDE's and so on. I doubt anyone will argue that we should use it for commercial projects.

Well, I fear Brainfuck does not have commercial vendors and users willing to spend money on that. Lisp has that.

> whether it's mainstream or worth using.

It's not mainstream, but there are people finding it worth using.

> That Python, inspired by LISP, lacks every one of these problems is probably a reason for its success.

Python is not at all inspired by 'LISP' and not even by Lisp.

> My old trick was doing imperative programming in LISP for rapid iteration and then auto-generating equivalent code to be handed off to a C or C++ compiler.

Common Lisp compilers do that C code generation for me. No need to write macros for that.

> Another is to reimplement Python & its standard lib in LISP in a way that integrates code in both.

That has been tried already. Doesn't interest Lisp developers much. For a Lisp user there is little benefit of using a mix of Python and Lisp. Either use Python without Lisp or use Lisp, especially since Lisp has the better implementations and allows a wide range of applications.

> What do you think about the options?

The Lisp to C options exists already in multiple forms (GCL, ECL, MKCL, mocl and a few in-house implementations). Another Lisp to C++ is under development: CLASP.

Python in Lisp also exists.

https://common-lisp.net/project/clpython/

"It's not mainstream, but there are people finding it worth using."

Remember where this discussion started. They're trying to get Hurd mainstream. A person made a suggestion on that. It uses something that failed to mainstream almost every time and is mostly hated by mainstream programmers. So, I suggested not to use it if the goal is making something very popular. In light of that, LISP can have as many tiny communities as it likes: won't help this use case.

"Python is not at all inspired by 'LISP' and not even by Lisp."

My bad. Misremembered: it was ABC.

"Common Lisp compilers do that C code generation for me. No need to write macros for that."

Compiling a functional program to efficient C isn't the same as writing inherently efficient C in a functional language. An amateur like me threw together one in a short time while the other took many PhD's worth of effort to get in good shape. Most of them still don't outperform my old system because they try to solve a harder problem.

"That has been tried already. Doesn't interest Lisp developers much. For a Lisp user there is little benefit of using a mix of Python and Lisp. Either use Python without Lisp or use Lisp, especially since Lisp has the better implementations and allows a wide range of applications."

Makes sense. Appreciate the review. :)

"The Lisp to C options exists already in multiple forms (GCL, ECL, MKCL, mocl and a few in-house implementations). Another Lisp to C++ is under development: CLASP."

"The Lisp to C options exists already in multiple forms"

I think you misread my comment. Your list are all LISP implementations with C or C++ support. My tech was a C/C++ implementation on LISP. A subset, more honestly, as I never needed the whole thing. I macro'd useful patterns to make it more like a BASIC-style 4GL. Those macro's had straightforward mapping to highly efficient C. The rest, much like BASIC, had straight-forward mapping. My style was imperative, the constructs were imperative, the target was imperative, and data types etc were all consistent. The result was a productivity boost over developing in C or C++ with interactive development, better debugging, less coding, automated checks at the flip of a switch, & automated generation of all the source & makefiles.

I haven't seen anyone post something similar in years. So, what do you think of that kind of tool as a compromise between highly efficient, portable C and the safer, higher-level languages that aren't efficient enough for eg operating systems? Is such a thing worth rebuilding until functional compilers are good enough to wholesale replace C/C++ in their niche areas?

"Python in Lisp also exists."

In one paragraph, you say it's pointless and nobody is interesting. Then you give me a link to an implementation in another. Puzzling you are. Thanks for the link, though. :)

> Compiling a functional program to efficient C isn't the same as writing inherently efficient C in a functional language. An amateur like me threw together one in a short time while the other took many PhD's worth of effort to get in good shape. Most of them still don't outperform my old system because they try to solve a harder problem.

Lisp is not necessarily 'functional'. It's also quite imperative.

There have been in the past Lisp compilers which generated 'readable' and 'maintainable' C from Lisp code. The compilers I mentioned didn't really have that goal.

> My tech was a C/C++ implementation on LISP. A subset, more honestly, as I never needed the whole thing. I macro'd useful patterns to make it more like a BASIC-style 4GL.

I remember reading about users writing C in Lisp, with added use of macros to make code generation easier.

> The result was a productivity boost over developing in C or C++ with interactive development, better debugging, less coding, automated checks at the flip of a switch, & automated generation of all the source & makefiles.

Sounds good.

> Is such a thing worth rebuilding until functional compilers are good enough to wholesale replace C/C++ in their niche areas?

Hard to say. I doubt that C/C++ users will use Lisp-based tools - unless its really really good and useful.

> In one paragraph, you say it's pointless and nobody is interesting. Then you give me a link to an implementation in another. Puzzling you are. Thanks for the link, though. :)

'Nobody' is maybe the wrong word. 'Almost nobody' is probably better. The Python in CL implementation exists for some years, but I haven't heard of larger adoption or much further development. It was developed at a time when Python integration seemed to be more important. But it's definitely a nice try and may have some interesting bits in it...

Reply on news.ycombinator.com