Noah: Bash on Ubuntu on macOS | Hacker News Reader

Noah: Bash on Ubuntu on macOS

github.com

159 pointsprabirshrestha48 comments

This runs linux binaries on darwin. You know what would be awesome? Running darwin binaries on linux so I can do iOS development without a Mac.

Binary compatibility is one path, but another path is source / API compatibility. That’s been more or less a dream for a long while, with a fairly far along implementation:

https://en.wikipedia.org/wiki/GNUstep

But for whatever reason, it’s not really caught on. Even Mono managed to chase C# / .Net more impactfully than GNUStep has chased Cocoa.

Had GNUStep ever become more widely used amongst the Free *nixen, I feel like we could have had a credible, open alternative path to macOS/iOS (not to mention Linux) development.

ElementaryOS with Gnustep beneath is my daydream.

Weirdly, the groupware server I use (SOGo) is written for GNUstep. It's the only app I know that uses it otherwise.

There is darling, which could probably need some more development resources: https://www.darlinghq.org/

Hello, I am one of the Darling developers. Yes, we would love more help! A good deal of command line-only stuff is working at this point. GUI application support is limited but we are making progress.

We are also looking to improve the experience of first-time contributors and we welcome feedback related to how we can make it easier to give you the information and tools you need to contribute to Darling.

I’m curious as to what the current state of the project is, as well as what kind of things you were working on. I heard that you can use Xcode’s toolchain to compile binaries now?

I'm glad you're interested. We really should write progress report blog posts or something, similar to Dolphin emulator or RPCS3. Yes, the Xcode command line tools should work for the most part when running in Darling. If you try to use Darling for development please create issues on our GitHub if an individual tool crashes or otherwise fails to work.

Unfortunately, git doesn't work at the moment, however this can be worked around by using Linux git since Darling works in your Linux home folder.

One cool thing that works is lldb, which for the most part fully works.

Darling also supports mounting disk images and installing .pkgs, which is really helpful.

In short, we support most of what you can do from Terminal already.

Homebrew and MacPorts currently don't but once we get them working they should make it easier to install packages in a Darling prefix.

We also have binary compatibility with standard Mach-Os from macOS.

Unfortunately, the other project members (around four of us) and me are often busy with our lives and don't always have the time needed to advance Darling, so we are exploring how we can improve outreach to attract more developers.

While we are similar to the WINE project in goal, at the moment we have nowhere near the resources they do, so it is difficult to accomplish our goals.

> We really should write progress report blog posts or something, similar to Dolphin emulator or RPCS3.

Sure. I’m already pretty happy with the informational blog posts you have already; the linker one that was posted earlier this year was quite enlightening.

> we are exploring how we can improve outreach to attract more developers

No promises, but I might take a look sometime.I currently have easy access to macOS so I generally don’t have a good reason to run darling, but if I ever do and I run into a bug that annoys me I’ll see what I can do to help.

I'm quite interested in Darling, mostly for the purpose of cross-compiling from Linux -> Darwin. We cross-compile a lot of things for Julia, however cross-compiling executables that need to bootstrap something, (e.g. building something that then needs to run to generate output which is then used in further builds) can often get tricky. There are of course workarounds that can be utilized in order to get this to work (build a Linux-native version and a Darwin version, then use the Linux-native version of the software whenever a bootstrap is required) but this is tedious and not foolproof.

Bah... I had to check what it was called and it took me a few minutes.

Latest commit 19h ago https://github.com/darlinghq/darling

I'd love an iOS equivalent of mingw-w64, which is what I use to produce Windows binaries from my 100% libre GNU/Linux-based development environment.

I prefer VIM as my editor but when I write iOS/Mac apps I just suck it up and use XCode (with vim key bindings). While building apps outside of Mac would be cool I think you'll always find most engineers getting sucked back to XCode unfortunately. It is so verbose that I find the integration XCode has with the documentation to be super helpful. I'm sure you can get this with other IDEs but if it's not VIM it might as well be XCode for me…

I hate IDEs. I’m sure I’m not the only one. Graphical interfaces are not optimal compared composable command line interfaces, nor is the mouse/trackpad an ergonomic input device.

It’s all just a matter of preference.

I for example detest opaque keyboard shortcuts and prefer a mouse or trackpad. I still get plenty of work done so it’s optimal for me.

It's not a matter of mere preference when using non-ergonomic devices results in debilitating injury (carpal tunnel syndrome) which prevents me from working.

Yeah, I prefer vim for sure but XCode with vim key bindings is a decent compromise

With vim bindings are you still having to manually strip out xcode's code signature to install the plugin? That was the state of affairs last time i tried to do it.

Yeah, you still have to resign with your own thing in the keychain. It’s kind of weird and it caused me to hold off for a while. Eventually I caved in because I can’t stand using the mouse so much.

I think this would be much harder because Linux is a monolithic kernel and doesn't allow subsystems the same way Darwin does since it is a hybrid kernel (like NT)

Linux have the binfmt https://en.wikipedia.org/wiki/Binfmt_misc . It have been used to run java, windows pe or other arch binaries.

That's very different.

All binfmt does is allow exec() and friends to run a custom wrapper program; for example, qemu or wine.

Windows / NT subsystems (and specifically, here, picoprocesses), effectively intercept all entries into kernelmode from a particular process; when you execute a syscall instruction, the kernel invokes subsystem-specific code.

One of the child comments of the one to which you replied gives an example of a Linux subsystem for running Darwin binaries, so clearly it’s doable.

Yes, that would be great. At the moment I can't do any iOS dev simply because I don't have, nor want to run, OS X.

Expo offers to build IPAs on their servers.

https://docs.expo.io/versions/v31.0.0/distribution/building-...

I wonder if this could be used to get Docker running on a Mac without painful file system performance (a la Docker for Mac, running Linux in a VM). Could something like this project work for that? Are there Darwin equivalents for container stuff like namespaces?

Maybe it could work. The Mac is odd though in not having bind mounts so it is not going to be easy to fix the filesystem performance using this. Would use less memory though potentially. Maybe you could fake binds in the emulation layer.

(I work at Docker and used to work on Docker for Mac).

? Docker on Mac uses HyperKit. Mac already has container primitives and a hypervisor.

I think there’s an English as a second language problem with the title.

As oautholaf points out, this is a subsystem that runs Linux binaries under Darwin, not a subsystem for Darwin compatibility under Linux.

It's presumably modeled after: https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux

It's wrong because of where the "Subsystem" is, and because you start parsing the sentence with that word.

OK, it's a "Subsystem", which is a subset of a system. What kind of subsystem? A Windows subsystem! This part of the system does Windows... and you're already wrong. The Windows part isn't what the subsystem does, it's what the subsystem is a subset of. The subsystem does Linux and it's part of Windows. Saying it's a Windows Subsystem for Linux implies it's something like WINE, only moved into the kernel, if you're old enough to remember the "personalities" concept.

An alternative explanation for anyone who's still confused: it's one of Windows's potentially many subsystems -- i.e., it's Windows's Linux Subsystem -- and hence it's the Linux Subsystem for Windows, not the Windows Subsystem for Linux.

That said, I think their interpretation was likely different: it's the =Windows subsystem that is meant for Linux, kind of like how earlier they had SUA, the (Windows) Subsystem (meant) for Unix-based Applications.

Indeed, although even that is worded awkwardly.

Out of legal necessity; see this comment from HN user bitcrazed, who worked on WSL: https://news.ycombinator.com/item?id=13697918

Wow that is a bad excuse for inaccurate phrasing.

"Windows Linux Subsystem" would be better and still fit that criteria. Just like the old thing they used to have was called "posix subsystem". Throw "Windows" at the start and you are good...

I've been wondering if something like this could be done, but using actual syscalls - tagging processes as Linux or Darwin and dispatching syscalls differently, as with Linux personalities or (AIUI) Solaris branded zones - instead of with a hypervisor. I think you could do this with a kernel module that hooked the syscall entry point, assuming SIP lets you do such a thing. I'm surprised they used a VMM, but I guess it's not that hard to implement via Hypervisor.framework.

Maybe there's no significant overhead to a VMM if it's hardware-accelerated?

With SIP enabled you are still free to load arbitrary “signed with an Apple-approved signature” kernel extensions, so I don’t see why your technique wouldn’t be possible. You’re natively executing most of the code and intercepting syscalls either way, so I doubt there would be much of a difference in performance. I haven’t done anything serious with noah but it’s been relatively performant.

Very cool.

I found this presentation by the authors of Noah that explains more about how it works:

https://events.static.linuxfound.org/sites/events/files/slid...

I wonder can this be used as a replacement to XQuartz? Especially, if what I only need is for ssh X11 forwarding.

BTW, XQuartz seems to be dead now.

That's for two reasons:

1) Mac users are persnickety about UI and cannot suffer to use the standard tatty UI that prevails in the Linux desktop world.

2) X in general is moribund, as everyone knows Wayland is the future. What's really needed now is a Wayland compositor that can render to Quartz.

Anyway, XQuartz doesn't give you the capability to run Linux binaries -- only to run software commonly available under Linux that's been source-compiled to macOS. Noah allows you to run a full Linux binary distribution under macOS without virtualization.

The only use case of XQuartz for me is the ability to ssh into a Linux server with X11 forward so I can remotely control the GUI app running in the server. That’s why I ask whether Noah would be good replacement for such purpose.

I agree with you that Mac users are persnickety about UI. In fact, one of my complaints on XQuartz is the lack of retina support. As for wayland, as far as I understand, it does not support network forwarding (yet?)

> As for wayland, as far as I understand, it does not support network forwarding (yet?)

Indeed, because Nobody Uses That.

Why the downvote?

How well does this work in practice? What is compatibility like? I assume most GUI apps don't run?

I have used this successfully to demo throwing shellcode at vulnerable (command-line) binaries and getting a shell when I’m too lazy to fire up a VM, and it does a pretty good job of this. Unfortunately ptrace isn’t supported, or I’d ditch the VM entirely and be able to use a debugger when writing and testing exploits in addition to just demonstrating them.

Why not just use a proper hypervisor? Emulation is fragile, imperfect and not a good use of time.

I assume that by "proper hypervisor", you mean a hypervisor that runs a full kernel. This project does use a hypervisor, but apparently only to set up memory mapping and trap the syscalls for translation. Hypervisors that run a full kernel typically end up emulating hardware host interfaces, which leads to its own set of challenges and often introduces overhead that isn't present when emulating at the kernel ABI level (e.g. operations going through both the guest and host VFS/FS/SCSI stacks). I don't think it's obvious that one approach is clearly superior to the other overall.

This uses macOS’s Hypervisor.framework.

Reply on news.ycombinator.com