I get my stuff from Homebrew, just Rails + MySQL, no Docker, no fancy stuff. I'd love to have a fast macOS VM to run software I don't trust like Zoom or Skype and a second one to run my work projects, so they don't spill over my personal stuff, but AFAIK the virtualization story is still pretty incomplete on the M1 (or is there a way to run an arm64 macOS VM without a gigantic performance hit?).
My initial attempt at getting a running system was to install an x86 VM via qemu / utm. It worked but was about 1 /4 speed of my old MacBook.
Round 2 was to an arm64 VM via qemu / utm (utm is using qemu here, but I guess it’s almost running purely on the M1?). That’s blazing fast but I don’t have everything I need compiled for it. So I’m recompiling what I can and whatever I can’t I’m then using qemu user mode within the arm VM to run the x86 binaries. The VM itself is really fast, the x86 code, obviously isn’t, but I’ve at least minimised the times I need the emulation.
Not sure of a better approach, but as I say, definitely open to hearing what else I can do.
Forget about full system emulation, especially with Qemu’s approach which doesn’t leverage the host MMU. Emulating an MMU alone is very costly perf-wise. That means that you should use an arm64 OS and then use an x86 compat layer on it.
For Qemu’s user mode emulation, it doesn’t exactly provide good performance either. But it is still much better than full system emulation.
Mac OS is arm64, with an arm64 Ubuntu VM, then I’m running qemu in that for specific single x86 processes.
To clarify, I’m using UTM to run the VM, so it’s using qemu - but it looks like that doesn’t give you a performance cost in itself.
(I assumed FSE from your 40x worse perf figure, which is a very bad case)
Not sure about Parallels.
In Parallels it's on by default but you need to allocate a lot of RAM to the VM: https://kb.parallels.com/en/125105#section6
Pinned tracking thread:
https://forum.parallels.com/threads/big-sur-installation.351...
It’s not as easy to get working compared to a windows vm because I believe it uses the OS recovery image but it’s not that hard to setup and works well. I’ve used it in the past to test my dotfiles setup on a perfectly clean install.
As developers we trust so many different libraries. And it is important that they are safe when used in production code.
But we shouldn't have to worry about accidentally installing a library which uploads our emails or our browser data. By working from a VM we can prevent that.
The worst a malicious library can do from a VM is upload our SSH keys or source code (which is still bad).
I'd much rather we solve the security problems using better local sandboxing for software, like how it works on our phones. That would help end users as well, and it would stop crypto ransomware and all sorts of other attacks. Or alternatively, run my dev tools from a solaris zone or a freebsd jail or something, both of which have no performance impact.
Docker on Mac has a lot of IO overhead. This is mentioned in the article:
> The way I made it work is by having a full “dev environment” running virtually. That means that I check out repositories on the virtual disk and run everything from there. This has the slight inconvenience that I can’t easily access those files with Finder, but the upside is that there is no noticeable IO latency issues like when running Docker for Mac.
If you just want improved performance with bind mounts instead, you can use :cached or :delegated flags.
I think Fusion comes with a driver for Docker
https://code.visualstudio.com/docs/remote/remote-overview
Virtualized Linux on x86 AND being able to run VSCode on your Mac for great fonts, WiFi, screen etc without having to fuss with drivers.
Of course ssh and tmux also work well for this purpose.
Mixed scale DPI screens are not a problem if you use Wayland, which is basically the default now.
Font rendering though? YMMV.
And thats sort of the theme everywhere. Its super fast and I love it, and its mostly stable and mostly great. But tiny bugs shine through all over. I've been totally spoiled by Apple's spit and polish.
For example, I get random graphics bugs after waking from sleep sometimes. Sometimes my mouse cursor is either invisible or for some reason duplicated, so I see a second stationary cursor hovering over my windows. And for some reason my second display doesn't vsync, so I get obvious tearing when I scroll or move windows around.
I use a trackpad, and smooth scrolling works properly in most apps. But Firefox needs an obscure XInput environment variable to make it work. (That trick is only mentioned deep some a bug tracker). Smooth scrolling doesn't work at all in IntelliJ. Intellij also doesn't let me use the Meta key (Cmd / Start) as a shortcut modifier - so my muscle memory for navigation is all messed up and I can't rebind keys to fix it. I hate how Ctrl+C is copy everywhere except the terminal, which needs me to Ctrl+Shift+C instead. Etc etc. Forever.
Its unbelievably responsive compared to my 2016 MBP though. If you haven't upgraded in awhile and you can afford it, its a fantastic time to get a new system. But linux on the desktop still isn't entirely pain free. Way better than it was a few years ago though.
...or use Wayland where smooth scrolling should work by default
I have a laptop running Ubuntu and connected to my 4k monitor in front of me. Has an Intel gpu. I do not even want to use multiple displays, I’m gonna go with the external only.
Wayland can do fractional scaling but then VsCode, Firefox, Discord and whatever apps I tried became blurry shit. Firefox has an environment variable to fix it. VsCode has an experimental build and command line flags. I have no idea about Discord. I don’t care if its the app developers fault. It is just plain bad.
X11 can do fractional scaling but fonts are a little blurry for some reason and I have the worst screen tearing I’ve ever seen in my life. It is unusable.
macOS and Windows do this perfectly. It is not something you think about. Windows has some weird looking apps here and there but the shittyneas is not even close.
For context, I have (currently) a precision 5520 laptop;
I run arch with sway, I have a big USB-C 4k at work and 2*16:10 FHD Dell USB-C monitors at home.
The only issues I have with linux is that I don't have Microsoft Office, and Zoom+Wayland is buggy as hell.
Use the closest integer scaling and adjust default font sizes (and if needed other sizes like window title height or panel height).
It's also best if you just don't buy monitors that have dpis that are not close to integer multiples of 96.
Maybe I should try 2x gui scaling and 0.75 font size? Never occured to me that I could go below 1 on the scales, it might work.
edit: 2x on wayland is still blurry. 1x + 2x font works but as I said, tiny buttons and stuff.
Still, my point is; why the fuck am I dealing with this?
In mobile where there is market, Google did it.
I stopped trying a couple of years ago (after I got a 4K monitor) to convince myself that desktop Linux is workable.
Linux for desktop = console
X11 apps do that. Firefox can run in Wayland mode, so run it in Wayland mode. vscode and other electron apps do not support wayland yet, so until they do (yes, they are taking their sweet time), you either need to run with integer scaling, or tolerate them being blurry.
In Windows, the shittyness is much worse. Basically any Qt app is unusable with hidpi in Windows.
In my experience, it isn’t. On my setup (200% + 100% screens, Retina MacBook Pro in Boot Camp) KeePassXC, qBitTorrent, Qt Designer all look great on both screens; VLC has slightly weird fonts on the HiDPI screen but still works fine.
Tangentially, I want to mention that the archlinux wiki is a terrific catalog of machines and their known quirks (and sometimes, fixes for the quirks!).
In Fedora maybe. Not in most distros. And recall many apps have to run under xwayland for compatibility so all of those apps won't support scaling properly.
It's still not there yet.
Battery life also takes a huge hit on the Dell XPS and HP Elitebooks I've tried, even after trying to apply some tweaks (not that I should need to). For this reason I use Windows+WSL on laptops even though I'd prefer Ubuntu.
People are advocating remote development heavily, so there must be some *huge* benefit that I don't understand. So much so that people are willing to give up amazing features of an IDE. To get VSCode to a usable state requires 16 third-party plugins, and you're left with about 1/3 of JetBrains functionality, in which the features aren't quite as good. YMMV I guess?
It's still beta so they have a lot of work to do.
Have you tried the Projector beta yet?
However, the 2012.1 releases now support running code on via WSL2 or SSH, which is closer to VS Code's setup. https://blog.jetbrains.com/idea/2021/02/intellij-idea-2021-1...
https://www.jetbrains.com/help/go/creating-a-remote-server-c...
Their old server config is just pointing to a repo I think, but isn’t remote dev.
I looked into this last year before their recent announcement and at the time they said it wasn’t on their roadmap, but looks like that thankfully changed.
You mean this literally? It took just that one plugin for me, when experimenting with using another machine on my network as the dev environment.
for me, a TS using node nerd, vs code does these things well:
1. ts language server
2. decent plugins (that i can commit to scm)
3. yarn 2 support
4. nice debugger interaction
Off the top of my head, VSCode lacks a decent code formatter and a "search everywhere"-type command palette. And the Git and database support is not good either, but that's kind of a given.
EDIT - child comment is right, the following paragraph is not true! I was writing from my phone and remembered installing this extension [0] but now that I've checked it is for counting offsets from the beginning of the file, not lines and columns. Which I'd concede is not so much of a basic functionality expected in any editor.
[0]: https://marketplace.visualstudio.com/items?itemName=ramyarao...
And I mean *really* basic stuff... like a status bar label
that shows the line and column numbers where the cursor
is! Yes, you already need an extension just for that.Usually, the only plugins I need to install is the language server plugin for whatever language I'm using. At least, that works for Go and Python while Node and JS/TS work right out of the box.
Benefits are more obvious on massive code bases that take a long time to build, test (or really just do anything).
IntelliJ will get stuck indexing forever and can be frustrating to use - having the code live on beefy servers that can run the computationally expensive bits faster can make a big difference.
However, JetBrains unfortunately has nothing coming close to the transparency and speed of the VSCode remote connection, with your source living only on the remote side.
In cases where being able to work on remote, for example very particular configurations, or docker containers, or WSL2, this makes a huge difference.
For me this is mostly Python and TypeScript these days, where VSCode has grown particularly strong in terms of IDE features.
VS Code Remote is a game changer. It's how IDEs should work. It allows you to run all the GUI chrome locally for responsive editing, while letting the remote do all the heavy lifting (including building, debugging, testing, and deploying). It finally overcomes the latency and other usability issues of using a wimpy local box to connect to a powerful remote box to write software.
I never used VS Code much and otherwise prefer JetBrains myself. But the remote development extension changed my workflow permanently, and I now recommend it to all my colleagues who develop cloud/data intensive code.
Recently, I've settled for using my remote machine as the source of truth for my codebase (a monorepo) and only sync specific directories I work with locally.
Remote docker interpreter (python) + this is a workable-enough solution. It's not ideal - I still have to switch to the terminal to run git, tests, etc. Compared to this, the VSCode way is just simply better; especially if you don't use all the features of a full-blown IDE.
If so, why do you need rust compiler in the first place? May be you don't? May be it's wrong compiler?
We probably have a long ways to go before we get there and it does come with its own sets of challenges and usability quirks even if the technical implementation is good.
For example, 8 years ago I used to run Windows 7 with xubuntu running in a graphical vmware VM using Unity mode[0]. Basically a way to seamlessly run graphical Linux apps in Windows. Each GUI app you launched from the Linux VM would have its own floating window that you could move around like any other Windows window. As an aside, this feature has been removed from vmware for years now when it comes to Linux guests.
It worked well enough for then, and I spent 99% of my time in that VM (browser, code editor, everything) and I only used Windows for recording / editing videos and playing games to avoid having to dual boot.
But even with vmware's really good disk performance there were performance issues at times, you're also splitting your memory up between your main system and your VM, it's not that efficient. Then there's little quirks like your main OS not really fully being able to integrate with files and apps from the VM, so you have to do hacky things to get apps to launch from a taskbar, search doesn't work because your stuff is in a VM, etc.. Plus you always feel like you're split between 2 worlds, the main OS and the VM. It doesn't feel really nice and cohesive.
To a lesser extent nowadays we have WSL 2 on Windows which is what I use. It solves a lot of the VM problems from above and running an X-server lets you run graphical apps very well, but you still feel like you're running 2 operating systems and the user experience suffers because you don't feel like you're running 1 streamlined OS.
A prime example is having to put all of your files in WSL 2's file system to get good performance but having certain types of files there is an inconvenience or you may not want to do it because it doesn't make sense to put 100GB of storage files on your SSD. That happened to me because I have a podcast site which is mostly code, except for a ton of raw wav file recordings + mp3s. Instead of just having a git ignored directory in my project, I had to create a 2nd directory outside of WSL to store these files. There's many other examples like this too.
I don't know what the Mac story is like, but I would imagine at minimum you're dealing with files being split into 2 worlds and will experience the unfriendly split OS feeling overall. Does Parallels let you seamlessly run floating windows across your VM and macOS?
[0]: Here's a really old video of that set up https://nickjanetakis.com/blog/create-an-awesome-linux-devel...
> we have WSL 2 on Windows which is what I use
My new work laptop runs Windows, so I'm interested in WSL2. I gather that it has good integration with Windows (eg you can type notepad and notepad will open)- which is convenient but removes any security boundary.
> Does Parallels let you seamlessly run floating windows across your VM and macOS?
I don't use Parallels, or Mac. But I believe so- they call it Coherence https://kb.parallels.com/4670
I agree that not everything is as convenient. My papercuts were not being able to type `code .` to make a new VSCode window. And not being able to use the new tab shortcut in my terminal to make a new tab in the current directory.
I've made a little project to solve both those issues for me https://github.com/ccouzens/ssh-nicety It works, but is fairly bespoke to my setup. It uses Unix Domain sockets (which probably excludes Windows). The only new terminal it can launch atm is gnome-terminal.
If you decide to use it, I have another more up to date video with my whole WSL 2 / Docker / all the tools I use / etc. set up at: https://nickjanetakis.com/blog/a-linux-dev-environment-on-wi...
It worked well, except that I occasionally want to run a container within my dev-environment. Running a container inside a container is possible, but not easy.
I settled on running a Vagrant Libvirt VM. I do not know what the performance impact is- but I have not noticed either my laptop or the VM being slow.
Do you know of a set of settings or tips to follow for making containers as safe as a VM?
You can use your windows or Mac laptop with visual studio code to do web development. ng serve works as if you ran it from your laptop.
The biggest drawback I can think of is if you ever needed to leave the house. Thankfully, I have no life so I don't have this problem.
Add ZeroTier to the equation and as long as you have Internet connection when you're outside the house you'll probably not even notice it.
I just use ssh. SSH to a VPS and map 22 to a port on the VPS.
I have a really small VPS for this.
I think poking a hole on port 22 is mostly ok if you only allow key authentication and no password authentication but I don't know enough to give advice on security.
I use a VPS with SSH. I have to ssh in, then I can ssh into the machine at home.
For safety, key authentication and fail2ban would cover a lot. I mainly have the 1 port.
If I need to expose another SSH port to the internet, I can do it, but yes, it would need extra protection since logs are coming from the machine ssh'ing in.
Changing the port from the default 22 to something else is also recommended, if only because it makes fail2ban logs way less verbose.
Does it solve the security problem by preventing access to the majority of your home directory?
Or solve it by only allowing you to install vetted libraries?
A better solution would be a desktop OS with proper application sandboxing. Mac OS is taking many steps in that direction.
Linux - as usual - has a multitude of solutions, all of them problematic. (AppArmor, SeLinux, Firejail, Snap, Flatpak)
Many devs just blindly install things from npm, copy random snippets from GitHub gists, execute remote shell scripts via curl that they just copy and paste into the terminal because they read it in some blog post, and on and on.
Anyway, just my worthless < 2 cents.
1: https://docs.microsoft.com/en-us/windows/security/threat-pro...
Additionally, VMs might be a great help to complicated debugging. You can crash a VM without taking down your desktop. I'm not sure about things like kernel debugging, but it might be easier in a VM.
Are containers a hack that will go away as VMs become lightweight or will containers replace VMs?
I run proxmox, and when I first set things up I used VMs, but over time I moved most server kinds of things to containers.
EDIT: docker is a special thing - creating an entire environment from one Dockerfile is pretty powerful.
I ended up going with a VM, as it still allows (IIRC) the thin client to be thinner. There's definitely bits of containers I miss, like I'm back to keeping notes in a markdown file for commands that would be added to a Dockerfile, but I'm also not having to do weird things to get something like my shell history to persist.
I've been reluctant to contribute to nodejs (e.g. electron) projects for some time, because I just don't want to run npm on a computer with any kind of remotely private data.
Lately there were just too many itches to scratch, so I went for a VM replicating my normal setup (dotfiles etc.), and I just use x2go, locally. Quick and dirty setup which is good enough when used infrequently.
My ideal setup would probably be closer to https://blog.jessfraz.com/post/docker-containers-on-the-desk..., but it's more setup than I could be bothered with at the time. Maybe one day.
It has become remarkably seamless and trivial to switch any of the local/remote pairs over time, and definitely cleaner than managing various app runtimes on my local machines (I have cloud-config templates to bootstrap fresh Go, Java and Node boxes as required).
edit: forgot to mention I'm posting this from another of those combos, a Windows VM I remote to from my iPad whenever I need a desktop browser
My dream is to code web apps from my iPad from the couch/bed as I sit at a big desk and monitor all day for work and I just want to chill in the evenings on something smaller and more comfortable.
I use blink for the terminal app and connect using mosh instead of ssh. I found that mosh handles the connection (reconnection) way better since iPadOS is pretty aggressive in killing the terminal app if I switch to a different app. I use also tmux on the server and just detach it when I'm done in case I want to work on it on my laptop or desktop. Overall works great, my only issue is that the 10.9" iPad screen is a _little_ bit too small for my liking so I don't do work like this that much. If I had the 12.9" iPad it would probably be something I use daily.
Remotes can be anything: I have a KVM host at home (that I remote to from my Mac for Docker dev) and plenty of Azure VMs.
I am currently running Parallels Tech Preview on the MacBook Air M1 and primarily use PyCharm (remote interpreter and deployment to the VM). The whole thing works better than expected considering it’s still a preview release. Battery lasts around 12 hours, sometimes an hour or so more depending on what else I run.
I am currently working on a Django app. When saving while the debug server is running I can command tab to my REST client and make an API request and the change was already deployed and the server restarted. Despite dealing with a VM the whole thing is just fast.
Recently I’ve been doing this with VSCode which has a remote dev mode that works amazingly well. Before that I was just using ssh and tmux/screen which, as we know, also works and has worked for decades.
When running remote VMs I usually run them on my ESXi box in the basement and VPN back home when traveling. This is especially nice when a project needs more resources than whatever device I’m working on has to offer. But beside this very specific use case I haven’t personally found any advantage of this setup.
> I had two requirements for developing that I wanted to achieve: macOS UI, Linux-based dev environment
What exactly is meant by a linux-based dev environment? Seems like the idea is to run the whole dev environment is in a virtual disk in a VM. I'm puzzled, but ok. It then goes on to set up Ubuntu Server in this VM, which is then used to host the dev environment.
Wouldn't simple running a docker instance both be less cumbersome, far more resource efficient, and quick to iterate, than literally installing an OS on a virtual disk?
---
To summarize, unless I missed something, this looks to explain that it is possible to run a VM on a MacOS. Add "M1" and it's the top post on hacker news? What's going on here?
Docker on Mac can be dog slow. This is appealing to me.
On Mac, docker works by installing a VM- so the two aren't so different.
I (not the author) prefer using a VM as a development environment, because at some point I'll want to run a container and nested containers are tedious.
Any piece of information that untangles this mess is helpful to me. Of course this may not be the same for others, but it could be 'what's going on here'.
As an alternative could you use React Native and borrow an Apple product during compile day?
It's definitely a long term strategy, but people love eating Candy even if it gives you a stomach ache.
Have built an Ubuntu 18 environment running through UTM (https://getutm.app/ - running qemu). Took a bit of tomfoolery where I had to install using a console only display and then flick back to full graphics to get the machine to boot.
Use port forwarding to talk to the machine - haven’t figured out a way to do bridge mode like I can with virtual box.
We’ve got a bunch of crazy dependencies that I’m in the process of rebuilding. Most seem to be ok. There’s a third party one we’re a bit trapped with but we’re running that emulated x86 within the vm. You can also use docker the same way within the vm.
Performance wise the arm vm is blazing fast. Seems to be about 20x faster than an x86 vm on my old MacBook 2015. It’s about 2x the speed of an 8 core graviton2. When running emulated x86 code, the speed drops to about 1/4 the speed of my old 2015 MacBook. Not ideal but in our case we’ve only got a single non arm dependency and it’s not used often, so it works for us.
It’s a bit of a leap, and if you have more crazy binary dependencies like we do, you will have do to a little work to get things running.
Having said that, the machine itself is _amazing_. It’s a real joy to be on a machine with this level of responsiveness.
Pretty much.
I had bad experiences running Docker directly on MacOS. The IO latency was unbearable. I know they are working hard on it so maybe it's better now, but this setup works well for me.
What exactly are you guys running that causes it to be "unbearable"?
Most of the performance issues with Docker on Mac happen in setups where source code or other files are volume mounted from the Mac filesystem into the container filesystem.
All the Docker pseudo-ports just run a Linux VM on the host OS and set up Docker inside it, for you.
Docker images are not portable, they run on Linux or on Windows, but they can never run on both.
https://hub.docker.com/_/microsoft-windows-base-os-images
And WSL2 is Linux running in a VM on Windows.
A simpler solution I had - One linux vm, SSH connection plugin in VSCode and a simple 4 line SSH config file (~/.ssh/config) does magic.
Here's my config file -
Host <hostname>
HostName <Hostname IP>
User <User>
IdentityFile <Identity File Path>
LocalForward 127.0.0.1:8000 127.0.0.1:8000
LocalForward 127.0.0.1:7000 127.0.0.1:7000
I use the terminal inside VSCode - which means I can manage docker(-compose), microk8s, etc and anything I spin up, I'll just be able to access from my local host during testing.
1. Low-end Chromebook (good battery life), remote server, VPN.
2. High-end Chromebook (there are a few i3 and i5 models with 8GB RAM), Linux environment.
Are you often in locations where you don't have Internet access?
Some laptop built around the new Ryzen 9 5900HS cpu [0] seemed like an obvious choice. But although it seems like AMD has released it, I'm having trouble finding any actual laptops that have it as an option. Maybe I'm just not looking hard enough?
[0] https://www.amd.com/en/products/apu/amd-ryzen-9-5900hs
UPDATE: Maybe I just needed to wait a little longer: [1]
[1] https://www.ultrabookreview.com/35985-amd-ryzen-9-laptops/
For my personal projects I've been able to switch from using VirtualBox to Docker as a Vagrant provider, and it works well enough for what I need it to do.
I created a cookiecutter template for Django projects at https://github.com/tmiller02/cookiecutter-django-react-ansib... that I use for development on my M1 mac using Vagrant + Docker.
Thanks for the tip, that's good to know. I'm running RC2 and haven't come across any issues like that, although I don't run my Docker containers in 'privileged' mode when using Vagrant.
Any idea if Parallels would work better on the M1?
I'm currently using Docker for Mac but will move to UTM (a.k.a a nice UI atop qemu-hvf) when I have some time at hand.
Vagrant I only used for some other OS VMs (e.g smartos) but the base images are x64 so there's no chance it works well (if ever) on ARM either.
Is it because some kind of beef with ARM or they think it is technically impossible?
A port to Arm wouldn’t be really a port but a rewrite.
Then the author could access the files via Finder.
It would still result with VS Code running more on your client than it would when using VS Code Remote.
Port tunnelling, while totally possible with an SSH command in a new terminal, is something VS Code just sets up automatically (and makes it easy to add your own).
Yes. Except on Windows it is easy to access WSL files.
Unsurprisingly, WSL2 does something similar (it uses the 9p protocol instead of SMB though).
I want to run multiple VMs at the same time and my current quad core iMac struggles. So I was thinking of getting a beefy Lenovo Workstation to use as a dev environment and use the VScode remote ssh thing. But in that case I don't really to upgrade my iMac.
(a) HN has people who like new technology.
(b) This particular new technology is also very good.
What's difficult to understand about it?
People have been tinkering with WSL and Linux to have all kinds of things working "in random ways that arent supported yet etc", M1 is not that different...
I've been using Linux development laptops for the past decade and have had all of these benefits!