Android in Docker without QEMU/emulator | Hacker News Reader

Android in Docker without QEMU/emulator

github.com

226 pointsPascalW71 comments

Came across this very useful project. It provides Docker images with Android running directly in Docker, without qemu or an emulator. I've never seen this before, all other solutions that I'm aware of either run the Android emulator in Docker or use qemu directly (like Anbox).

Advantage of this is that it's very lightweight and does not require VT-X or AMD-V, ideal for running in cloud environments that typically do not expose this CPU capability.

Anbox doesn't use QEMU. It uses the same underlying kernel tech as Docker, LXC and other isolation tech. This is why you need the ashmem and "binder"(?) kernel modules to run Anbox.

I'm currently "trying" to get Anbox working on NixOS (It's currently broken on 5.x kernels but should be fixed by https://github.com/NixOS/nixpkgs/pull/102341) so that's why I know that Anbox is the "same" as this.

I personally haven't managed to get anbox to run at either acceptable performance or any kind of level I would consider stable. Someone posted waydroid as an alternative here a few months ago, but I haven't tried it yet.

Waydroid seems performant enough to me. SuperTuxKart is very playable (though I didn't try much as on-screen controls are hard to handle with a mouse). Microsoft excel also works.

A lot of android apps only come with arm library, and need an extra emulation layer on x86, I haven't tried that.

Waydroid is a more performant successor to anbox.

Interesting. I've never been able to make Anbox run in a cloud environment though, don't recall the exact details. But indeed I guess it should be able to work as there's also a commercial offering targeted specifically at cloud setups. https://anbox-cloud.io/

Anbox and Waydroid are others, though they don't use Docker.

https://github.com/anbox/anbox (https://news.ycombinator.com/item?id=24684187)

https://github.com/waydroid/waydroid (https://news.ycombinator.com/item?id=28616985)

ReDroid: Requires Docker. The images seem to be based on pure Android. Multiple versions available: 8.1, 9, 10, 11, 12.

Anbox: Requires 'snapd'. The image is based on AOSP based on Android 7.1.1.

Waydroid: Requires Wayland. The image is based on LineageOS that is based on Android 10.

Consider taking a moment to look into the difference between "emulator" and "virtual machine".

What the link refers to is an kind of Android emulator, but not the virtual machine kind of emulator.

If you are tempted to write a rebuttal in the "reply" box, answer this question first: Why are programs like xterm and urxvt called "Terminal Emulator"? What do they emulate?

Hi blueflow, I see that today you are not tired of arguing semantics.

The issue here is context. What GP means by Android Emulator refers to specific piece of software, the AVD emulator provided by Google which emulates full android device, it emulates more than just VM, it's also have skins and physcal buttons.

https://developer.android.com/studio/run/emulator

What exactly is Docker enabling here beyond a Docker workflow, which is quite orthogonal to the presence of a vm or emulator?

Isn't the enabling technology here just kernel-level android bits? Docker is fluff, yet ends up dominating the headline.

Does it not provide the mobile-like environment that apps expect?

I mean, cgroups, firewall rules, and chroot are also already provided by the kernel, but using them for running containers without Docker, or Podman, or LXD, or other piece of "fluff" is slightly unergonomic.

If it were Docker providing the secret sauce, Docker would be all you need to make it happen on non-Linux hosts.

It's the Linux kernel, configured with the appropriate Android modules, not Docker making this happen.

Interesting concept but seems to require custom android-specific kernel modules (ashmem, binderfs), which means the android containers aren't very containerized but leak into the host kernel.

The Docker container also needs to be run in privileged mode, so you should probably treat this as if it were an emulator running root. It does work around dependencies on VT-x and such, though, and cuts down on the performance overhead of virtualising an extra kernel.

Or try running it with podman.

All Linux containers leak the host kernel no?

The Anbox project relies on the exact same 2 modules: https://docs.anbox.io/userguide/install_kernel_modules.html. While these are "android-specific", they are present in the linux kernel tree, and even come pre-installed in Ubuntu 19.04 (and probably more distros).

> All Linux containers leak the host kernel no?

Curious in what way?

See any explanation of what a linux container is[0]. Containers in linux are a patchwork of various features that "contain" a process group in different ways while sharing the kernel with the rest of the system.

Running either of

    docker run --rm --entrypoint /bin/uname alpine:latest -a

    uname -a

gives you the same kernel version. The only difference will be in the hostname.

[0]: https://embeddedbits.org/introduction-linux-containers/

That's the difference between a container and a VM. VM runs its own kernel. A container relies on the host kernel and the host machine's kernel does all the scheduling, memory management, etc.

See the other replies and if you're on Linux run

  ps -ef

on the host and you'll see the processes in the containers too. That doesn't happen if those processes run in a VM.

This is by design. The kernel still runs the processes inside the container. If you namespace the user running `ps`, it will not be able to see the container processes. The only reason you're seeing them is because you're in the default (root?) namespace.

The container on the other hand cannot see the host's processes or other things. At least not without an exploit.

Not all linux kernel leak to host. Docker security is way better than nefore. And there also projects like podman and docker unprivileged that don't require any root intraction to set up a kernel

Depends on your definition of "leak" I suppose. You can't get a different kernel version inside a container so there must be a "leak" in some sense. Reducing capabilities etc will just leak lesser information about the system in general, but your kernel is always shared.

Sounds scary. That's a no from me dog.

Don't forget waydroid: https://waydro.id/ . Last time I tried, it worked reasonably well.

This is interesting, but I don’t see this as something for use in development of Android apps. Given all Android’s quarks and fragmentation the use of real devices and Google’s emulators will provide a better surface to catch bugs during dev. What’s the primary use case for these containerized versions of Android?

Software, websites testing?

I'm using it myself to run Whatsapp for the Matrix Whatsapp bridge [1] on a VPS.

[1] https://docs.mau.fi/bridges/go/whatsapp/android-vm-setup.htm...

required params (already added in docker image)

    qemu=1

How is that without qemu?

Yeah not reading anywhere in the docs that it works without qemu. Also the need for kennel modules to be available on the host and passed through makes it less portable than most containers.

Those kernel modules (`ashmem` and `hwbinder`) are standard when building the Linux kernel for Android and they are already part of the upstream Linux kernel tree [1][2]. Because this solution uses containers (which share the same kernel with the host OS), these additional modules are needed.

From the ReDroid kernel modules readme:

>Custom Kernel

>If use custom kernel (5.0+), you can enable binderfs and ashmem configs; So the kernel modules in this repo are not needed any more.

https://github.com/remote-android/redroid-modules#custom-ker...

More info here on Android-specific kernel modules here:

https://elinux.org/Android_Kernel_Features#List_of_kernel_fe...

[1] ashmem

https://github.com/torvalds/linux/blob/master/drivers/stagin...

[2] hwbinder

https://github.com/torvalds/linux/tree/master/drivers/androi...

It's just a parameter to let the Android OS know that it's running in an emulated environment as opposed to running natively on a physical device.

For example, the GPU stack has to be somehow emulated using a custom OpenGL driver (in this case `mesa3d`). There's software emulation (swiftshader) and host GPU mode (GLES).

For performance reasons when using the host GPU, the GLES commands from the guest have to be serialized and sent over a kernel pipe to the QEMU and to the host OS. The commands are deserialized on the host and executed on the physical host GPU through the Shader Translator library (taken from the Google ANGLE project). The Google's QEMU fork (aka Android Emulator) usually loads this library and then takes care of proper rendering, such that users can see the rendering output of the emulated OS properly inside the emulator window on the host machine. You can do the same for the docker, otherwise you'd have to implement some other trick to share the GPU. `virtio-gpu` is a Linux kernel facility that is meant to replace QEMU pipe in the future.

The high-level details of this OpenGL emulation mechanism are described here: https://android.googlesource.com/platform/external/qemu/+/ma...

You can find the implementation of the QEMU OpenGL pipe mechanism here: https://cs.android.com/android/platform/superproject/+/maste...

Will it work on WSL? It can be a good way to run Android on Windows.

Windows (11?) has the feature built in right?

That feature's still in beta (https://docs.microsoft.com/en-us/windows/android/wsa/). You can use it by joining the Windows Insider Program.

Or you can manually install it following the various guides explaining how to do so without joining Insiders

According to its README, it does run (and is tested) on WSL 2 (Ubuntu) (amd64).

How is this accomplished? Is this using Anbox or something similar I don't understand how the Dockerfiles linked here end up running an instance of Android.

It runs an Android userspace on the host kernel, inside a container

That Dockerfile is just an image to _build_ the Android container

See instructions to build it (or pull the published version)

https://github.com/remote-android/redroid-doc/tree/master/an...

Then you can run the Android container

For development work I use the default Android emulator that comes with Android Studio.

Are any of these ones superior in terms of performance, resource utilization, etc? The cloud angle is mildly interesting although in my specific use-case I'd probably just end up paying for a service that can provide access to emulators in the cloud.

Is there more scope to do things like automating inputs etc this way? How would one go about that?

> Is there more scope to do things like automating inputs etc this way? How would one go about that?

adb or UIAutomator

For tips on how to control the Android Emulator from the outside with adb, see https://stackoverflow.com/questions/7789826/adb-shell-input-...

This one specifically seems to run Android in the Linux Kernel using the usual suspects of container tech. It should have next to zero performance impact so it will definitely have better performance compared to the default Android emulator considering that's using qemu, even if it's qemu-kvm.

Whether it's as reliable as running a VM, is something that'll need to be tested. Personally I got an error when trying to exit out of the shell (and thus was unable to exit), something I've never seen in the VM version of Android.

Did you try

  :q!

?

Sounds like QEMU doing that. QEMU has a command mode - ctrl+A followed by X should kill it.

This isn't running on QEMU.

Trying to fix some Rust docker builds that are being killed by QEMU on Apple Silicon right now. I always believed Docker was cross platform, I had no idea how much of a pain it can be to use with buildx and --platform to get stuff working. SIGSEGV errors at random points, non-deterministic errors, GAH!

That you (and many others apparently) had the impression that docker was cross platform says a lot about docker's markering, considering docker as far as I know it is a wrapper for a series of extremely linux-kernel-specific APIs. :-/

Yep! Amusing to think I used to think cross-platform just meant running on Windows and Linux and that was fixed with Docker. Had no idea the rabbit hole was deeper until Mac released the new CPUs.

This could be useful for running WhatsApp inside a vm instead of your phone for those rare instances where it is needed.

What do you mean? Where is it needed?

i believe they are referring to the whatsapp bug that the saudi prince used on jeff bezos. simply sending bezos a message gave the saudi prince root access on his phone and he dumped all his messages and photos and released them to the press. running in a vm would just have whatsapp so they can send and receive messages without exposing all the private information on their phone in case of some 0day

So you would run the VM on a phone? Because if not, you'd need to carry an open and connected laptop around.

You could connect to the VM using somelike like TeamViewer running on your phone.

You would run TeamViewer on your phone 24/7 with a persistent connection to a server? Now would you receive the notifications?

There are unfortunately situations where the only way to reach someone is via WhatsApp. I have avoided them but i can see WhatsApp in a VM being useful for those purposes.

Exactly, what I am going to use it for. I tried using whatsapp for long in a VM of android x86 image didn't work too well for me.

Frankly I just install Island, and then run Whatsapp (and all not-always-needed apps) inside it. Plus bonus: you can freeze the apps you don't need and keep the island running.

TIL about the Island app. Thank you!

Shelter.

https://f-droid.org/en/packages/net.typeblog.shelter/

https://github.com/PeterCxy/Shelter

Also very useful, thank you!

I recommend you to use web.whatsapp.com then, it's pretty decent now (except that it does not support calls)

That needs whatsapp running on your phone to work as far as I know (because it basically runs the communication on the phone and then syncs that to the browser)

If you can work out how to get on the beta release of web.whatsapp.com then it's no longer required for your phone to be online.

but you still need to feed a QR code to the android's "camera" on each browser you want to use.

Multi-device [1] which is now in beta lets you use WhatsApp on the web without the phone connection.

[1] https://faq.whatsapp.com/general/download-and-installation/a...

I'd rather use waydroid to avoid the attack surface of docker and its complexity.

Perhaps a simple question, but... why?

Because alternatives have some requirements that may not suite some people.

1. VMs have relatively big resources overhead and this approach requires you to download some Android image first.

Of course you'd want some clean distribution and there simply aren't many such ones (that I know of) that are easily available. The ones I tried - were a 2-in-1 shitshow+slideshow for me. Maybe it's because I used ISO (for QEMU/KVM), as I didn't want to run proprietary crapware (VirtualBox/VMWare).

2. Going with Anbox requires having crapware snapd and their Android image is quite old (Android 7.1.1).

3. Going with Waydroid requires having Wayland, which if you don't use it - is an overkill to install just to have Android on your PC.

4. Going with Android from AndroidSDK - is probably an even harder way to get access to some Android on your PC (I don't really know, I haven't investigated that option).

Reply on news.ycombinator.com