QEMU 6.0
qemu.org
qemu.org
Is there good CLI tooling available for virtualization? I would say that I'm looking for the "on-prem" experience where I start by configuring a virtual network, download some unmodified ISOs directly from the operating system vendor, provision my headless virtual machines with a combination of Powershell, Answer files, Bash and Kickstart. I then provide services like DHCP and DNS using my virtual machines rather than the built-in mechanism provided by the virtualization tooling.
If your want high level networking stuff, use virsh or virt-manager GUI. Or maybe proxmox if you want a whole distribution for it.
I wrote a script that pulled some info from a sourced-in bash "config file" for a given VM (such as amount of CPUs, RAM, and where the disks were), executed the appropriate ip commands to create the taps needed for network access (including a private inter-VM network), and then built/ran the long QEMU command with it in a screen session.
Which file is the ISO for the virtual CD ROM is just another QEMU command line parameter.
I had my own bind running in a VM and created a view on the same subnet as the private VM network. Since this was only a few VMs I didn't bother with DHCP, I just statically assigned IPv4s. I used VNC to setup the OS in them.
Larger virtualization installations will already have a management layer, something like oVirt or vSphere, that has similar tools available.
Also LXD's command line tool being
lxc
(with c instead of d) while LXC's being lxc-*
is an abomination. Who wouldn't be confused by that?• virsh — This[1] is libvirt's shell interface; and gives you access to the rich set of libvirt APIs.
• virt-builder — Use this for rapidly building minimal or customized virtual machines; it's greatly flexible; check out its man page[2]. And here's[3] a quick example that connects both virt-builder and virsh together.
• virt-install — Use this if you don't like the default build of the template images from virt-builder; it lets you create "headless" servers via 'kickstart' and Linux OS trees from the command-line.
• guestfish and libguestfs suite[4] — This rich set of tools help you in a variety of use-cases: repairing your broken disk images, editing, cloning, debugging disk images, and more. It has saved my behind a lot of times.
• qemu-img[5] – This Swiss Army knife lets you powerfully manipulate disk images (QCOW2, raw, et al) offline. Example operations include: create images, backing chains, offline snapshots, disk image merging, ability to convert disk images from one format to another, and more.
[1] https://libvirt.org/manpages/virsh.html
[2] https://libguestfs.org/virt-builder.1.html
[3] https://developer.fedoraproject.org/tools/virt-builder/about...
[5] https://qemu.readthedocs.io/en/latest/tools/qemu-img.html
Terraform is dog slow sometimes due to cloud provisioning. Would be nice to just be able to build the VM locally then push the AMI/VHD into AWS or Azure.
$ virt-builder fedora-32 --size 10G --format qcow2
Then the (xz-compressed) template Fedora 32 image will be cached under ~/.cache/virt-builder. So your subsequent Fedora 32 image provisioning will be much faster.You might want to give it a whirl and see if it satisfies your needs; `virt-builder` should be available on most major Linux distributions.
$ virt-builder fedora-33
[ 1.5] Downloading: http://builder.libguestfs.org/fedora-33.xz
[ 2.3] Planning how to build this image
[ 2.3] Uncompressing
[ 8.6] Opening the new disk
[ 13.8] Setting a random seed
[ 13.8] Setting passwords
virt-builder: Setting random password of root to mZbPJw9d1ZHgowBk
[ 14.8] Finishing off
Output file: fedora-33.img
Output size: 6.0G
Output format: raw
Total usable space: 6.0G
Free space: 4.7G (79%)
By the way, interesting virt-builder factoid: It uses a STRIPS-based planner to optimize the order of steps when building the image, so it doesn't waste time doing multiple copies.
https://en.wikipedia.org/wiki/Stanford_Research_Institute_Pr...
https://github.com/rwmjones/guestfs-tools/blob/0cffcbb7848af...
https://github.com/libguestfs/libguestfs-common/blob/74bc5c5...
https://github.com/libguestfs/libguestfs-common/blob/74bc5c5...https://drewdevault.com/2018/09/10/Getting-started-with-qemu...
I stopped bothering with libvirt and other frontends for local dev environments. Now I use shell scripts to start my VMs. I only create one per month or two so it isn't that much of a hassle.
I also have just a few 20 line scripts to run my VMs, and that's all. Works with no changes/hassle for 5 years already or more. No extra SW to learn. Man qemu gives me all the answers. Easy.
I guess if you want to manage some complicated setup and dynamically add/remove disks, network cards, or whatever, all the time, migrate machines, etc., some solution like virsh would be good, but for having everything on one bridge, in one subnet, to run some throwaway VM with Windows/browsers for web testing, the simpler setup is so nice.
https://www.packer.io/docs/builders/qemu
https://github.com/dmacvicar/terraform-provider-libvirt
https://blog.ruanbekker.com/blog/2020/10/08/using-the-libvir...
I include the blog link because there is some nuance in how to get the path right for community Terraform providers that aren't in the Hashicorp registry. The documentation on the GitHub project isn't quite up to date with respect to how the latest versions of Terraform expect the plugin paths to be set up.
I've done this pretty successfully with all the major Linux distros minus Arch, which requires some bootstrapping to get an iso that Packer can work with (no such thing as an answers file for Arch). It's not that big a deal, though. Just find some instructions on how to create and mount a cloud-init iso in addition to the installer iso and use that to add an ssh public key so you can script the installation steps externally. I actually think Packer can do this, but I just haven't gotten it to work yet and have relied on shell scripts.
Hyper-V actually has a very comprehensive PowerShell module that is pretty well documented, by the way: https://docs.microsoft.com/en-us/powershell/module/hyper-v/?.... I've found it pretty easy to use and actually got the Arch auto-provision working on Hyper-V in Windows before I got it working in KVM in Linux.
Another thing is you can just use the cloud images and cloud-init for bootstrapping everything pretty easily, even on-prem. cloud-init has a "no cloud" config option, as mentioned above, where you just mount an iso with the config data as a DVD drive and cloud-init will find it automatically when the distro iso boots.
This guy has a pretty comprehensive example of how to set up a kubernetes homelab entirely using the libvirt Terraform provider from Ubuntu cloud images bootstrapped with cloud-init: https://github.com/zloeber/k8s-lab-terraform-libvirt
I use Hyper-v for years and I rarely create new VMs, this is why I use GUI, it is convenient if you don't do this often enough to remember the syntax and params. Most people that I know are in the same boat, but for mass deployments or automated deployments PowerShell is the way to go.
Sorry for the approximative English, I'm losing my vocabulary with WFH and I'm pretty sleep-deprived.
I've been building a game streaming platform for influencers for the past months. We scaled to 9k users in a month.
We have android x86 running on QEMU-KVM and there are a bunch of issues I need to address.
- virsh (and virt-manager) always add a PS2 mouse to the guest. The issue is PS2 mouses have a relative movement and not an absolute one -> When you use a mouse or the touch screen through VNC you have to move the pointer to a position instead of directly clicking/capturing the mouse
- My VM resolutions are 720x1080 but VNC always launches in 1920x1080. I have black rectangles on both sides of the Screen.
- Streaming the app viewport with VNC has some (small) latency, any way to fix it?
- Can we host an arm android emulator on QEMU? (armeabi-v7 or v8)
- We had a sales presentation from Canonical for their anbox cloud solution, but they never returned to us with a proposal. Is there any other way for us to have the Android VMs in K8S? Or even Lxc if possible, we are flexible. Also, the juju charms for anbox is nowhere to be found
If you want to get in contact, my email is jadiaheno[at]ludexgames.com I can even show you a demo, we have influencers increasing their revenue with our Beat the boss campaign. We're not VC backed.
You can try to get support in #anbox on freenode, though. There are postmarketOS people that know how to get the system working. I've never seen someone use anbox with qemu, though. It has always been same arch.
Trying it and i'll let you know.
Will see if i can make some pull requests. Thank you for your help, it's appreciated.
Hosting ARM is possible but performance will be ~10 times worse.
https://github.com/robherring/generic_device/wiki
I have personally run this through KVM on arm64.
I have been using android x86, quite successfully even with touch input. When I did it I didn't use virsh, instead I just used ffmpeg directly via CLI.
>> - Can we host an arm android emulator on QEMU? (armeabi-v7 or v8)
It is very possible to run arm code on a x86 machine, this would however be slower of course.
If you are running on a VPS you might be able to get one with an ARM CPU that supports virtualization.
>> - Streaming the app viewport with VNC has some (small) latency, any way to fix it? There would be other ways that you can use to remote connect to it such as spice. You might also want to consider running it on an headless X server and then use basically any remote control program.
>> - - My VM resolutions are 720x1080 but VNC always launches in 1920x1080. I have black rectangles on both sides of the Screen. This sounds as one of the points in the changelog, maybe try to use the newest version?
Also: how do you pass files into QEMU? Does it have software similar to Virtualbox's 'guest additions' software?
For Windows, I have samba running on my host.
QEMU doesn't have much of a GUI to speak of. It's a virtualization framework with some CLI tooling on top of it. Frontends/management UI's/etc are left as an exercise to the community so you can find one that suits your needs.
For example, a popular one for macOS these days is UTM: https://mac.getutm.app
QtEmu is another: https://qtemu.org
Edit: I realised GUI might mean creating and managing virtual machines; virt-manager might be good? Take a look at https://wiki.archlinux.org/index.php/Libvirt for the setup needed. (I think it's easier to learn to use vanilla QEMU, though.)
You are looking for Virt-manager or Boxes or any other GUI tool based on qemu if you want something similar to the vbox or vmware experience.
> it have software similar to Virtualbox's 'guest additions' software?
Yes, theyre called spice tools. Work for both linux and windows and infact pre installed on popular linux distros.
There are of course tools that provide a friendlier GUI wrapper if the command-line isn't your thing.
I have had a lot of trouble with machines getting corrupted, not working at all, locking up, but I did get a Debian machine working that runs ARM Linux on my Macbook Air M1.
[1]: https://github.com/benpye/nix-config/tree/main/overlays/qemu
[2]: https://gitlab.freedesktop.org/slirp/libslirp/-/commit/72713...
Are both original Xbox emulators built off of QEMU (Xemu is a fork of XQEMU). I've only used Xemu, but performance was pretty good for the games I tried on it (it doesn't have a way to upscale rendering yet though).
My Workstation runs on Windows, and up until some years ago I was using VMware Workstation for virtualization, and then moved over to VirtualBox, which I've also started to use on a Windows-based home server. I've always used both of them via the GUI.
Now, if I want to build a beefy Linux server (headless) and run some virtualization on it, would I then use QEMU and get the same thing I've been getting with VirtualBox on Windows?
How do both of these compare performance- and feature-wise? Are they alternatives, or do they server different purposes?
It offers a nice Web UI for QEMU and LXC and makes the whole experience quite good.
I came from VirtualBox too and never looked back.
Where the virtualization bit comes in is with kvm. Kvm is a Linux kernel module that lets you take advantage of x86 CPU hardware virtualization. Qemu can use kvm to provide the virtual CPU while it itself emulates the rest of the PC hardware, resulting in a complete virtual system.
That said, qemu can also emulate the CPU itself, allowing for example an x86 OS to run on an ARM system, or vice versa, but that of course is slower.
> TCG > Added support for Apple Silicon hosts (macOS)
I tried an WinXP VM on my M1 Air just for fun but got the performance of an early Pentium. I know emulation is slow but was hoping for a little more.
For those that missed it, here is a recent article on the project...
https://www.pressreader.com/australia/edge/20210422/28424953...
It was sufficient to boot Linux in some configurations but not complete enough of an emulation to run other things. This is in general the story with many things in QEMU.
I didn't keep notes, I suppose I should have. I might revisit this again someday.
How do things stand these days?
Also, given that QEMU is somewhat platform independent, how well does it run an x86 OS on top of an ARM chip?
Would be nice if they linked a changelist with all of the commits instead of this^.
QEMU Internals - https://news.ycombinator.com/item?id=26941744 - April 2021 (33 comments)
Show HN: QEMU front end for M1 and Intel Macs - https://news.ycombinator.com/item?id=26260390 - Feb 2021 (122 comments)
How to run FreeBSD 13 in QEMU on Apple Silicon Mac - https://news.ycombinator.com/item?id=26053983 - Feb 2021 (39 comments)
QEMU Advent Calendar 2020 - https://news.ycombinator.com/item?id=25262608 - Dec 2020 (30 comments)
Changed-block tracking and differential backups in QEMU - https://news.ycombinator.com/item?id=25189078 - Nov 2020 (5 comments)
Booting a macOS Apple Silicon Kernel in QEMU - https://news.ycombinator.com/item?id=25064593 - Nov 2020 (70 comments)
QEMU 680x0 support [video] - https://news.ycombinator.com/item?id=25027213 - Nov 2020 (13 comments)
QEMU should move from C to Rust - https://news.ycombinator.com/item?id=24133128 - Aug 2020 (209 comments) (god help us)
macOS in QEMU in Docker - https://news.ycombinator.com/item?id=23419101 - June 2020 (186 comments)
Xnu-QEMU-Arm64: iOS on QEMU - https://news.ycombinator.com/item?id=22870905 - April 2020 (34 comments)
QEMU for iOS - https://news.ycombinator.com/item?id=22385370 - Feb 2020 (145 comments)
QEMU VM Escape - https://news.ycombinator.com/item?id=20796446 - Aug 2019 (57 comments)
QEMU 4.1 released - https://news.ycombinator.com/item?id=20713932 - Aug 2019 (14 comments)
QEMU v4.0.0 released - https://news.ycombinator.com/item?id=19736309 - April 2019 (159 comments)
QEMU Advent Calendar: A surprise disk image each day until Christmas - https://news.ycombinator.com/item?id=18588043 - Dec 2018 (34 comments)
Emulate Mac OS 9 with QEMU - https://news.ycombinator.com/item?id=18585157 - Dec 2018 (64 comments)
From VNC to reverse shell - https://news.ycombinator.com/item?id=18051264 - Sept 2018 (9 comments)
Osx-kvm: Run macOS on QEMU/KVM - https://news.ycombinator.com/item?id=17763855 - Aug 2018 (106 comments)
Almost booting an iOS kernel in QEMU - https://news.ycombinator.com/item?id=17544689 - July 2018 (19 comments)
Running Amiga-like OSes on QEMU - https://news.ycombinator.com/item?id=17531490 - July 2018 (12 comments)
Accelerating QEMU on Windows with HAXM - https://news.ycombinator.com/item?id=15759080 - Nov 2017 (24 comments)
VM Escape: QEMU Case Study - https://news.ycombinator.com/item?id=14218600 - April 2017 (44 comments)
QEMU: user-to-root privesc inside VM via bad translation caching - https://news.ycombinator.com/item?id=13928675 - March 2017 (28 comments)
Multi-threaded emulation for QEMU - https://news.ycombinator.com/item?id=13759283 - Feb 2017 (9 comments)
QEMU: virtfs permits guest to access entire host filesystem - https://news.ycombinator.com/item?id=13753950 - Feb 2017 (43 comments)
QEMU Advent Calendar - https://news.ycombinator.com/item?id=13118291 - Dec 2016 (37 comments)
OS X-KVM: Running Mac OS X El Capitan on KVM and QEMU - https://news.ycombinator.com/item?id=12556609 - Sept 2016 (89 comments)
Announcing qboot, a minimal x86 firmware for QEMU - https://news.ycombinator.com/item?id=9585631 - May 2015 (43 comments)
QEMU Advent Calendar 2014 - https://news.ycombinator.com/item?id=8712998 - Dec 2014 (16 comments)
QEMU 2.0.0 Released - https://news.ycombinator.com/item?id=7604916 - April 2014 (74 comments)
Virgil3D - a virtual 3D GPU for qemu - https://news.ycombinator.com/item?id=7042299 - Jan 2014 (1 comment)
QEMU 1.7.0 is now available - https://news.ycombinator.com/item?id=6883988 - Dec 2013 (16 comments)
QEMU 1.6.0 is now available - https://news.ycombinator.com/item?id=6220565 - Aug 2013 (18 comments)
QEMU 1.4.0 released with 95% of native IO performance - https://news.ycombinator.com/item?id=5231545 - Feb 2013 (51 comments)
QEMU 1.0 released - https://news.ycombinator.com/item?id=3303958 - Dec 2011 (19 comments)
Qemu + KVM is the future of open source virtualization - https://news.ycombinator.com/item?id=1226598 - March 2010 (2 comments)
Also readconfig/writeconfig
It sucks that Qemu has drifted so far from normal users