FreeBSD on EC2 FAQ | Hacker News Reader

FreeBSD on EC2 FAQ

daemonology.net

49 pointsl0stman13 comments

I've been a FreeBSD fan since 1997. It is nice to see FreeBSD running on EC2. However, for die hard performance junkies, I have built something faster and more efficient. For people that love all the BSD variants out there, my startup - tegataiphoenix.com - offers FreeBSD, NetBSD, and OpenBSD virtual machines (in addition to all the other usual suspects). The performance we are getting is around 500%-600% faster than Xen based BSD virtual machines. Mention HN, and get a discount. :)

Hmm. User for 30 days; karma 1; one submission, promoting your startup; one comment, promoting your startup.

Welcome to Hacker News! I hope you'll stick around for a while and contribute.

I've been a fan of your work for ages Collin, but your attitude needs adjustment.

Please don't feel too offended. It's totally normal for people to check out profiles when someone posts something interesting. Your know-how on BSD virtualization IS interesting to a lot of folks.

The performance we are getting is around 500%-600% faster than Xen based BSD virtual machines

I don't know where you're getting your numbers from, but Xen doesn't have a 5x performance penalty. I'd believe you if you said that QEMU or Bochs had a 5x performance penalty, but Xen is more along the lines of 5%. (Of course, if you run an instance and give it a small fraction of a CPU, you get what you pay for.)

I'll go ahead and publish the benches shortly.

I don't know much about virtualization, but could anyone explain how sites such as [1] and [2] are able to virtualize OpenBSD? Do you think they develop the Xen support internally?

[1] http://bsdvm.com/

[2] http://www.rootbsd.net/blog/2010/openbsd-4-8/

Short answer: There's two types of virtualization.

With full virtualization, you can run unmodified operating systems (in theory, at least). VMWare is an example of a fully virtualized system. Recently Intel and AMD added features to their CPUs which make this much easier.

With para virtualization, the guest operating system is aware of the hypervisor and cooperates with it -- for example, rather than modifying page tables directly, it sends a message to the hypervisor saying "please map this page for me".

(There's also a hybrid 75%-virtualized model, where the system is fully virtualized except that some devices are exposed via hypervisor calls -- this is seen most commonly as Xen "HVM with PV drivers".)

Great, thanks for the explanation.

I see virtualisation as a promising way forward for FreeBSD. Hardware support has always lagged Linux (and still does). The limited resources of BSD hackers could be more appropriately concentrated on needed areas instead of constantly playing hardware catch-up.

Hardware support has always lagged Linux (and still does).

Not everywhere. Last I checked we do considerably better for 10GbE, for instance. We certainly consistently lag behind Linux for desktop hardware support, but I think we hold our own pretty well in the server space.

As a BSD hacker who hates anything hardware-related, however, I do love the idea of outsourcing such irritants, though.

I held this view ("hold own in server hardware") until I actually tried to run FreeBSD on modern server hardware. The Intel 10G NICs worked, but slower in benchmarks than Linux and Opensolaris.

SCSI/SAS support has been rotting slowly, with the now industry-standard LSI cards I tested, working, but without interrupt coalesce and thus using more CPU and reaching only 60% of the speed of the same card in Linux or Opensolaris. Infiniband support is nearly all happening in Linux.

Then there was the problems with the new Core i7 CPUs and Turbo Mode. NUMA support has been missing until 9.0(unstable), and even now its only for the allocator and not the scheduler. The final straw was when I had USB issues (modern server motherboards have mostly dropped PS2 ports) which meant keyboard wasn't working at all on a server.

ZFS is less stable and slower than in OpenSolaris, and for all these years FreeBSD has lacked a journalling filesystem. Which I consider basic stuff, after all, Linux has had XFS for 10 years now. Linux has several virtualisation options, and while VirtualBox has been ported to BSD, it isn't really suited for server use. FreeBSD Jails have stagnated for years and still cannot handle running multiple copies of PostgreSQL because of SYSV support.

I believe FreeBSD is the best and most enjoyable OS for hacking on, but this was 2 years ago, and things have further stagnated since then, so like many others, I have been forced to give up and move on.

The Intel 10G NICs worked, but slower in benchmarks than Linux and Opensolaris.

That's interesting. Which NICs are these? For most 10GbE cards we blow linux out of the water.

ZFS is less stable and slower than in OpenSolaris

That was true a couple years ago; I don't think it's true any more.

for all these years FreeBSD has lacked a journalling filesystem

True, but we've had softupdates, which is better in most circumstances. (And in HEAD we now have journalled softupdates, which gives you the best of both worlds.)

FreeBSD Jails have stagnated for years and still cannot handle running multiple copies of PostgreSQL because of SYSV support.

You have an interesting definition of "stagnate". Jails have recently gained full network stack virtualization, for instance. I can't remember if SYSV has been virtualized yet, but if not it's certainly coming soon.

I believe FreeBSD is the best and most enjoyable OS for hacking on, but this was 2 years ago, and things have further stagnated since then

Personally I'd say that the past 2 years have been the most exciting years of FreeBSD development for a long time. Maybe when 9.0 is released you'll give FreeBSD a chance again.

Reply on news.ycombinator.com