DigitalOcean now supports FreeBSD | Hacker News Reader

DigitalOcean now supports FreeBSD

digitalocean.com

552 pointsbarium196 comments

The fact that DO had to make this announcement at all is a sign that things have gotten worse for VPSs.

Before, when a company provided Xen or Kvm, you generally would get to have low-level access such as the ability to virtually connect to a serial port or vnc session of your box as it booted. You also, typically, could provide your own ISO images.

Even if you couldn't provide your own iso, being able to interact with the VPS in the above way would allow you to use one of the provided disks and then bootstrap the install of another (this is how I installed gentoo on many providers that didn't "support" it)

DO's stance that you must use one of their images, you can't upload your own, and you can't even use your own kernel (I'm not kidding! If you "sudo apt-get update" to get a new kernel security update and reboot, DO will IGNORE your shiny new kernel because they hardcode the kernel as one they control. See [0]).

This is terrible. We shouldn't be happy that they're adding FreeBSD to the list of images they allow you to use, we should be showing, with our wallets, that their restrictive setup that doesn't allow you to touch anything outside of their tiny garden and exposes you to security issues is unacceptable. We should be using other providers, like Linode, AWS, and GCE, all of which allow bringing your own image in some form.

[0]: https://digitalocean.uservoice.com/forums/136585-digital-oce...

If DO users don't need the ability to upload their own images, why is this a problem?

Some DO users will want this, others who aren't DO customers will also want this. As a growing company DO want to attract as many new customers as possible.

They're still in early stages though, I'm sure this is in the pipeline.

>They're still in early stages though, I'm sure this is in the pipeline.

I would not call Digital Ocean all that early. This has been an issue for years.

Why would a growing company want to attract as many new customers as possible? This seems to be a fast lane to default. Growing company wants as many profitable customers which can be served as possible. Or should growing companies try to offer everything for all?

Growing companies generally want to attract as many new customers as feasible because it generates buzz, and gives them a shot at becoming the go-to place for their kind of service.

Not if a large subset of those customers are not profitable, was the point.

Read the link. There are over a thousand votes and six pages of comments of people saying "this is a problem for me".

The one that's for custom images, not just kernels, is actually even higher up at over 3k votes and second highest on the list currently: https://digitalocean.uservoice.com/forums/136585-digitalocea...

Regarding custom kernels, DO says that BSD will be able to boot them: http://disq.us/8lqxh7.

I assume you also complain about the availability of microwave meals? That'll only become a problem when you can't buy raw fruits and vegetables anymore, but I didn't see that happen. Likewise, you mention more customizable VPS options in your post.

There's a market for everything. You don't understand my use case. My use case is "I want to click a button and then I want to be able to `apt-get install what-i-want` and then it should work. I don't even care whether it's Debian or Ubuntu, as long as it has apt-get because that's all I understand.

Granted, maybe I shouldn't be running VPSes at all but hey, it works, and I bet DO has many customers like me.

Part of his point is DO will ignore apt-get where kernels are involved. Sure, it'll look like you're running the latest, but unless you undertake additional steps, it'll be booting their kernel, not yours.

So? My whole point is that my Rails app will run on every kernel DO will ever care to support.

If you want to do such low-level things as upgrade kernels, then maybe DO's one-click-and-poof-you're-running is less important to you than some other features and DO isn't the best option for you.

Also, it's $5/mo. I'm sold (and have been for a year now). No complaints.

> I assume you also complain about the availability of microwave meals?

DO is well know, judging by the fact that I know of it and I don't know a lot about VPS business. The fact that you and many of their customers want easy to configure VPS doesn't preclude them from offering a more configurable option for other users where you are able to install whatever kernel you want. Use more imagination and less microwaves.

You can select the kernel your droplet uses via their admin interface. Just remember to update the VM first.

> Before, when a company provided Xen or Kvm, you generally would get to have low-level access such as the ability to virtually connect to a serial port or vnc session of your box as it booted. You also, typically, could provide your own ISO images.

It's remarkable that you start your sentence with "before". As a prgmr.com user, I can still get an out-of-the-band console and run my own kernel without any fanfare. (no affiliation at all, just a happy user)

This is the reason why I no longer have VPSes at DigitalOcean, sometimes I couldn't update my CentOS kernels when important updates were available because DO lagged for a few days in making them available.

Ubuntu DO instances have a modified apt sources.list file that points all apt requests to a DO-run repository. If you revert to the default repositories, you should be able to get updates as they are released.

Not sure if the same applies to CentOS.

There are plenty of companies that will let you upload your own images. It's not very convenient. Provider's images are somewhat easier to use.

The best solution is to have maintainers of the OS prepare cloud distributions. Many already do for AWS or OpenStack. Our own 'cperciva is responsible for EC2-compatible version of FreeBSD.

Until this becomes a standard, there's nothing wrong with partial solutions. I created a FreeBSD droplet right away.

Allow me to again express to @cperciva how appreciative I am of the FreeBSD EC2 image(s). We now run our entire stack on FreeBSD in EC2, and it's a joy.

Sounds like they aren't using kvm or xen (even though most providers do not allow for image uploading). OpenVZ is even more frustrating when your working with ipv6 (UGH!).

With FreeBSD, bootloader and kernel are stored on the root filesystem. So you can change them as you desire. Only repartitioning is painful a bit.

Just deployed a FreeBSD droplet and I'm not sure if it's just because the host network is busier than my other droplets, but I seem to be getting about half the network performance that I can in a default linux droplet. They are using Virtio, which is good since it doesn't require hardware emulation like the E1XXX devices on KVM. I should probably use a better test than cachefly but just wondering if theres any known tweaks/tips that should be done for FBSD on KVM with virtio devices.

Disk performance is also lacking in comparison to the ubuntu droplet as shown in the pastebin. Could just be because everyone's spinning up fbsd boxes on this host? :)

http://pastebin.com/raw.php?i=E8Q06XgM

I am also seeing really poor write speeds when comparing to my Linux droplets. 11MB/sec vs. 216MB/sec on Linux

Are these the same datacenter? The same rack? I suspect that degraded instances and overly-busy disks are not endemic to AWS.

Mine are both in NYC3, so same datacenter. No clue regarding rack/host. I can't comment on AWS...

With FreeBSD, in order to be able to have automatic backups work, we were required to disable journaling on the disk. This will cause the disk speed to be much slower than our Linux distributions.

If you wish to have higher disk speeds, but not use backups, we recommend for you to remount your disk with journaling. https://www.freebsd.org/doc/en/articles/gjournal-desktop/con...

Why did journaling interfere with automatic backups?

journaling generally interferes with dump(8). It seems odd that they would be using dump to do backups though, since they are virtualizing the OS.

The parent link to gjournal is weird too, since gjournal is not the same as softupdates w/journaling (SU+J). It makes me wonder if they actually disabled softupdates too.

I found a random post[1] about issues (unmapped io?) running i386 freebsd with su+j in virtualbox (apparently a virtualbox bug[2]).

[1]: https://forums.freebsd.org/threads/freebsd-10-i386-data-corr...

[2]: https://lists.freebsd.org/pipermail/freebsd-current/2013-Nov...

DO disabling journaling is a big deal.

What else has DO disabled and/or modified from a vanilla install?

DO modifies a lot of stuff on vanillas installs. They're one of the only providers I know of that removes the swap partition.

Where does DO document all of the modifications they make to a vanilla FreeBSD install?

That deserves an explanation - you can backup non-ext2 linux partitions, surely? What have I misunderstood?

This should be done by default on droplets which do not have backups enabled.

> required to disable journaling on the disk

Shouldn't this make it much faster as the FS no longer maintains a journal?

So I enabled Journaling and that brought it up to 30MB/sec. Still a substantial difference vs 216MB/sec for Linux.. Something with the D.O. KVM HOST setup needs to be tweaked a bit - 30MB/sec is pretty poor quality disk throughput for SSD attached storage.

There is no way to boot into single user mode for now (or at least instructions provided aren't working).

And the reason was invalid /boot/loader.conf:

console="vidconsole,comconsole" autoboot_delay="10" console="comconsole,vidconsole" autoboot_delay="1"

Second console actually disables access from web console access, which is fatal.

Autoboot is not critical, although default to 3 sec should work much better for opportunity to change boot options from web console.

This got apparently fixed, in my "ams3" instance I've had to issues getting web console to work (needed to "tunefs -n enable /")

I just use Vultr and install from a FreeBSD downloaded ISO. No modifications to the base install at all and works as expected.

DO have screwed something up with their configuration if they need to make so many changes to make it work. No other VPS provider I have used that support FreeBSD (or OpenBSD for that matter) require any changes to the default install.

On UFS soft-updates have a lower overhead than soft-updates with metadata journaling. The downside of soft-updates without journaling is that it requires a background fsck to recover leaked space after unclean shutdowns. I prefer UFS2 SU (not SU+J) on FreeBSD for small systems, because SU+J is incompatible with UFS snapshots.

Yes, it's really sluggish. Also both power off and resize do not work for me - as I wanted to upgrade it to compare the performance.

Could you open up a support ticket so our support team could help you out? Both of those should work without issue.

It's not only DO. I had to create a Linux VPS in order to run a Sinatra application because when deployed on FreeBSD it took more than 60 seconds to send a response to the remote API and the connection was timed out!

After performing some tests[2] I figure out that the problem was not FreeBSD per se, but the FreeBSD deployment on the specific virtual server... I think that *BSDs should be avoided because they tend to be a lot slower than linux deployments on virtual machines.

[1] http://www.transip.eu

[2] https://gist.github.com/atmosx/14efea27eb2c1e38af09/

something must have been horribly, horribly broken on your freebsd installation or hosting environment, because there's no way any sinatra service takes 60 seconds to respond. And, your gist doesn't show a significant performance difference between linux and freebsd.

> I think that *BSDs should be avoided because they tend to be a lot slower than linux deployments on virtual machines.

Many virtualisation providers don't support it properly, but "should be avoided because my suppliers are stupid" is a terrible plan.

FreeBSD often has better performance than Linux when both are properly hosted. I think the lesson is more like "when a provider is new to a particular system, there will be problems; for production systems use a provider that's experienced at supporting the kind of system you want to use".

I disagree with this. FreeBSD is running fine and speedy at other providers in a virtualized platform.

FreeBSD is plenty fast. I expect Digital Ocean simply needs to work out some kinks. It will be fast soon enough.

> time dd if=/dev/zero of=/tmp/test bs=64k count=16k

16384+0 records in

16384+0 records out

1073741824 bytes transferred in 57.605991 secs (18639412 bytes/sec)

0.023u 6.128s 0:57.61 10.6% 25+172k 7+81916io 3pf+0w

> sudo mount -o nosync -u /

> mount

/dev/gpt/rootfs on / (ufs, local, soft-updates)

devfs on /dev (devfs, local, multilabel)

> time dd if=/dev/zero of=/tmp/test bs=64k count=16k

16384+0 records in

16384+0 records out

1073741824 bytes transferred in 5.135908 secs (209065631 bytes/sec)

0.016u 2.274s 0:05.16 44.1% 24+169k 8+8193io 0pf+0w

Smart move. I will definitely be spinning up some FreeBSD droplets. This will attract people like me who enjoy building lean and mean BSD servers, and give people an alternative to Linux if they choose.

Nice work Digital Ocean, love the way you folks keep pushing forward. Need some tutorials written?

We've already put together a lot of basic documentation in-house, [0] but we'd love to expand on what we have in the library. Checkout our "get paid to write" program. [1]

[0] https://www.digitalocean.com/community/tags/freebsd?primary_... [1] https://www.digitalocean.com/community/get-paid-to-write

If you take requests?

After reading the tut on HN the day before on how to be your own vpn provider with openbsd [1] I started to search for a tutorial that was either openbsd or freebsd with softether without much luck. I was about to do an instance of debian & softether.

Perhaps my comment would be better served in another way. I'm new at this and have no idea what I'm doing. :) How can I go about from setting a vpn server with a webpage for paying customers?

I'm looking at it more like a learning experience than to make it into a business, but if it works great. Could you or someone point me into the right direction into what needs to be read for each step of the way? I have very little linux experience, non in bsd and a little in python.

Thanks in advance.

[1] http://networkfilter.blogspot.com/

As a web developer who knows enough Linux to do minimum dev-ops, could anyone recommend some things worth playing around with in FreeBSD? Like "do this and see how easy it is vs Ubuntu!". Or are the gains more long term like better stability?

Dtrace alone will turn you from a developer to a Developer + Systems admin + practical OS engineer, you will understand how your stack is performing within various lower levels of the operating system and be able to tune the hell out of your stack from bottom up.

Come on, 0.1% of the people need Dtrace...

But that doesn't mean it's only useful to 0.1% of people.

If it's more widely available, more developers will reach for it.

The more comfortable people are with tools like DTrace, the better they will be able to write and use software.

Everyone needs to know all of the things. Just because it's a pain to get now doesn't mean people wouldn't use it if it were there. Remember when monitor mode meant patching drivers and running a weirdo kernel? It's like that.

Two things here:

- If you use Dtrace to troubleshoot a problem it's way beyond your scope of knowledge as a web developer. Dtrace is too deep and very specific to certain problems that you will never see because your regular nginx + php works fine in all cases.

- Not everyone wants and should be good at everything.

Just be good at what your doing well, I'm not saying that one shouldn't be curious but Dtrace isn't "an excuse" to install *BSD as a web dev.

If you don't use DTrace to debug php, you're missing a lot: http://php.net/manual/en/features.dtrace.dtrace.php. Same for python and ruby and node.js. All these have DTrace support.

DTrace is not hard, it's essential; and it's significantly easier, than say, awk, which is another crucial tool.

Under OSX it seems that DTrace support is incomplete thanks to OSX' dtrace implementation [1]. [1] https://github.com/joyent/node/issues/3617

I think only Solaris has full support?

Its interesting to some of us: http://www.postgresql.org/docs/9.4/static/dynamic-trace.html

0.1% of people in the world? That seems a little high. 0.1% of people who describe themselves as *-dev? That seems really low. Also, how do you define "need"? Dtrace isn't just for tracing functions down to the kernel level, and I can imagine all sorts of things that a web dev would find useful in dtrace. Just google "ruby dtrace".

If you do any kind of programming or system administration, you need DTrace (or some lame Linux equivalent). It really pains me to see that people reject it because it's "too advanced" for them, or such nonsense. Dynamic tracing is the one technique that will enrich your life as an IT professional more than absolutely anything else ever did.

And DTrace in particular is really, really easy to use (the Linux alternatives, not so much).

Yeah that said, Dtrace on FreeBSD sucks. The DSL is hard and many ready-to-run scripts don't run at all.

Other than Dtrace... Tools like dtruss & co are not as clean as strace, that's the feeling I've got when I had to trace some calls.

I switched from CentOS to FreeBSD as my daily driver a little over a year ago, for the ports tree. I needed a bleeding edge version of valgrind, but my ~/bin and ~/lib were already pretty unwieldy - so that is what caused the switch. That and ZFS. But I've found a couple of other things that I really like: the documentation is awesome, and config files are where you'd expect them to be. Being able to tune system internals online with sysctl is really awesome as well. Wanna change the lowest possible C-state one cpu3? sysctl hw.acpi.cpu.3.cx_lowest=C3.

I dunno how useful that is for web devs, but as a C programmer and perpetual tinkerer - FreeBSD suits my needs very nicely.

Linux has sysctl as well, and other parameters can be set by writing to the right files in /sys. For example, the maximum c-state can be set by writing to /sys/module/processor/parameters/max_cstate (if you have support enabled in the kernel).

The linux sysctl is a wrapper for kernel state represented in the file system, freebsd sysctl is direct to kernel. So that is nice, everything in one place. Before my switch to freebsd, I had been a linux user since 2000 - and while I was aware of the existence of sysctl, I don't remember seeing it used very often. It was much more common to see direct interaction with stuff in /proc (rockectraid, never again). Why is that? Is it an issue with standardizing driver interaction or what?

My prior post wasn't written as a statement of exclusivity, simply that I prefer the way freebsd does it.

The /proc/sys/ and sysctl(2) methods are really just down to which syscall is used to talk to the kernel - either way it's just reading or writing a kernel-side data structure, there's no fundamental difference there.

The /proc/sys/ pseudo-filesystem interface has the advantage that it's discoverable. The sysctl(2) interface requires userspace to have knowledge of a swag of hardcoded ID numbers identifying each node in the sysctl tree.

I haven't poked through the code, but the linux man page isn't encouraging: "use the /proc/sys interface instead" [0]

Compare that to the freebsd man page [1]

[0] http://man7.org/linux/man-pages/man2/sysctl.2.html#NOTES

[1] https://www.freebsd.org/cgi/man.cgi?query=sysctl&sektion=3&m...

I think perhaps we might be talking at cross-purposes here.

My point is that when you call write(2) under Linux on a file descriptor derived from opening a /proc/sys pseudo-file you are talking directly to the kernel code responsible for updating the sysctl variables in the same way as you are when you call sysctl(2) under FreeBSD.

The /proc/sys files aren't real files - they're just a way of representing the sysctl variables in the existing file namespace.

Ah, when you said sysctl(2) I thought you were talking about the linux call - which is basically dead code. Yeah I don't really have a beef with the whole "everything is a file" design, I actually quite like it, but I really like how well freebsd has done sysctl - both the subroutine (3) and the command (8).

You probably mean "knows enough Linux to do minimum ops".

My reasons for using FreeBSD are a little more philosophical:

- I want to have a stable base O/S to which I can always easily return.

- I want to be able to customize installed packages in an easily scalable way.

- I want a server O/S to be simple to maintain, relative to Windows or Solaris.

- I want the goddamned documentation installed.

During development it's difficult to get RHEL or Ubuntu back to a known-good set of base packages without fiddling a lot with the package manager. It's better nowadays with package groups and autoremoval supported in both yum and APT, but with FreeBSD, you can always punt, do "pkg delete -a && rm -rf /usr/local" (or "pkg_delete" in the before time), and start over. The base configuration is also pretty simple and centralized, with most of what you need in /etc/rc.conf or /etc/periodic.conf.

pkgng + poudriere + a suitable web server makes custom package management really, really easy. I haven't tackled Spacewalk or similar tools for Linux, but even building locally customized versions of packages on RHEL or Ubuntu is a moderately complicated process compared to the FreeBSD Ports Tree. On RHEL or Ubuntu, you typically have to install the developer tools, hunt down the source RPMs/DEBs, edit the package definition, run rpmbuild/debuild, and install the resulting RPM. Compare that to the FreeBSD Ports Tree, where you run one command to download/update your copy of the package definitions, add whatever package-specific knobs you need to /etc/make.conf, and run "cd /usr/ports/category/packagename && make install" (the compilers and everything come built into the base system).

I'm not going to start an argument about the relative merits of init systems, as systemd (Linux), SMF (Solaris), and SCM (Windows) all have their merits, but I personally like the simplicity of configuring everything through /etc/rc.conf on FreeBSD. It's definitely old school, but then I cut my teeth on NeXTSTEP, SunOS 4, and Slackware Linux, so rc-style init scripts feel pretty natural to me.

As for documentation, I cannot tell you how many times I've wanted to run "man something" only to find out I need to install the -doc package. (Also, I cannot tell you how many times I've wanted to compile something, only to find out I need to install the -dev package.) Compared to Linux, FreeBSD has superior documentation. Even kernel bits get manual pages, and not just syscalls in section 2, but kernel interfaces and modules in section 4.

Of course I use both FreeBSD and Linux to great effect at home and at work, as well as Windows and Solaris. I just _like_ FreeBSD better.

> I cut my teeth on NeXTSTEP, SunOS 4, and Slackware Linux

That's the real reason. Everything else is rationalization added later.

(Don't get me wrong, I've also started on Slackware. But sometimes you need to break old habits to learn something new.)

Well, I won't pretend to be unbiased, but don't worry - I'm on good terms with a wide variety of operating systems and O/S distributions. ;)

by this logic, everyone should move all software to node.js, and any complaints are rationalization, because it's newer than C.

> On RHEL or Ubuntu, you typically have to install the developer tools, hunt down the source RPMs/DEBs...

Nitpick, but on Debian/Ubuntu this is a lot more simple than you make it sound:

    apt-get source xxx     # download the source package for 'xxx'
    apt-get build-dep xxx  # install everything needed to build it

I didn't know about that! Thanks! I'd be surprised if yum didn't support something similar, but whenever I've had to build custom RPMs, it's been the old manual srpm download way. That said, on top of my to-do list is getting Spacewalk up and running, so that I can maintain my own package repository for RHEL/CentOS/Amazon Linux similar to what I do with Poudriere for FreeBSD.

I asked a related question from the database developer perspective just yesterday: https://news.ycombinator.com/item?id=8884125

When bored, you can run `make world` (it recompiles the kernel and every binary on the system).

Jails are awesome

IMO, just having something of the quality of FreeBSD's handbook (https://www.freebsd.org/doc/handbook/) is a significant point to consider. It is the best piece of documentation for a system I have ever read.

Yes. From my experience:

* PF (default on OpenBSD, a fork exists on FreeBSD) configuration is way more human-readable than iptables. Makes a lot easier to create custom complex rulesets.

* Documentation is much cleaner on FreeBSD (or OpenBSD) compared to GNU/Linux. Again helps you deploy complex solutions easily.

* The upgrade process (using ports or pkg) is well documented, easy to execute[1].

* ZFS makes FreeBSD a very solid file server

So, other than specific software, a clean approach on how start/stop services, where goes what, etc. I don't see any other reason for someone to switch from Linux to BSD.

However, given my experience ruby (I'm a ruby programmer) under-performs on FreeBSD VPSs compared to Linux VPSs while on bare metal doesn't. There are reports citing NetBSD as fastest ruby bare-metal OS. But again, differences shouldn't be all that much between BSD and Linux deployments in bare metal to justify a switch on VPSs though, if deploy ruby apps, I'd say stick with Linux.

[1] Hm. It's easy to execute if you are not afraid to read some extra documentation. But once you get the hand of it, it's really a breeze, never had serious issues with FreeBSD in ~3 years.

+ Dtrace

+ Jails

+ Capsicum [1]

+ Netmap [2]

+ Most performing network stack

+ Resource Management (pretty low memory usage)

+ The userspace tools come with the source (no GNU/Linux duality)

+ Clang/LLVM as default compiler stack

[1] - https://www.freebsd.org/cgi/man.cgi?query=capsicum&sektion=4

[2] - https://www.freebsd.org/cgi/man.cgi?query=netmap&sektion=4

I never understood the main differences between jails and chroots. Would you be willing to explain?

chroot (often referred to as a "chroot jail") limits a process to a certain subset of the filesystem - e.g. you could limit a httpd process so that it can only see /var/www, and it would not be able to see anything outside that, so if there was a security compromise of the web server, an attacker would not be able see anything outside that folder tree.

A FreeBSD jail is a like a lightweight virtual machine, and is very similar to a Docker container in Linux (though it has been around for about a decade longer than Docker). It provides isolation for processes etc., but uses less resources than a full virtual machine. It is limited in that it has to be the same operating system as the host.

Chroots only lock you to a particular part of the filesystem. Jails add process and network resource restrictions.

FreeBSD jails are like a really mature, full-featured version of LXC as opposed to "just a chroot". In addition to being chroot that provides real filesystem isolation without many of the security issues of a Linux chroot, it also has CPU and memory limits, disk quotas, network isolation, root privilege isolation, all the magical ZFS goodness (provided you're running the jail on ZFS). It's really, really nice.

This is a pretty good overview: https://en.wikipedia.org/wiki/Operating-system-level_virtual...

Running ps in a jail will show the processes in the jail only.

Processes in the jail will only see network interfaces and other devices that have been explicitly exposed to the jail.

As others have said, it's like a vm with no virtualization overhead. You can set up jails with the entire Freebsd fs hierarchy so it runs like another host with its own users. Note that even the root user in a jail is not the same as the real root user. You can then use pkg to install packages within the jail too.

The best thing about FBSD is that it mostly works the same way it did a decade ago. It's a lean, mature, core that isn't subject to flavor-of-the-month changes to core systems.

the PF available on FreeBSD is woefully out of date. If you use FreeBSD regularly and don't mess with pf, learn ipfw. it is quite powerful and much more performant.

I'm surprised this is the case, since pfsense is a very capable FW built upon the pf version on FreeBSD. I've been running it for years with nary a problem. Would you say the outdated version of pf puts people running pfsense at risk?

Everything he said is true and I know this for sure. But I like PF so much and even the stripped down version of FreeBSD feels awesomely good. You're in no danger what-so-ever. OpenBSD's version of PF has loads of advanced features and way better performance but you'll only notice these things if you need an (very?) advanced configuration.

As a developer you might find jails useful. I use them to create multiple isolated 'virtual machines' on the same machine. In each one I can install a different set of packages and I like the my base system in clean. With zfs, I also snapshot each jail before major changes so rollback is easy. Try this and see how easy, well-integrated it is as opposed to something similar on linux.

I use ezjail, btw.

While this is great news, BSD support is currently the second most widely requested enhancement to the Digital Ocean service.[1]

I wonder if we'll now see additional storage addressed soon?

[1] https://digitalocean.uservoice.com/forums/136585-digitalocea...

Edit: I've had this theme bookmarked for ages, now might be the time to build it! http://daemon-notes.com/articles/desktop/example

They have a bunch of stuff that was "planned" for Q1 2014 (separate hardware for master/slave setups) that aren't even close to shipping. Pretty frustrating. I like Digital Ocean and I use them in production for some apps but it's very hard to take their techops team seriously when they are missing deadlines by 12-18 months or more without regular updates. It's pretty unprofessional.

Yep. I was running Arch Linux on a personal droplet, and they decided to deprecate support for it. I don't even think there was an official announcement about it. There's even quite a bit of backing to bring it back [1], and oddly, they use an old video on their front page with when Arch was still listed [2].

"Built for developers" => can't use developer's favorite OS. :(

[1]: https://digitalocean.uservoice.com/forums/136585-digitalocea...

[2]: https://www.digitalocean.com/assets/video/create-e68d7b3c.we...

Because Arch is what % of the userbase and how much does it cost to support that userbase? It gets kind of old hearing people whine about this. If you really want Arch there are providers that support it.

I would expect FreeBSD to also be in the minority compared to Linux distros [1].

There are also other providers that support *BSD and people still requested it on DO. Why can't I ask for Arch?

[1]: http://w3techs.com/technologies/details/os-unix/all/all

Hey there, Sam here from the DO Product team.

You're right, we dropped the ball on UserVoice. There aren't any excuses to be made. I know it's a lot to ask, but please trust that going forward we will be much more transparent through UserVoice & elsewhere. We'll also be much more intentional in communicating our priorities through that medium, as this seems to be at the root of the concern about missing deadlines.

Again, no excuses, we're sorry we let you down, and we're going to get it right going forward.

Cheers!

You are what you do.

I'm a DO customer, and while you've never let me down, I'll be paying attention now that you've given your word.

Challenge accepted :).

Do you have a timetable on when tickets and announcements will be set up? There are outstanding issues with hundreds/thousands of votes with zero updates, and promises made 12 months ago with no progress. I've heard DO reps in the past say they were going to update things but lack of follow-up is always what happened.

I think I speak for a lot of DO users who have to rely on AWS/Linode for larger clients that we'd move more infrastructure over to larger plans if we could just get better ops timelines. Your support for servers has been nothing short of great in my experience, but no timelines and no communication on stuff as important as retaining IPs and uploading custom ISOs (both available on cheap $5 VPS providers that are nobodies on LowEndBox) is really frustrating.

Unsurprisingly, there's no answer to this ticket. Sigh.

> ..retaining IPs..

I noticed since few month back, if you destroy your droplet and create fresh droplet even after few days, you will get the same IP. Only in same DC though.

Maybe some sort of temporary policy while they're working on proper IP retainer.

The biggest mistake we've made previously was promising dates where none really existed. We don't have concrete dates for you. When we do, and _only_ when we do, we'll update those UserVoice items. Bear with us, it won't be long before I have more for you.

that's what you (pl.) said last year.

https://news.ycombinator.com/item?id=7723751

The You (pl.) has been expanded, changed, and we're working with better processes (and more people!) to make sure that doesn't happen anymore. Trust is a lot to ask for, but do stay tuned. There will be a lot of pleasant surprises this year.

I want that too. Now I'm using Backupsy[1] when I need a VM with more disk space.

[1] https://backupsy.com/aff.php?aff=367

It's the highest voted request and there was no response for over 2 years. The first response after that time didn't even address the request but tried to distract with an unrelated feature request. This is pretty frustrating and the reason I will move my VPS from DO to another hoster. If you think I'm bitching around that's because I am. The way DO handles this is ridiculous.

Hopefully it stays around and isn't axed like Arch was :)

I am very intrigued by BSD as it comes highly recommended here. I just need an excuse to dip my toes.

I need to set up a nginx -> nodejs server for a project soon. Given I have set up a number of linux servers without trouble, how much of a struggle would it be to just use BSD for this new project? Would it be worth holding off and just messing about in a VM, or would my linux experience just transfer directly to setting up on FreeBSD?

If you haven't had trouble with Linux, I'd say you could manage in FreeBSD.

We've prepared tutorials that can help you get started with the basics: https://www.digitalocean.com/community/tags/freebsd

I just started running the same thing a couple of months ago without issue. Go for it. It works great.

It shouldn't be a struggle at all just be conscious that freeBSD does not try and protect the user from him/her self. Case in point, "kill 1" won't do anything on linux but in freeBSD it will kill the init process.

You're still using the 2005 sysadminning model of instances/hosts running services. Use elastic beanstalk or similar to pop up a layer of abstraction to "app". Your time is finite.

> or similar

Is this alternative method another SaaS with a different API and performance characteristics? That takes time as well, in addition to the vendor lock-in. BTW, stacking abstraction x infinity is what causes systems to be bloated, unreliable security risks. Time spent gaining a greater understand of components on layers below the level of the stack you're operating on is time well spent, you'll be a better developer for it.

That, or do everything with VMs and guests, like they did in the 1960s and 1970s assuming you were using IBM mainframes.

There does seem to be one part of their announcement that's a bit off:

  While similar to other open source unix-like operating systems, it’s unique in that the development of both its kernel and user space utilities are managed by the same core team, ensuring consistent development standards across the project.

Wouldn't it be Linux that would be unique in that they don't do this? Solaris, AIX, HP-UX, all the BSDs, Mac OS X (which is certified Unix) does this as well. Correct me if I'm wrong here.

Maybe they talk about Linux, Hurd, OpenBSD and NetBSD?

So two things:

None of the OS's you listed are open source.

The second part of the statement, the way I read it, is comparing FreeBSD to DigitalOcean's current offerings - Linux. Keep in mind the post is for someone that's already using DO's services, which is a person who has only ever used Linux on their platform.

Good point, I glossed over that detail, but they are comparing to only open source OS's, so my examples are bad. Still, looking at http://en.wikipedia.org/wiki/Comparison_of_open-source_opera... I think Linux is the odd man out here. Anyway, I suppose most of them aren't popular enough to be considered by Digital Ocean or their customers, so you're right there.

AFAIK, AIX, HP-UX, and Mac OS X are not open source (I'm only positive about OS X)

OSX is largely based on DarwinOS + some lipstick and customization... so it's close-ish to being Open Source... but I see the point.

OS X is mostly open source, the big thing that isn't is XQuartz IIRC.

They may be treating the various linux distributions as separate entities. Then the number of Linux based operating systems should far outpace just about anything else ....

For anyone else having trouble reading the quote:

> While similar to other open source unix-like operating systems, it’s unique in that the development of both its kernel and user space utilities are managed by the same core team, ensuring consistent development standards across the project.

In case someone missed it: The header graphic in this article is a great homage to Beastie, the (original?) FreeBSD mascot, analogous to the Tux Linux penguin: http://en.wikipedia.org/wiki/BSD_Daemon

Great news! I'm personally looking forward to OpenBSD, but now that this is done I bet that will be a cinch.

You might want to check out vultr.com. They don't directly support OpenBSD, but allow you to install an OS an a VM instance with an ISO image, either by you supplying it or they'll pull it for you through an ftp or http link. Their pricing is similar to DO.

I managed to get an instance running without too much trouble.

(I do not work for vultr, or affiliated in any way).

I use vultr too and find them pretty good if very slightly pricey

Is there anything like http://serverbear.com/compare/vps that lets you browse OS's offered by various providers?

Also wondering how reliable Vultr's been for you guys.

And yet, still no universal support for IPv6, and the droplets that do get it only get 16 addresses. Yes, I am going to complain every time DO comes up in the news until this is fixed.

Pardon my naivety, but why do you need more than 16 addresses per droplet?

Here is a good explanation: http://etherealmind.com/allocating-64-wasteful-ipv6-not/.

You still get a /64 at DO, though. So this doesn't hold true.

EDIT: Should add that you do not get a complete /64, but 16 addresses from a /64. Probably in a private VLAN or something.

That is an artificial limitation. I got an allocation as a ::c001/64 and can use up to 15 more addresses, but I strongly suspect that I share that /64 with many others. I continue paying DO, but I believe they just plain don't get IPv6 and DNS want to spend the time learning. It's a shame as they are likely to be affected by the IPv6 shortage much more than the larger/more expensive providers and it would be in their interest to have more of te Internet upgrade.

Whoever wrote this doesn't seem to quite understand how bit math works.

"IPv6 addresses are 128 bits long, compared to 32 bits long for IPv4. In other words, IPv6 addresses are 296 times more numerous than IPv4 addresses."

IPv6 addresses are actually 2^128/2^32 or 7.9e+28 times more numerous than IPv4, which would strengthen the argument that it's hard to be "wasteful" with them in the way described.

I get the impression that 296 is meant to be 2^96, which is exactly the same as your figure.

That article completely ignores NDP exhaustion attacks. As autoconfiguration has no place in a server environment anyway, which is the protocol that breaks, there's absolutely no need to assign a /64. The more important goal is to avoid fragmentation; it makes more sense to allocate a single larger subnet than multiple smaller ones so as not to fill up routing tables. Externally to your network, you'd aggregate anyway, but it helps to keep routing tables lean on layer 3 switches which aren't capable of as many routes. Also, some vendors' IPv6 implementations have FIBs that use /64's, and may need buckets for multiple subnets within, which can be inefficient. A good compromise is to reserve a /64, but only assign something more reasonable like a /121, with 123-125 usable IPv6 addresses being more than enough in most cases.

One small step for man, one giant leap toward a PFSense VM in the DigitalOcean cloud.

what will you do with a pfSense VM on DO?

You could connect all of your VMs to it via ipsec or openvpn and have your own little private network.

Unfortunately, you can't. IPSEC isn't enabled by default in the FreeBSD kernel.

You can build a customized kernel.

Yes, but you can use custom kernels with FreeBSD on DO. In the comments:

"FreeBSD droplets do allow you to customize your kernel. Unlike other droplets these boot from the kernel within their filesystem. This is the reason that FreeBSD is not available in NYC1, NYC2, and AMS1 as these regions do not yet support this option."

EXCELLENT!

Thank you very very much for supporting FreeBSD!

Agreed, this is really great news, I'm trying it out now.

Finally, finally, finally! I've been waiting for either DO or Linode to offer this since forever. Now the only thing left on my wishlist is OpenBSD support.

Linode doesn't (last i checked anyway) officially support FreeBSD but people were doing it years ago in the irc channel when I used to hang out there

Last time I looked, Rackspace (at least the Sydney, Australia centre), supports FreeBSD.

As one of the authors of bsd-cloudinit, it's super cool to see the project being used by other people.

http://pellaeon.github.io/bsd-cloudinit/

I'm wondering what the chances of any other BSD being supported are...? Dragonfly? Open? ...Net? :)

edit: after actually reading TFA, it seems unlikely. Well, it seems like Dragonfly is most likely, if any others.

Depending on demand, adding other BSD variants is certainly a possibility. We had to start somewhere, and the FreeBSD community has been very vocal about wanting to see this happen. This is the first non-Linux OS we've decided to support, so we're excited to get feedback on it.

Thanks! I'm a long time FreeBSD vps user. It'd be great to check if DO droplets can support CARP failover within the same datacentre, and then expand to be able to do this across datacentres.

https://www.freebsd.org/doc/handbook/carp.html

My use cases so far haven't involved CARP but I'd like to start experimenting with that!

Any recommendations of other BSD VPS providers?

RootBSD [1] has been around for quite a while. I personally use fileMEDIA [2] and LunaNode. fileMEDIA provides a set of isos, among which you can find FreeBSD, OpenBSD and DragonflyBSD (i personally requested them the DFBSD one). On LunaNode you can upload any ISO image or qcow2 file you'd like.

1: http://www.rootbsd.net 2: http://www.filemedia.de 3: http://www.lunanode.com

I run FreeBSD on Greenqloud.

RootBSD is insanely awesome and is the one I'd recommend whole heartedly.

Compile your own kernels, your own world, and they are always on hand for assistance for things that you need help with.

I've been a happy customer for a number of years.

I think FreeBSD was a great choice! I currently have an Ubuntu droplet, but it feels great to know that I can switch to FreeBSD if I like.

Just created a droplet and sadly, it is 10.1 amd64 only. Won't be very useful on low-memory VMs. I hope they add i386 too.

EDIT: Anyone cares to explain downvotes?

What's your idea of a low-memory virtual machine? For test purposes I'm running FreeBSD/amd64 under Hyper-V in 128-MB RAM without any problems, although it is using around 32-MB of encrypted swap. That includes the Salt minion, Postfix, and an untuned static Apache 2.4 installation. Of course, it's much more comfortable in 256-MB RAM with around 44-MB RAM free according to top, and of course that's workload-dependent (e.g., my mail relay running amavisd-new and ClamAV wants 1.5-GB RAM after loading all of the spam and virus signatures). I could definitely see wanting to run FreeBSD in 128-MB or less RAM, but I'm very curious about your specific workloads. (It's the gearhead equivalent of wanting to look under the other guy's hood. If you're doing something cool, I want to hear about it!)

P.S. Hyper-V will let me go as low as 32-MB RAM, so thanks to you I'm keen to try out different operating system installs (and workloads) in low-memory environments.

P.P.S. Upvoted parent - I think the parent comment contributes to the discussion, even though I would personally love to see commenter go into more detail.

I used to run a typical apache/mysql/php/perl stack on amd64 image on 512 MB VM. Actually, I run many things in very tight memory environments on FreeBSD, there is always significantly more room on i386 (in comparison to amd64).

Do they support IPv6? The IPv6 link from their list of features page just links to their blog entry about Singapore.

What kind of IPv6 allocation do they provide?

Yes, in most regions. The allocation is an incredibly stingy 16 addresses.

I've been using Vultr.com for this for a while and they're pretty nice. Slightly cheaper, promises that they don't oversell their servers, and they've had FreeBSD for long enough to have got the kinks out.

They also let you just upload an ISO and install any OS you like from there, which is handy for non-default FreeBSD configurations like ZFS-on-root

BOOM - time for me to spin up more of these!

This is great news, thanks DO!

A service that is a bit ugly, here is what I feel about, you register, you give your credit card, and you just don't know how it will cost. That point is just bad and make me feel that will cost an eye.

Don't know how much it will cost?

The price you see in the huge font is the price you'll pay at the end of the month...

Nice, but I wish they didn't deprecate Archlinux though :/

Couldn't you do a bootstrapped install if you really wanted it?

https://wiki.archlinux.org/index.php/Install_from_existing_L...

Someone wrote a script to do just that: https://github.com/gh2o/digitalocean-debian-to-arch

Tried it, works really well.

Well, it's more of a pain to setup right. It doesn't take 10 seconds anymore.

Then like all cloud services, things get bad when you can't properly boot or start the network services. I used to run Arch with a more recent kernel and that kind of happened after an upgrade.

To be fair, I can understand why they don't want to support Arch as it's probably a bit of work, but they are also a cheap service, so some people (like me) like to have a little server running there just for personal stuff (IRC bouncer, ssh tunnel etc.). And in this case, you want something closer to your own system.

They still provide a good service in my opinion as I still use them :)

I wonder how much does DO's offer differ from other vendors that allow you to basically boot from your own virtual CD, like AWS or Ramnode.

I'd love if someone explained it.

I have so much experience with Linux I feel like FreeBSD I would have so much to re-learn. What makes it worthwhile and how transferable is my knowledge?

Really wish they'd support IPv6 in all of their datacenters. Comcast and T-Mobile universally support it, why don't datacenters?

You guys are awesome! Been waiting.

Time for a benchmark.

Migrating away from DO. because if the host dies, your vm dies.

Same as ec2 yes, but aws provides ebs.

Why not just build your infrastructure correctly? Then it doesn't matter if the host dies.

It's kind of ironic that they list FreeBSD's excellent documentation as one of the reasons for consideration, especially considering that their own documentation is so bad!

I mean, what kind of company links directly to blog entries, with incomplete and outdated information, all across their web-site?

Ain't nobody got time to read the blog comments and figure out what's the current status of stuff is.

And the above post is downvoted to -2 for which precise reasons?!

Does anyone really disagree that documentation at DO is total crap?!

If it wasn't total crap, why would their employees link (on social media) to the upstream www.freebsd.org instead of any kind of FAQ on their own website? https://news.ycombinator.com/item?id=8890383 Oh, right, because DigitalOcean's documentation (about their own features (and disabling of features from FreeBSD)) is absent and non-existent!

Should have gone with OpenBSD instead to be honest. Half the requests on your UserVoice are for OpenBSD. All the coolest stuff in FreeBSD comes from OpenBSD.

OpenBSD -- the world's simplest and most secure Unix-like OS. Creator of the world's most used SSH implementation OpenSSH, the world's most elegant firewall PF, the world's most elegant mail server OpenSMTPD, the OpenSSL rewrite LibreSSL, and the NTP rewrite OpenNTPD. OpenBSD -- the cleanest kernel, the cleanest userland, the cleanest configuration syntax and some of the world's best documentation.

FreeBSD, on the other hand, is becoming more of a testbed for experimental, some would even say unnecessary technologies: https://news.ycombinator.com/item?id=8546756. It's also having a hard time catching up to OpenBSD: http://itwire.com/business-it-news/open-source/62641-crypto-....

I was going to downvote your comment, but instead:

> All the coolest stuff in FreeBSD comes from OpenBSD

This is juvenile "I'd rather push a Ford than drive a Dodge" level commentary. It's not true, and isn't even interesting.

That any BSD is getting support is a good thing -- it opens the door to others following, and is good news.

> All the coolest stuff in FreeBSD comes from OpenBSD.

I disagree. Jails, ZFS, and DTrace did not come from OpenBSD.

For security probably. But security isn't the only reason that I choose an OS. OpenBSD's security comes at a cost. They are usually late to the party on non-security features. Many of the security features make OpenBSD much slower. Even for security software OpenBSD isn't as big a win as the devs make it out to be. Take for instance PF, OpenBSD developers will be quick to point out that the OpenBSD version is more up to date. But that doesn't tell the whole story, FreeBSD is using a fork which allows for multi-threaded execution which is a must most non-trivial deployment scenarios. Further more OpenBSD often takes to hard of a line on security enhancements with the belief that the kernel should be the line in the sand. Usually, one prefers multiple layers of security but OpenBSD says the kernel is often good enough. See OpenBSD's refusal to add a MAC framework for an example of this. Jails also don't exist for similar reasons, though they are useful for reasons other than security.

The source you have for the 'testbed' for new technologies makes the claim but barely has warrant for it. On the other hand, OpenBSD is much more liberal about breaking compatibility especially when it involves security. While I'm not going to excuse OpenSSL, NTP, or Sendmail they are all general robust software that has been in use for decades. Aside from LibreSSL the OpenBSD rewrites have been incompatible.

FreeBSD also offers a number of incredibly compelling features outside of what OpenBSD can, or will offer in the short to medium term. I'll just list them: virtualization with Bhyve, boot from zfs, a linux compatibility layer, a much more modern package manager, official java support, the ability to install binary blobs.

None of this is to say that OpenBSD isn't a great choice, but recognize there are reasons to choose both platforms and that one doesn't need to spread FUD to advocate for their favorite platform.

> See OpenBSD's refusal to add a MAC framework for an example of this. Jails also don't exist for similar reasons, though they are useful for reasons other than security.

I think you've incorrectly interpreted OpenBSD's intentions. OpenBSD doesn't support a MAC framework because they believe the best approach to security is correctness, rather than trying to achieve security by adding features which results in more complexity, making it more difficult to ensure correctness. A common mistake people make is thinking that OpenBSD's primary goal is security; their primary goal is correctness. This just happens to result in better security more often than not.

Note that the package system used by OpenBSD is explicitly borrowed from FreeBSD.

That's not true. OpenBSD's package system was rewritten by Marc Espie many years ago in perl, the utility names are inherited from FreeBSD, but are now otherwise unrelated.

Ah, I misquoted. On the page http://www.openbsd.org/faq/faq15.html, they point out that the ports tree concept was borrowed from FreeBSD.

Reply on news.ycombinator.com