Odroid-HC4 is a $65 2-bay network attached storage device
liliputing.com
liliputing.com
If you go to Amazon and, even with heavy discounts, none of the products have even the smallest UPS built in.
$300 is actually a great price for a NAS, because at that price you're looking for only an exclosure on the open retail market.
[1]: https://www.cnx-software.com/2020/10/20/odroid-hc4-low-cost-...
The network interface is a single gigabit port so this does not matter one bit.
Yes I build something using a standard x86_64 (celeron) setup. I heavily experimented with an all arm setup thought. I still find it pretty hard to balance a cheap build that is expandable. I recently lost an OSD and my setup had a consistently high load while recovering.
I don't really like the power consumption of my current setup and am thinking of moving to arm. (I use multiple small HDDs between 1tb and 10tb and have different pools with different redundancys. Am thinking of replacing it with a simple raid or something and 3-4 12tb HDDs with a simple redundancy. Could use minio as s3 replacement and btrfs as fs [I won't use zfs for different reasons] btrfs has snapshots, compression and deduplication so everything I want and need. Never did use rbd over network)
Edit: This is my home setup I'm talking about. At work we got a real setup.
You can even 3D print a nice enclosure, these days.
If you're DIYing, then DIY.
Trying to go lower power doesn't feel worth it, particularly when the hard drives drink so much power.
I feel like I'm spilling the beans sharing this, but ServeTheHome has has an excellent "TinyMiniMicro" series on ~1L size mini-PCs, which are used in businesses a lot. They are much beefier boxes than my little celeron, now often 6 core with way higher clocks. They come with 35W and 65W cpus, but even the couple generations old models still tend to idle at ~10W & otherwise sip power only as demanded[1].
If power is your concern, get rid of your small hard drives. Switch to ARM? Meh.
[1] https://www.servethehome.com/hp-elitedesk-800-g4-mini-tinymi...
At least it's SATA.
This is exactly what I've been looking for, a cheap Ethernet connected device that can saturate 1GbE. It's perfect for backups.
Skip the middleman and directly connect your hard drive[1]! I think you may have trouble saturating the hard drive's dual 25Gbe links, and a decent switch is gonna be pricey, but that backup should be going very VERY fast indeed when you're done.
Kioxia (nee Toshiba) is not the only one doing this, thankfully, cause I think it's interesting as heck. A lot of hard drives have a good amount of speedy Cortex R processors on them already, why use those to talk NVMe or SATA when you could have the hard drive talking ethernet directly?
[1] https://business.kioxia.com/en-us/news/2020/ssd-20200922-2.h...
Paying about $1000 for 20TB of fault tolerant extra long term storage isn't a problem. I'm even starting to consider magnetic tapes for archives.
What's the price of a TB of high TBW NVMe storage?
This above workload scenario is 100% write based. What about drives in data-centers that have lots of user images that they are unlikely/not-incentivized to prune. The drive could, hypothetically, only need to be written once, then forever read. When do sectors start going bad here? Or is the failure mode otherwise, in other componenets? Will this drive last a century? More?
What other modes of failure should we expect, how probable are they?
I love your challenge here @JAlexoid. Switching from thinking in terms of TB/$ to TB/y/$ is a great twist.
I wonder why most SBC only use sata (when they even have a storage connector). Is M.2/PCIe so much harder to add ? Or at least M.2 with sata 6Gb/s.
2xmPCIe (m2), 1xmSATA (m2), 1xSATA.
There're also plenty of boards around Ryzen Embedded (V1605B/V1807B/etc) if you need more lanes. There's Udoo Bolt. Sapphire and iBASE also has several boards.
I'm not aware of any decent contenders with multiple m2 sockets based on ARM, if that's what you're specifically looking for. But you can always buy a cable or board that splits or bifurcates a single interface into multiple m2 slots. I haven't tried any of these myself but there are plenty of options on Ali, eBay, or directly from embedded/industrial vendors depending on your cost/reliability preferences.
I've spent quite some time looking around for the optimal low-footprint storage cluster options and eventually ended up with mITX form-factor after realizing I want plenty of RAM and need more cores per node to handle ZFS+gluster+encryption.
There are plenty of options if you don't, though!
I'd advice to check out the APU's, either way. They have served me very well until I started upgrading and have now been repurposed for other use.
I've had positive experiences with the APU2 line as well - getting ECC support on the RAM now that firmware is being developed in the open was a really nice change: https://pcengines.github.io
---
These interfaces can be really confusing, and it doesn't help that even manufacturers and vendors often incorrectly mix them up as well, which makes searching for adapters quite an ordeal...
The Wikipedia page[0] is good as a primer now, but I remember spending like an hour or so figuring out if I could put an M.2.2260 in an M.2.2280 port (hint: the last digits only signify the length of the card so yes). And then the B-key/M-Key/E-key varieties, which ARE incompatible. I swear PCIe/SATA are worse than USB-C/TB in figuring out what you can put where to what effect...
I did discover that JMicron cards[1] kinda work if you leave the drives powered off until the kernel boots and then power them on and let them hotplug. Wasn't ever able to make it work reliably though.
[1]: https://www.newegg.com/syba-model-si-mpe40150-minipcie-contr...
But man, that website is straight out of the late 90's/early 00's.
Which makes it 100 times more readable than the vast majority of today's crappy websites.
About the products, never used PCEngines boards for NAS usage, but made some firewalls with them in the past and they're very reliable.
https://www.qnap.com/en/product/tbs-453dx
(edit - they aren't nvme slots as I first said, they're m2 sata)
Marvell's EspressoBin[1] was a great board, because it was small, easy to power, easy to plug a drive into, had space for wifi, had decent ethernet. So much goodness built in, at a great price point. Please all, more ambition on the I/O front!
MediaTek MIPS CPU and only 512mb RAM though. The community is kinda moribund and the installation instructions don't match the firmware I got on the device. I've made a couple of halfhearted attempts to update to later versions but haven't had any success so far.
There are a few pitfalls so make sure to check your devices Mac address before and after the install. Also do a rudimentary performance check on the HDDs with hdparm. I was disappointed at the gnubee as well because of the (IMHO) stupid design decisions with the bootloader and the bugs it entails when running a newer system from an SD card. (You have to name your partition a certain way and it changes to that partition halfway through the boot. Also they never really tried (afaict) to mainline uboot and the version they used is ... not nice.
I really wanted to like and to use it, but for me it was a bit unstable with a full bay and since I didn't want to buy a bunch of new 2.5" discs for a unstable system and my 2.5" discs I tested with were all just 250gbit doesn't really work for me
I also discovered that the 2.5" drives I bought to use with mine (Seagate ST3000LM024) are DM-SMR and thus fall out of even mirrored arrays within 24 hours of booting. :(
Also because low RAM and MIPS, running Go-based software like Minio or Restic seems unlikely (or at least unstable).
One of these days someone will do a proper homebrew NAS board. Surely.
https://serverfault.com/questions/13839/does-orientation-aff...
The easier answer I'm comfortable giving (despite me not being a sysadmin by trade) is that good storage software/setups should abstract over and protect you from something like this, because SSD/HDD bit-rot is absolutely a thing regardless of orientation -- if you're not using a WAL-ing (ex. zfs) or checksuming (ex. ceph's Bluestore) file system you're exposing yourself to data corruption. Only thing left is whether you'll actually have your drives long enough for them to exhibit complete failure from orientation, can't know that except to test like Backblaze/other providers do.
Makes sense to me, the parts are fixed, they're not heavy, gravity does not affect them.
However I've not seen anyone test this in real life.
FWIW I've only ever had drives mounted horizontally, the normal way and upside down. All of them had about the same lifespan.
For what it's worth, a bunch of server OEMs sell top-loading storage servers, eg. https://www.servethehome.com/dell-emc-poweredge-xe7100-100-d...
Don't know about relative reliability to horizontal mounts, but afair the drive manufacturer recommends mounting either vertical or horizontal and we didn't observe any obvious issues.
So, at this price point your probably better off with a rpi4 and a USB3 disk enclosure. The nice thing about that solution is that the RPI is slowly gaining full out of the box OS support (vmware, windows, *bsds, along with the various linux's), it has much better cores (a72's), and the end user isn't limited to a 2 disk solution as there are a number of 1/2/4/5/etc bay USB3 enclosures. The end result is a flexable solution which can saturate the 1Gbit ethernet on the rpi4.
It has at least one advantage over the Raspberry Pi for NAS applications: a real SATA bus. That means no mucking about with USB-to-SATA bridges, many of which do awful things like meddle with SMART reporting or spin down disks when they should not.
The barrel connector for power is also an advantage IMHO, since mysterious failures due to voltage sags turn out to be very common with USB-powered single-board computers. (USB power can be done without such problems, but extra care must be taken to avoid the many cables, power supplies, and connectors that are not up to the task.)
The S905 system-on-chip might not be capable of maxing out the throughput of an SSD, but for people who want a spinning rust server that's light on power consumption, it could be a pretty good fit.
Thomas Kaiser, as usual, has some good insights over in the CNX Software comments:
https://www.cnx-software.com/2020/10/20/odroid-hc4-low-cost-...
Also worth mentioning: The RockPro64 has a faster CPU, a PCIe slot, a NAS enclosure, and mainline support in Debian unstable (though the Debian installer doesn't yet install a boot loader; that must be done manually for now).
OTOH, for double the money, the ODROID-H2+ kicks the pants off all these devices, by having dual 2.5Gbit Ethernet, actual sata ports _AND_ usb3, expandable ram, nvme boot options, out of the box software support for pretty much anything you can imagine, including particularly freenas, crashplan, plex, and various other things people want to "just work".
That might have been true once upon a time, but not any more. Once a board is supported by the open source community, it is no longer dependent on vendor updates. I think Armbian has a good track record of long term support, and mainline Debian has a fantastic record of it. The RockPro64 (which I mentioned above) doesn't even require any vendor blobs.
> the ODROID-H2+ kicks the pants off all these devices
That device uses an Intel Celeron processor, and probably more power, and costs nearly twice as much. I don't consider it comparable to a low-power ARM board outside of a superficial sense. (Nice to know it exists, though; it seems like an interesting middle ground.)
As far as whether the celeron is better or worse power wise, I couldn't answer. But for sure I wouldn't make such sweeping statements. A lot of people think ARM=low power, but the power mgmt on these more "open" boards is actually quite bad. A large part of that is simply the fact that mainline linux is missing much of the finer points of controlling these boards. So, maybe the board could be X watt's with a given workload, but due to the lack of frequency control on the memory controller (or whatever) it ends up being 5-10X. The intel boards tend to be much more dynamic at this point.
For example, comparing an rpi4 and an atomic pi, you wouldn't expect from the massive heatsink on the atomicpi that it ends up being much lower power than the rpi4 for quite a number of workloads.
Sure its gonna do that. Major difference to Synology is it has a pretty case and usable out-of-the-box software, while here you've got to be comfortable with some Linux stuff.
If you have a separate backup for the things that cannot be easily replaced (movies, music can just be re-downloaded), than I don't see the need to worry too much about redundancy or ECC RAM or ZFS etc.
By comparison this Odroid box comes with Ubuntu preinstalled and that's it. Much more flexibility, but much more work involved too.
there are many useful and easy to install apps from the store. and backup to cloud is just a few clicks away.
for me, i just want sth simple to setup and forget. synology does that and does it really well for me
I've had a good experience with ODROID U2 (I personally think ODROID is my favorite SBC producer, so I'm biased, but most have some debian-based or similar distro that you can run and be productive things with.
In the more concrete world, both ZFS and Ceph (Bluestore has checksuming now) run on ARM as far as I can tell:
https://openzfs.github.io/openzfs-docs/Project%20and%20Commu...
https://developer.arm.com/solutions/infrastructure/developer...
Releasing a board meant for NASes that you can't run either of those on seems pretty short sighted for a company that's definitely not new to the SBC game.
Fortunately the Odroid HC4 is 64-bits, unlike its predecessors (HC1 and HC2).
The short answer is: the Amlogic chip's ARM "Bifrost" GPU is still basically unusable, because ARM has a huge problem upstreaming anything, but that could maybe change some day. But the rest of the platform is what I would classify as "fantastically well supported", and will run excellently with a mainline kernel (and also with fantastic uboot support)[2].
We're still some time away from boot-systems being standardized, so uboot is "the" way to go pretty much now, which targets embedded systems, but eventually we all hope the Server Based Boot Requirements[3]- which mandate ACPI and UEFI support- will become semi-standard, such that we can use better secured & more capable standardized bootloaders.
[1] https://news.ycombinator.com/item?id=24813526
[2] http://linux-meson.com/doku.php#kernel_mainlining_progress
[3] https://community.arm.com/iot/b/internet-of-things/posts/arm...
Apart from iOS on supported hardware, I'm yet to experience a platform that's been entirely free of such issues.
Pick up a sketchy Android TV box from Ali that's not in the top 3 and yes, you're in for some "fun" times.
A couple of years ago I bought a C2 and it still hasn't progressed beyond the 3.14 kernel required by the versions of Android Samsung was shipping on phones using that chip[1]. Also, getting the display working correctly with a monitor attached via an HDMI to DVI-D cable was obnoxious (again reflecting the CPU's Android heritage - hardware expects to be told exactly what's attached and doesn't try to detect anything).
[1]: https://wiki.odroid.com/odroid-c2/software/building_kernel
If you expect a flashable vendor-provided OS image with long-term support and vendor repos that get continuous updates, I think you'll be frustrated with anything else than a Raspberry Pi or NVIDIA Jetson.
If you're fine with community images and configuring/compiling kernels yourself, vendors like Odroid/RockPi/NanoPi/OrangePi/Pine64 are fanastic. They're hardware manufacturers, not an OS vendor.
At least the spec is open enough that anyone has the ability to do things like Armbian without having to do reverse-engineering or extracting data from binary blobs (which is the case with a lot of other vendors).
The Odroid C2 was released in 2016 and is now EoL.
For all my disappointment, it has served as a useful lesson on early-adoption and expectation management. =) So I'm not regretful, just cautious.
> Better buy a RPi4 with a NAS Hat
Do you know of such a HAT that does USB-attached SCSI Protocol correctly and reliably, and passes through SMART reporting without meddling, and doesn't inappropriately spin down the disks under its control? I wouldn't mind bookmarking such a thing if it exists.
I considered the RPi for a recent NAS build, but I didn't want the higher CPU load imposed by a USB-to-SATA bridge (as compared to native/PCIe SATA), nor did I want to play chipset roulette looking for a SATA bridge free from the problems I mentioned above.
I ended up using a RockPro64 with a Marvell 88SE9235 PCIe SATA card. It works well, and boots mainline Debian with no vendor blobs. Assuming the electronics don't wear out, I expect it will continue to run fully-updated linux for a long time to come.
So you should be able to get a local command prompt and run things like rsync or SFTP over SSH, yes ?
Several people have posted HC1/HC2 gluster clusters but since gluster dropped 32-bit support there hasn't been a great alternative for that until this.
I realize the ethernet is the primary thing a NAS tends to need. Still can't be a bit disappointed that this board doesn't have USB3. Seems to be the chip, the S905X3, which multiplexes USB3 and PCIe on the same pins: you've gotta pick! And they use that PCIe for SATA, so there's no pins for USB.
Alas alas, PCIe switching is just too expensive. It'd be so nice for some small PCIe switches to be more robustly used. And like, why not more pcie micro-IO hubs? Big desktop CPUs talk to platform hubs that have a bunch of SATA, USB, PCIe on them. They have no where near enough bandwidth to the CPU to support it all at once, but it allows for good expandability. Systems like this would benefit so much from a ~$8 platform hub, whose uplink is a PCIe lane or two, and which exposes 4 SATA, a USB3 root & 4 port hub, and 2x pcie x1 links. One of the notables from the recent "Turing Pi 2" was discussion of how to attach a bunch of peripherals to a micro-cluster, and they discussed their shying away from PCIe, which I started some discussion on[2]. I think the hub would do them well, but any old PCIe switching, especially with NTB, would do them a world of good, but is just regarded flatly by most of the world as "too damned hard" and/or "too damned expensive". Enough "alases" in this post already, but it would be so enabling to see us get better at "small" pcie.
https://wiki.pine64.org/index.php/NASCase
The first unit I got (the case) had manufacturing flaws, but the second one was okay. The board it's built for has been doing well in the first few weeks of testing.
I was looking for a NAS that could take 2.5" drives a year or so back (I no longer believe in spinning rust).
In the end the prices of them put me off so much that I built a mini-ITX server instead. This might have swayed me (though a 4 driver version would be preferable)
https://magazine.odroid.com/article/200tb-glusterfs-server-u...
It would work as ceph node, but I would argue that it doesn't fit the bill since you should have ~1gb mem per tb storage. If your load is low you can do with less and you could potentially use swap, but it would regularly freeze on you or crash the osd processes depending on your setup.
If however you want to try it you could get raspberry pi 4s and use them over usb3. Get them with decent memory and use spinning rust to host the OS. To have an up to date version of ceph I would suggest using cephadm with podman to get it up and running with good builds.
1) enough memory
2) up to date ceph version
Reminder: cephadm is kinda beta right now
Maybe the odroid hc1 would be better, but I don't have one
Edit: fixed auto correct
I capped my OSDs' memory usage using osd_memory_target (set to 1.5GB), to make them run on HC2s with 4TB and 6TB disks, I never had stability issues with these.
I don't have logs anymore, since I upgraded my cluster to Nautilus which can't run on HC2s (because they are 32-bits and I can't roll back my cluster to Luminous). However I have one running on a N2 and a 6TB disk with the same memory target (1.5GB), and it didn't crash since the last reboot three weeks ago, and it's currently using 1.3GB.
Unfortunately Ceph stopped supporting 32-bits ARM in Mimic or Nautilus, so I'm looking forward to the HC4
I might be wrong, but I don't see why one wouldn't be able to build freenas for arm (all the included software should be arm64-compatible), it's just that so far no one took the time to do so and share it.
If you've ever lurked there, you'll probably know the answer already.
The source has more information, including detailed specs: https://www.cnx-software.com/2020/10/20/odroid-hc4-low-cost-...
Over the years I've accumulate 10+ Odroid boards that are all still in use, and non have ever died on me.
Actually that's not entirely true, I did have an Odroid Weather Board that got a burned BME280 sensor because I left it outdoors in the rain.