DIY Raspberry / Orange Pi NAS That Looks Like a NAS – 2023 Edition | Hacker News Reader

DIY Raspberry / Orange Pi NAS That Looks Like a NAS – 2023 Edition

instructables.com

174 pointsaxiomdata31673 comments

This is lovely, and it handles three of the largest problems with using SBCs for NAS:

1) Stability due to power delivery. Having dedicated, local power delivery can make a HUGE difference with SBCs.

2) Reliability of SD card media. Spinning rust, or even real SSDs, are worlds more reliable than SD cards. If an SD card is used just for booting, then it'll last forever.

3) Overall reliability of stored data. Having two drives in a proper mirror is definitely desirable.

It matches my criteria for my own colocated 1U Raspberry Pi server. Mine has additional criteria: I have another Pi for OOB serial and remote reset, an ethernet switch so I don't have to pay extra for a separate drop, and USB-3 to SATA adapters with UASP because the Pi 4 supports USB-3 (> 200MB/sec per drive, simultaneously).

https://twitter.com/AnachronistJohn/status/15515998004533084...

> It matches my criteria for my own colocated 1U Raspberry Pi server. Mine has additional criteria: I have another Pi for OOB serial and remote reset, an ethernet switch so I don't have to pay extra for a separate drop, and USB-3 to SATA adapters with UASP because the Pi 4 supports USB-3 (> 200MB/sec per drive, simultaneously).

That doesn't seem worth it compared to just buying some old used server, especially if you going to get separate pi just for oob.

Hell, currently on ovh/kimsufi $6/mo gives you 2c/4t atom with 1TB of disk and 4GB of RAM

I pay ~10 eur for colocated 2c/4t atom with 2TB

It's not worth it to you, but it's worth it to me.

See, I know that my hardware is secure. I don't have to worry about bullsh*t closed-stack IPMI / iLO that requires a specific JVM from a decade ago. My server takes less power than "some old used server", which means I pay less money, anyway.

Also, my server is my own hardware that I own. Is your €10 a month "2c/4t atom" your own hardware, or is it a "dedicated server" that is controlled and managed by OVH / Kimsufi? If the latter, then how do you know it's secure? You don't. You have to trust OVH / Kimsufi. Real, owned hardware is not comparable to "dedicated servers" when it comes to security.

What's fine for you isn't fine for me. I actually care about who has direct access to the systems I run.

Regarding the rented hardware, is your main complaint the remote management features?

In terms of physical security, I don't think there would be much difference between rented and owned hardware.

If I rent hardware, I have no real way of knowing whether remote management is accessible by other people, nor whether it has been secured properly.

At least with hardware I own, I know every part of it has been configured by me and by nobody else.

Of course rented hardware, where the customer has full control of a whole system, is preferable to any sort of "dedicated server", but IPMI / iLO is, quite literally, a hardware back door. Yes, the biggest issue is the remote management.

What's the problem? I backup all my data to a Hetzner storage box, but I encrypt everything on my side before sending. They can peek at my files all day long.

I'm not just backing up my data. I'm colocating a whole server that does lots of different things, some which I'd like to keep private and safe.

Do you have anything detecting if the hardware has been tampered with?

2U servers are really noisy, but colocating is expensive.

I thought a server made out of Pi's would make it more affordable to keep at home, if anything.

It's a wonderful arrangement you have there. I'm embarrassed to say my idea of a cluster is a handful of netbooks.

You’d probably want ECC memory in a NAS host, as well as IO and CPU performance that can handle an overnight zpool scrub for whatever amount of important data you have (baby photos yes; Sailor Moon archive no.)

If your NAS also handles offsite backups and you’re using something like borg or rsync then that too is going to require quite a lot of compute to be able to checksum entire filesystems.

Are there SBCs that have these features? I would be surprised (but delighted) if there were any.

Most off-the-shelf NAS boxes do not have ECC.

I should have added that if you do want a NAS host with ECC then the HPE ProLiant Gen10 is a good platform. Instead of drive caddies you screw little bosses into your drive and they slot right into the front bays.

Not hot swappable though. The advantage of hot swapping is that, in a data centre, your ops team doesn’t have to coordinate with your server admin team so that the host is powered off at the exact same time as the operator comes past your rack with a trolley of new disks. Hot swappable means new disks can be added without having to synchronise the host being shut down.

If you’re doing this at home it’s easier to coordinate your admin (you) running sudo poweroff right before site operations (also you) comes around with a new drive.

You go through the trouble of redundant power and serial but refuse to spring for a second ethernet port? Strange.

There's nothing strange about that. Why pay $10 a month extra for something that can be eliminated with $20 worth of hardware?

This is a cool project and I am happy to see interest in this sort of thing.

Too bad though that general purpose SBC's kind of suck for NAS/Storage though. Both the rpi4 and orange pi zero mentioned in the project lack any kind of SATA port and relies on usb to sata.

The best project I saw was the Helios4/Helios64 (https://kobol.io/), but sadly, they folded in 2021. Is there anything even remotely similar around?

Right now I am still using intel-based mini-pc's for diy NAS storage as they simply work better.

My “NAS” is a “slim” Intel NUC 8 with a 3D-printed “riser” that allows it to fit a 4TB (15mm) 2.5” HDD. No redundancy of course, but otherwise it’s superior to a legit NAS or Pi NAS in every way.

I just bought a NUC 8! Do you have an STL anywhere that I could use?

I used this one, which fits a 15mm drive, but it’s tight:

https://www.thingiverse.com/thing:5441566

There’s also this one that I did not try, which might fit 15mm drives a little better:

https://www.thingiverse.com/thing:5140192

There is also the odroid hc4. It only has two slots, but the argument is, for the price you may as well expand by getting multiple of them.

ODROID-HC4 (toaster), ODROID-HC4-P KIT (horizontal drives in enclosure)

it seems like none of the consumer-level ARM chips expose enough PCIe lanes to do anything interesting, no SATA controllers or 10Gb+ ethernet

Orange Pi 5 (rk3588 based) has m.2 pcie 2.0. I think maybe only a lane?

Yep, you can get an m2 to SATA breakout board pretty cheap.

If you pickup a Rock 5b (rk3588 as well) you will get 2.5g ethernet too.

There's a lot of different m.2 sockets. This m.2 has a PCIe not a SATA connection. You'd need not a breakout board, but an actual SATA adapter with a PCIe <-> SATA chip on it. Still cheap, not a big barrier, but the terminology of "breakout board" is quite incorrect.

https://www.reddit.com/r/OrangePI/comments/zm3sms/does_the_o...

RK3568 / RK3588 have 3 SATA controllers and for example the Rock Pi 3A seems to have the SATA routed to the bottom m.2 connector, as well as the possibility to switch the USB3 port over to SATA

https://linuxgizmos.com/pi-like-rk3568-sbc-grows-sata-out-of...

"The Rock 3 Model A exploits the RK3568’s ability to multiplex SERDES lanes to enable SATA support on the dual USB 3.0 ports (OTG and host) via a SATA breakout cable."

Not ARM, but there's the Odroid H3 and H3+.

https://www.hardkernel.com/shop/odroid-h3/

https://www.hardkernel.com/shop/odroid-h3-plus/

Yeah, these seem pretty great for a small NAS, especially with a simple mirrored ZFS setup.

I'd love to have something similar with ECC RAM.

Expensive, but the FAQ says it supports ECC:

https://morefine.com/products/morefine-s500-mini-pc

And it has 2 NVME slots.

At that point just get normal MiniITX board and case

ITX cases are often 10 liters or more, and the smallest ones are still a few liters. Some of those embedded boards can have cases of a liter or less.

...and you're building a NAS with a minimum of 2 3.5 inch disks

There’s a hundred ways to gate-keep what a NAS is. My personal minimum is redundant power supplies, 10GbE, and ECC memory, but I realize that some opt for less. In the end it doesn’t have to be anything other than storage attached to a network.

There is Argon Eon which gets good reviews, but I haven't decided to try it yet. A bit more expensive than this diy - 180 EUR [0]. It has space for 2x 2.5 inch and 2x 3.5 inch disks. If someone has it, I would be curious to know how good it is.

[0] https://www.en.galagomarket.com//item/display/3130/10443_ohi...

> Is there anything even remotely similar around?

There is GnuBee [1] with two models for 2.5" drives [2] and 3.5" drives [3].

1.: http://gnubee.org/

2.: https://www.crowdsupply.com/gnubee/personal-cloud-1

3.: https://www.crowdsupply.com/gnubee/personal-cloud-2

I have this thing. It's just about useless imo. The CPU is so low power that it maxes out doing a file transfer making it incredibly slow. It's maybe neat if you want to tinker with open source hardware.

Also weird that they acknowledge that the GNU foundation asked them to change the name yet they seemingly won't ever do that.

I helped a friend to create home storage with two units (redudancy) of gnubee PC1 (2.5" drives), and setup is ultra useful, fast, low power and reliable (contrary to sibling comments ;) -> highly recommend it.

Usage (few years of happy usage):

* openwrt custom build with patches

* FDE for all drives using linux cryptsetup, including custom patches for cryptoengine (critical)

* big (read x TB) repos served by locally (on gnubee) run syncthing

* big files served for local media devices over ksmbd, (read 4k streams ;)

* drives in raid1 (linux mdraid)

ultra low power for 24/7, with 90% of energy budget for drives.

Highly recommended! But yes, I agree that it required tinkering, skills and time to set up.

There is such a thing as good enough. For instance, are you trying to watch a two hour movie in five minutes, or two hours?

For most things, particularly video streaming, 100 Mbps is more than enough. If you want faster, a Pi 4 can saturate gigabit with mirrored storage on USB. You don't need dedicated SATA ports to do that.

I've tried to use a pi as a cloud storage but I just couldn't find a usable setup. SFTP mounts are unreliable and slow, NFS is insecure so can't be exposed over the internet, Nextcloud had the best UI but run like a dog on the pi 4.

I eventually built a full x86 desktop but found nextcloud is slow with any amount of processing power. Put 20k photos in a directory and it just stops functioning.

Gave up and went with Google Drive.

20k files in one directory could hit filesystems hard in general - not only a PHP app on top of that

They don’t really present any alternatives. I wouldn’t care if internally it split it in to multiple directories and presented it as one in the UI.

Performance is important but my main problem is reliability of the usb to SATA controllers.

In practice you often get weird timeout issues and unintelligible errors in kernel logs, hard resets, drives disconnecting or worst, data corruption. It's also always difficult to tell if it is a failing drive, or if it's a controller bug, or power supply issue.

Does SMART work over usb? If not how are you alerted of drive problem?

This is another thing that really depends on the USB to SATA controller you use.

Some are OK, some are great, and some are garbage before they make it to a warehouse shelf.

It depends on the USB-SATA adapter. Yes, I get some SMART data using smartctl. I also get alerted if a pending write takes longer than a few ticks, which would indicate issues.

The primary issue with USB-SATA is power. Give a good USB-SATA adapter, the drive, and the host computer good power, and you'll see excellent reliability.

My 1U Raspberry Pi server with two mirrored spinning rust disks on USB-SATA has been up continuously since September and has been busy with running SearXNG, DNS, email, web (including dynamic sites), MySQL, and so on, while also being used to compile while heavily going in to swap.

In other words, it has been doing non-stop disk I/O since September without an issue.

This is because everything in the machine has excellent power :)

You get that with cheap PCIe SATA ones too, gotta watch out. I eventually traced my NAS problems to that, shitty controller (I had 4 ports off mobo + 2 extra off controller)

Radxa has the Taco hat for the RasPi CM4, with 5 SATA slots, an M.2 Slot and 2.5GbE!

https://wiki.radxa.com/Taco

Around a year ago I picked up an Axzez Interceptor[0]. It's a Compute Module 4[1] carrier board with a 5-port SATA controller hanging of the Pi's single PCIe lane. They also sell a piece of metal to adapt the board to a Mini-ITX form factor. Bought a NAS-like case for it, and I'm really happy with the result.

[0] https://www.axzez.com/product-page/interceptor-carrier-board

[1] The module version of the RasPi4.

I also fell down similar rabbit hole.

I clustered five raspberry PIs using Ceph. It's not super great performing, but currently getting 30MBps reads and writes for different pool types; most notably 3 times replicated and 3+2 erasure coded one. There are 6 Adata HD720 2.5" 2TB USB spinning drives. Planning to add SSD for Journaling and wal now.

I've put that cluster through hell - like pulling one straight out and resoldered one USB PSU pass through mod and it didn't lose data at all.

I'm mightily impressed that it works at all

the scope is impressive. After showing all the hardware he goes on presenting his own developed "webOS" (https://arozos.com/) how to make the best use and get a lot of value out of the NAS.

By the way. Did anyone try to use a large sd-card as storage ?

I meant 256-512GB. In my logic it must decrease power consumption when keeping the same IO performance as external ssd connected through USB.

I am uncertain if USB can be a bottleneck there, but if I am not mistaken, SSDs offer better (more?) cache. This dramatically reduces time needed for non-sequential read/write operations.

USB - 30Mbyte/sec theoretically for RPI, but in reality somewhere around 20Mbyte/sec

SD cards - class 10 tested on RPI show the same performance 21-27Mbyte/sec

So the issue mostly in throughput performance of the interface

First time I’ve heard of an Orange Pi.

There are a ton of these sorts of SBCs, and they're all more readily available at reasonable prices than the Raspberry Pi. Most have Cortex-A53s, which are in the same class as a Raspberry Pi 3; the RPi4 has Cortex-A72 cores, but for most things that you'll use one of these for, that's not super relevant.

This lack of name recognition is a real pain for people. A lot of folks in the 3D printing community kind of hurt themselves by scouring the internet for marked-up Raspberry Pis from scalpers when a Le Potato or an Orange Pi or a NanoPi or a Rock64 would do just fine. (I use Rock64 boards when I don't need wifi, and Orange Pi 3 LTS boards when I do.) My printer hub is now a Raspberry Pi only because all of the decent touchscreens I found really depend on the Raspberry Pi pinouts to connect power and HDMI, but if you just need a computer in a case, the alternatives are very good and reasonably priced.

Good brand. I've been running several different Orange Pi SBCs for almost 7 years 24/7 now since back when Orange Pi PC still cost just USD 15 and lower specced Raspberry Pi 2 cost almost 3x that.

These days with EUR 22 no-VAT EU import limit gone, and post COVID price hikes, it's not such a great deal anymore. The price of Orange Pi PC is 2x it was 7 years ago, and the technology is now 7 years out of date.

I have been eyeing Orange Pi for a while. They made great boards but the limited choice of cases for them, especially for their latest board, has made me hesitate. The limited space in the city limited my access to 3D printer at home. FriendlyELEC does a much better job by selling board with cases.

Anyone install Plex Media Server on this? Biggest issue I see is the slow transcoding.

Great DIY project. I was expecting a simpler solution.

Why I don't see mdadm in this article? imho: useless project

The main point of this article is building a NAS pretty much from scratch. Setting up mdadm, is just a tiny part of the overall steps.

If you read Step 15, the writer mentions some sort of sandboxing built in to their operating system. I'd say that's the RAID you're apparently missing.

Is this booting from USB or from the sd-card? I can't figure it out from the installation instructions.

If it is the sd-card then poor installation.

What is an ideal setup that doesn't use that method?

I am not sure if I understand your question, but what I was hinting at is the reliabilty of the sd-card.

I've had several Raspberry installations get corrupted due to power outage, which the sd-card can't handle at all.

Back then I didn't get booting from SSD through a USB-Sata adapter working. I've read that it is easier now though.

What is your experience?

Read-only sd card works pretty well though in my experience. You can make it rw for upgrades and remount to ro once done. All the files which need write access (but you don't care about preserving) can be "saved" to tmpfs.

I'm currently using the read-only Alpine install for my Raspberry Pi based wireless AP.

Quite functional so far.

PITA

With normal install you can just configure unattended-upgrades (on Debian at least) and mostly forget about security updates, they will just happen (can even set a schedule for reboots for kernel updates IIRC).

But IIRC boot on card + root FS on SATA or USB-SATA works just fine and you only get some writes off occasional kernel update

I just moved my Home Assistant from SD to SSD and it couldn’t have been easier. Using the USB-Sata adapter I plugged my SSD into my Mac and then used Balena Etcher to dump the HA image directly to the SSD. Plugged it into my Pi and it fired up just like it would’ve if it had imaged to the SD card. I was pleasantly surprised with how easy it was and it’s nice to have HA running off of an SSD now.

You can clone it while running as well [0]. For speed make sure the adapter is USB3.0 and plugs into one of the USB3.0 ports on the pi and also that it supports UASP and enable trim on the drive [1].

[0] https://github.com/billw2/rpi-clone

[1] https://www.jeffgeerling.com/blog/2020/raspberry-pi-usb-boot...

Pi4 will boot straight from a USB drive. Given the unreliability of sd-cards this is preferable if you need a USB drive anyway.

Mine is running from a micro-SD card (+adapter on the old boards) since... I don't know, which year did the Pi 2 came out? It's not the same Pi anymore, but I'm re-using that 8gb card ever since.

Why would where this boots from matter? Linux can mount the rootfs from wherever, regardless of where the bootloader and kernel is stored. You can have /boot on sd card and the rest elsewhere on pretty much any SBC.

That's a poor evaluation ;)

Almost all BSD and Linux distros can boot off the SD card and mount their root on USB attached storage. This is better than doing the permanent fusing stuff, for instance, on Raspberry Pis, and there's absolutely nothing wrong with it at all.

Reply on news.ycombinator.com