Hacker News Reader

Comment by jnxx | Hacker News Reader

You can use a Raspberry Pi (perhaps a Pi 400) if you work around the fact that the internal flash memory is not made for constant writing. Thus, you have to arrange the system so that it is read-only in normal state - which is not difficult. The Pi 400 also has an advantage that it has a keyboard, so you can cleanly shut it down without logging in, which makes a lot of sense for a storage device.

Also, for RAID1 you need an extra powered USB hub. The normal power supply of the Pi 400 is not sufficient for more than one disk.

Then just install a software RAID, NFS and possibly rsync and Samba. A simple NAS built in this way will come at about 240€, depending on disk size and will be mostly equivalent to a commercial one for twice the price. As an advantage, you can run well-known, standard software which is completely under your control. And since it is Debian-based Raspbian, it can be upgraded easily without breaking stuff.

View on HN

what ever you do, dont do this.

The Pi400 is a brilliant machine, but this is not a way to make networked storage for data you want to keep.

if you want to use a raspberry pi for network storage, and you want to keep your data, make sure you have n+1 pi & disk machine.

Raid isn't going to help you here, you need redundant nodes. for the same €240 you could get two minimum spec pi4s PSU and usb HDDs

> make sure you have n+1 pi & disk machine.

Why? The thing that could fail are the disks. So, one can use a RAID1 with two disks and this should, for normal applications be enough. I had a DNS-232 before as a NAS, it had 32 MB RAM, a tiny ARM CPU, and two large disks. It was working nicely over about ten years, and one could even run a web server plus a (slow) wiki on it (I had to replace the casing once, for a botched firmware update). The main problem I had with it that it was that it was impossible to update (I was running Alt-F firmware, which was developed by the community after the original firmware had constant problems with unneeded RAID synchronization, and the vendor had to release its code due to being bound to the GPL).

The Pi400 is wildly more powerful, has a much faster CPU which only draws a few Watt, and can easily be connected to several USB disks.

> Raid isn't going to help you here, you need redundant nodes.

Why exactly? My set-up is working well.... did I miss it needs to be somehow lighter than air (because heavier-than air flying machines are not possible, y'know)?

> Why exactly? My set-up is working well

congratulations I am very happy for you!

The issue is not speed, its the USB disks. firstly you are reliant on the disks presenting themselves in the same order at the same time. (historically that was challenging) ZFS can cope with this quite well, MDADM, not so much.

You can use device IDs, which gives better results. Another issue I've bumped into is that mdadm has come up before the USB disk is initialised, causing all sorts of issues. in short raid over USB is just plain fragile.

> Raid isn't going to help you here, you need redundant nodes.

This setup will have four common failure modes:

o power issue leading to corrupt "/" or any other SD card failure

o usb disk breaking

o OS crash borking everything

o Accidental deletion

Raid might protect against a usbdisk breaking, assuming you spot it in time (most people don't have monitoring.) its not even going to give you a performance boost either, its USB3.

This is why n+1 is better. assuming you're not keeping each node next to each other then you are much less likely to experience a simultaneous outage.

> firstly you are reliant on the disks presenting themselves in the same order at the same time.

In Linux, which is what Raspbian is, it is standard nowadays to use UUID disk labels.

> Another issue I've bumped into is that mdadm has come up before the USB disk is initialised, causing all sorts of issues. in short raid over USB is just plain fragile.

No issue with that. I guess systemd handles this. Only thing I found is that the hub needs to provide enough power on booting up.

> power issue leading to corrupt "/" or any other SD card failure

For that, I have a backup of the sd card. I am using an "industrial" SD card by the way which costs a few bucks more. Also, the card is running in read-only mode, using unionfs. Logs are written to a partition of the RAID disks.

> usb disk breaking

That's handled by RAID1 and backups.

> OS crash borking everything

I've never seen that on Linux, in 22 years using it. The closest I came to was botched EFI parameters from an Ubuntu install gone wrong, on a Desktop PC. Wait, I also had a problem in about 2003 with a broken ISA disk controller. Ah, and I am using EXT3 which is a journaling file system.

> Raid might protect against a usbdisk breaking

Yeah, that is what it is for. It is not a substitute for backups (but nobody said something like that, I think). In fact, it is easier to make backups from a Pi, using the standard Linux tools like tar, ssh and rsync.

> its not even going to give you a performance boost either, its USB3

For my purposes, it does not need to be faster (and it is much, much faster than the old DNS-232). The Pi 400 has Gigabit Ethernet and plenty of RAM. I am running a MoinMoin wiki instance on it, and it is more than fast enough for personal information management.

It can well be that a small thing like that is not fast enough for a medium-scale business with, say, a dozen clients. But this was not what my requirements were. I wanted to have a solution which is very low-power. And the Pi 400 is probably one or two Watt if idle, and the disks are spun down:

https://en.wikipedia.org/wiki/Raspberry_Pi#Specifications

https://www.raspberrypi-spy.co.uk/2018/11/raspberry-pi-power...

> This is why n+1 is better

So, what kind of set-up are you exactly talking about, a distributed system of some kind? A storage cluster? Ceph? DynamoDB?

Just to complete this:

As said, it is not a good idea to leave the root file system of the Raspberry, which is on an SD card, writable for heavy-duty, long-term tasks where reliability is important. It is not made for that. Among other things, this includes logging.

Instead, I am using this script:

https://github.com/fitu996/overlayRoot.sh

which mounts an overlay-filesystem with the bottom layer being the root-fs, the upper layer a new fs on writable media, which is a RAID partition in my case, and swaps the overlayfs with the root fs. That means that from then, the sd card of the Raspberry is not modified, and all changes go to the RAID disk. If you want to modify the system configuration, you need to re-boot with a different kernel parameter, which is detailed on the linked page.

> > OS crash borking everything

> I've never seen that on Linux, in 22 years using it

Its surprisingly common, especially when a machine is underload and spending ages flushing to disk. A kernel oops happens and bam, broken partition.

> So, what kind of set-up are you exactly talking about, a distributed system of some kind? A storage cluster? Ceph? DynamoDB?

Noooooope!

just a second machine, with a copy of whatever you want to keep. when you are doing backups, have each one as a round robin target.

Ceph is not something you'd want to run for redundancy/reliability. I've spent many years supporting "HA" filesystems. The simpler the better (unless its GPFS, that's actually good.)

> Its surprisingly common, especially when a machine is underload and spending ages flushing to disk. A kernel oops happens and bam, broken partition.

1. That is what journaling file systems like ext4 are for. These are exactly to prevent that unfinished writes of metadata corrupt the file system.

2. Of course, it is possible that the Linux kernel goes all bonkers, writing random stuff to you disk and firing up nuclear missiles. But I have never seen that. I also have never seen a kernel Oops on a NAS. Kernel crashes are usually caused by faulty device drivers.

3. Also, in this aspect, there is no technical difference at all to a commercial NAS since they use Linux as well. They also use ARM CPUs, like the Raspberry, since they save power and do not need a fan (which in turns makes them less prone to failures).

> just a second machine, with a copy of whatever you want to keep.

Keeping backups is something I would seriously recomment for any kind of NAS or server.

I do really not see how your argument applies in any way to a NAS based on a Raspberry Pi. The main difference I see is that it is substantially cheaper, and better suited to people who want to install extra software and occasionally like to tinker a bit. And I am obviously not recommending that the Central Bank of Canada or an institution like that uses it.

> The issue is not speed, its the USB disks. firstly you are reliant on the disks presenting themselves in the same order at the same time. (historically that was challenging) ZFS can cope with this quite well, MDADM, not so much.

I have checked about that and mdadm uses uuids to identify partitions on USB disks:

https://serverfault.com/questions/739580/mdadm-can-i-reconne...

https://serverfault.com/questions/460138/mdadm-disk-configur...

https://www.linuxquestions.org/questions/linux-software-2/md...

Here is how it looks on my array:

   root@arau:~# mdadm --detail /dev/md1
   /dev/md1:
              Version : 1.2
        Creation Time : Thu Dec 10 08:56:22 2020
           Raid Level : raid1
           Array Size : 4821105600 (4597.76 GiB 4936.81 GB)
        Used Dev Size : 4821105600 (4597.76 GiB 4936.81 GB)
         Raid Devices : 2
        Total Devices : 2
          Persistence : Superblock is persistent
   
        Intent Bitmap : Internal
   
          Update Time : Thu Apr 15 11:18:54 2021
                State : clean 
       Active Devices : 2
      Working Devices : 2
       Failed Devices : 0
        Spare Devices : 0
   
   Consistency Policy : bitmap
   
                 Name : arau2:1
                 UUID : 7c1e57ea:1f7ebff2:415c1b0a:8f959031
               Events : 21463
   
       Number   Major   Minor   RaidDevice State
          0       8        2        0      active sync      /dev/sda2
          1       8       18        1      active sync     /dev/sdb2
   
   root@arau:~#

   root@arau:~# mdadm --examine /dev/sda2
   /dev/sda2:
             Magic : a92b4efc
           Version : 1.2
       Feature Map : 0x1
        Array UUID : 7c1e57ea:1f7ebff2:415c1b0a:8f959031
              Name : arau2:1
        Creation Time : Thu Dec 10 08:56:22 2020
        Raid Level : raid1
      Raid Devices : 2
   
    Avail Dev Size : 9642249512 (4597.78 GiB 4936.83 GB)
        Array Size : 4821105600 (4597.76 GiB 4936.81 GB)
     Used Dev Size : 9642211200 (4597.76 GiB 4936.81 GB)
       Data Offset : 264192 sectors
      Super Offset : 8 sectors
      Unused Space : before=264112 sectors, after=38312 sectors
             State : clean
       Device UUID : 52594d2c:77f9c0cc:0b4243f6:6a8e6bcb
   
   Internal Bitmap : 8 sectors from superblock
       Update Time : Thu Apr 15 11:18:54 2021
     Bad Block Log : 512 entries available at offset 32 sectors
          Checksum : ca81c1ad - correct
            Events : 21463
   
   
      Device Role : Active device 0
      Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
   root@arau:~#

You see the second UUID entry? It is an USB disk.

And I looked into the madm changelog and git log and I did not find any reference to problems as you describe since 2014. I am a bit confused why you refer to hypothetical problems or ones which do not exist anymore for years?

Clean shutdowns are indeed preferred, but synology devices will do a clean shutdown (delayed as required) if you press and hold the power button until it beeps and starts flashing, then release.

Yeah. That is a problem with older Raspberry Pis, which do not have a keyboard. It is possible to use a GPIO port for this, and shut it down with an attached button. But one has to be careful not to use the port unless it is configured properly, because that could cause a short-circuit, and the R-Pi is not protected against that. Having a keyboard on which you can type Ctrl-Alt-Del is much easier to use and will work on most relevant circumstances. In modern Raspian editions, this can be configured in systemd.

I'd be really curious to know why people are downvoting this. I can't see any technical reason against it. Of course, for some people this might be too complicated or too much hassle in comparison to buying a more expensive commercial thing. But for other people, the cost savings might be worth the work. It is also easier to keep an almost completely open source solution fully up-to-date.

Reply on news.ycombinator.com