Hacker News Reader

I did not expected such a sad news this Friday ...

AnandTech - one of THE sites that literally done hardware upbringing for me ... will be no more.

Thank you for all the in depth reviews and explanation how hardware work - I use this knowledge to this day ...

Farewell.

> As far as I know you can't have mismatching drives as a single contiguous space in ZFS. Maybe it has changed.

Yes you can, example below:

    vermaden ~ % truncate -s 1G FILE.0
    
    vermaden ~ % truncate -s 2G FILE.1
    
    vermaden ~ % mdconfig.sh -c FILE.0
    IN: created vnode at /dev/md0
    
    vermaden ~ % mdconfig.sh -c FILE.1
    IN: created vnode at /dev/md1
    
    vermaden ~ % lsblk md0  
    DEVICE         MAJ:MIN SIZE TYPE                                    LABEL MOUNT
    md0              2:17  1.0G zfs                                         - -
    
    vermaden ~ % lsblk md1
    DEVICE         MAJ:MIN SIZE TYPE                                    LABEL MOUNT
    md1              2:18  2.0G zfs                                         - -
    
    vermaden ~ % doas zpool create test md0 md1                 
    
    vermaden ~ % zpool status test
      pool: test
     state: ONLINE
    config:
    
            NAME        STATE     READ WRITE CKSUM
            test        ONLINE       0     0     0
              md0       ONLINE       0     0     0
              md1       ONLINE       0     0     0
    
    errors: No known data errors

I do not know how you buy your disks but when I need to buy storage I know exactly what SIZE I want to buy - I do not just get ANY disk and then think 'how' to utilize it :)

But with ZFS you have plenty of options to use disks of different sizes.

Example below, RAIDZ from 4 disks of 2TB and added a mirror of two 5TB disks to the same ZFS pool.

    vermaden ~ % doas zpool create test raidz md{0,1,2,3}       
    
    vermaden ~ % zpool status test
      pool: test
     state: ONLINE
    config:
    
            NAME        STATE     READ WRITE CKSUM
            test        ONLINE       0     0     0
              raidz1-0  ONLINE       0     0     0
                md0     ONLINE       0     0     0
                md1     ONLINE       0     0     0
                md2     ONLINE       0     0     0
                md3     ONLINE       0     0     0
    
    errors: No known data errors
    
    vermaden ~ % doas zpool add test mirror md4 md5      
    
    vermaden ~ % zpool status test                 
      pool: test
     state: ONLINE
    config:
    
            NAME        STATE     READ WRITE CKSUM
            test        ONLINE       0     0     0
              raidz1-0  ONLINE       0     0     0
                md0     ONLINE       0     0     0
                md1     ONLINE       0     0     0
                md2     ONLINE       0     0     0
                md3     ONLINE       0     0     0
              mirror-1  ONLINE       0     0     0
                md4     ONLINE       0     0     0
                md5     ONLINE       0     0     0
    
    errors: No known data errors

Hope that helps.

Regards,

vermaden

vermadenon FreeBSD Desktop Series

Hi and thanks :)

I did not created such thing ... but You can use SingleFile browser extension:

- https://addons.mozilla.org/en-US/firefox/addon/single-file/

To save entire page with images and videos as a single HTML file.

Use it this way:

- https://i.imgur.com/vFiPhFv.png

Not exactly what You needed - but still IMHO a step into right direction.

Regards,

vermaden

vermadenon Perfect NAS Solution

Lets assume You got 4TB 6TB 5TB and 8TB drives ... just do a 4TB 1st primary partition on each of them and create a ZFS pool from that - I do not understand where the 'pain' is ...

You can then add 1TB 2nd partition on 6TB and 5TB and 8TB drives and add another ZFS VDEV to the same pool - or create another pool.

... and You can also add 3rd partition.

Also its really easy to get used drives in the same or similar size ... unless you live in some really 'hard to live' part of the world.

vermadenon FreeBSD Desktop Series

Thanks, feel free to reach me out directly for any FreeBSD help.

Regards,

vermaden

vermadenon FreeBSD Desktop Series

Using FreeBSD and also Linux/AIX/... I would say FreeBSD is easier to use/learn.

Also - important stuff/commands/subsystems does not change on FreeBSD every 5 years.

With Linux for example things that changed:

- Init => systemd(1)

- ALSA => PulseAudio => PipeWire

- ifconfig(8) => ip(8)

- route(8) => ip(8)

- netstat(8) => ss(8)

- network-scripts => nmcli(8) [@RHEL/@Fedora/@CentOS/...]

- LILO => GRUB => NMBL

- X11 => Wayland

... and its not the end of these 'rewrites' unfortunately.

vermadenon FreeBSD Desktop Series

Check the + UPDATE(S) on the right.

Details here:

https://imgur.com/xUL9gQa

My 'fault' is that I do not add exact date of the update but ... this is the message I keep on my front page:

Keep in mind that I try to update articles posted here. I follow all supported FreeBSD versions – you can check which FreeBSD versions are currently supported on this page – FreeBSD Release Information: Currently Supported – from official FreeBSD website. If something gets out of date or works differently on newer versions – I add a needed UPDATE to a post to make it up to date.

Hope that helps.

Regards,

vermaden

vermadenon FreeBSD Desktop Series

I started in 2018 but I keep adding new articles and update the older ones with relevant data.

So it means - its ALL up-to-date.

vermadenon Perfect NAS Solution

I agree with You.

I believe there would be a 'follow up' to this article when 8 TB (or at least 6 TB) NVMe SSD modules will be available on 2280 and 2242 sizes :)

Then I would make ZFS mirror of them ... and hopefully with ECC RAM.

vermadenon Perfect NAS Solution

To be honest - no - I already have something 'big' - https://vermaden.wordpress.com/2023/07/30/amd-based-freebsd-... - described here in details - and needed something tiny :)

... but thank You for another point of view - these 'Celsius' machines does look interesting - need to spend some time to check them.

vermadenon Jart: FreeBSD kernel can natively run APE binaries now

Nice to see that :)

vermadenon Discord vs. IRC Rough Notes

Discord is more or less just IRC in the browser ...

vermadenon Change to FreeBSD release scheduling and support period

After using FreeBSD for so much time I can not get used to 'OpenBSD-current' naming :)

vermadenon Change to FreeBSD release scheduling and support period

It was always THIS naming scheme. Tradition.

- 14-CURRENT

- 14.1-STABLE

- 14.1-PRERELEASE

- 14.1-BETA1

- 14.1-BETA2

- 14.1-BETA3

- 14.1-BETAN

- 14.1-RC1

- 14.1-RC2

- 14.1-RC3

- 14.1-RCN

- 14.1-RELEASE

Details here:

- https://docs.freebsd.org/en/articles/freebsd-releng/

Hope that helps.

vermadenon The Cheapest NAS

On FreeBSD getting S.M.A.R.T. status over USB drive works without any problem:

    # smartctl -a /dev/da0
    smartctl 7.4 2023-08-01 r5530 [FreeBSD 13.2-RELEASE-p1 amd64] (local build)
    Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org
    
    === START OF INFORMATION SECTION ===
    Model Family:     Western Digital Elements / My Passport (USB, AF)
    Device Model:     WDC WD50NDZW-11A8JS1
    Serial Number:    WD-WXP2EA07UYLA
    LU WWN Device Id: 5 0014ee 2be7aef41
    Firmware Version: 01.01A01
    User Capacity:    5,000,947,523,584 bytes [5.00 TB]
    Sector Sizes:     512 bytes logical, 4096 bytes physical
    Rotation Rate:    5400 rpm
    Form Factor:      2.5 inches
    TRIM Command:     Available, deterministic
    Device is:        In smartctl database 7.3/5528
    ATA Version is:   ACS-3 T13/2161-D revision 5
    SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
    Local Time is:    Sat May 11 10:26:04 2024 CEST
    SMART support is: Available - device has SMART capability.
    SMART support is: Enabled
    
    === START OF READ SMART DATA SECTION ===
    SMART overall-health self-assessment test result: PASSED
    
    General SMART Values:
    Offline data collection status:  (0x00) Offline data collection activity
                                            was never started.
                                            Auto Offline Data Collection: Disabled.
    Self-test execution status:      (   0) The previous self-test routine completed
                                            without error or no self-test has ever 
                                            been run.
    Total time to complete Offline 
    data collection:                ( 4680) seconds.
    Offline data collection
    capabilities:                    (0x1b) SMART execute Offline immediate.
                                            Auto Offline data collection on/off support.
                                            Suspend Offline collection upon new
                                            command.
                                            Offline surface scan supported.
                                            Self-test supported.
                                            No Conveyance Self-test supported.
                                            No Selective Self-test supported.
    SMART capabilities:            (0x0003) Saves SMART data before entering
                                            power-saving mode.
                                            Supports SMART auto save timer.
    Error logging capability:        (0x01) Error logging supported.
                                            General Purpose Logging supported.
    Short self-test routine 
    recommended polling time:        (   2) minutes.
    Extended self-test routine
    recommended polling time:        ( 560) minutes.
    SCT capabilities:              (0x30b5) SCT Status supported.
                                            SCT Feature Control supported.
                                            SCT Data Table supported.
    
    SMART Attributes Data Structure revision number: 16
    Vendor Specific SMART Attributes with Thresholds:
    ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
      1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       0
      3 Spin_Up_Time            0x0027   253   253   021    Pre-fail  Always       -       3508
      4 Start_Stop_Count        0x0032   096   096   000    Old_age   Always       -       4626
      5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
      7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
      9 Power_On_Hours          0x0032   083   083   000    Old_age   Always       -       12789
     10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
     11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
     12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       139
    192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       58
    193 Load_Cycle_Count        0x0032   192   192   000    Old_age   Always       -       24665
    194 Temperature_Celsius     0x0022   086   083   000    Old_age   Always       -       66
    196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
    197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
    198 Offline_Uncorrectable   0x0030   100   253   000    Old_age   Offline      -       0
    199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
    200 Multi_Zone_Error_Rate   0x0008   100   253   000    Old_age   Offline      -       0
    
    SMART Error Log Version: 1
    No Errors Logged
    
    SMART Self-test log structure revision number 1
    No self-tests have been logged.  [To run self-tests, use: smartctl -t]
    
    Selective Self-tests/Logging not supported
    
    The above only provides legacy SMART information - try 'smartctl -x' for more

vermadenon The Cheapest NAS

Its very hard to beat Dell Wyse 3030 LT with USB attached disks.

- https://vermaden.wordpress.com/2023/04/10/silent-fanless-del...

I use that and the costs are about $20-25 for a used Dell Wyse 3030 and $60 for used 5TB 2.5 HDD from Seagate in USB 3.0 case.

Then the power bills will also be tiny as it draw about 3.8W when idle and 10.3 W with CPU and disks stressed to maximum.

The only limitation is 'only' 2GB RAM - but with ZFS ARC set to 32MB minimum and 64MB maximum RAM is not an issue.

    % grep arc /etc/sysctl.conf 
      vfs.zfs.arc.min=33554432
      vfs.zfs.arc.max=67108864

Regards.

vermadenon Cut 90% Costs by Migrating from Kubernetes in Cloud to FreeBSD on Premise

Your comment made my day :)

vermadenon When are SSDs slow?

Sometimes its the LEAST expected thing (that one this article does not cover).

Some time ago I had some really cheap SATA 2.5 SSD 256 GB in my laptop.

If generally worked well - system (FreeBSD) was responsive but ... when I deleted large file (like 1GB+ size) then the system almost FROZE ... until file is deleted.

After some investigation with Allan Jude (thanks again) we came to the conclusion that a single TRIM operation (for delete) take about 500ms (half a second) ...

By default ZFS on FreeBSD uses up to 64 TRIM operations PER SECOND (the vfs.zfs.vdev.trim_max_active setting) - so the SSD was literally DoS with TRIM operation and it crippled it to its knees.

After switching this option from 64 to 1 it solved all the problems and everything was smooth and fast again.

Details below.

    % grep -A 1 -i trim /etc/sysctl.conf
    # ZFS DELETE FUCKUP TRIM
    # vfs.zfs.vdev.trim_max_active: 64 -> 1
      vfs.zfs.vdev.trim_max_active=1

Hope that will help someone.

Regards,

vermaden

vermadenon Senior employees, ordered back to the office, are jumping ship

Very healthy attitude.

Would do exactly the same.

vermadenon ZFS Resilver on SMR Drives

Thank You.

... and yes - this is the problem with 'dedicated' SPARE drives - they just sit there unused and You never know if they will work when the time comes.

ZFS resolved that with DRAID where there are not 'dedicated' SPARE disks but 'spare block' spread across the disks:

- https://github.com/openzfs/zfs/wiki/dRAID-HOWTO/001b44728e1a...

vermadenon ZFS Resilver on SMR Drives

Yes, it will 'hurt' more to use SMR drives with raidz1/raidz2/raidz3.

... but I believe ZFS DRAID mode will not hurt that much on SMR drives.

In ZFS DRAID there is no dedicated SPARE drives - but 'spare blocks' are spread across the disks along with 'data blocks' and 'parity blocks' - this means that all drives 'read' for the resilver and all drives 'write' for the resilver.

More here:

- https://github.com/openzfs/zfs/wiki/dRAID-HOWTO/001b44728e1a...

... also the Sequential Resilver helps even more:

- https://docs.google.com/presentation/d/1vLsgQ1MaHlifw40C9R2s...

vermadenon ZFS Resilver on SMR Drives

To be honest I would not expect using SMR drives in some important production systems.

I would even prefer to use CMR ones ... but the 2.5 drives with 4-5 TB capacity are only available in SMR - maybe in the future when 2.5 drives will ofeer 8-10 TB of space - there would be a 4-5 TB CMD 2.5 drive.

vermadenon I Miss BSD/Linux

I had a ‘side’ journey to the Mac wonderland as I got opportunity to use Macbook Pro with Mac OS X Leopard for a year. That allowed me to get real ‘feel’ of the Mac ecosystem and their hardware (and philosophy) so I will not repeat same stereotypes over and over again like a lot of anti-apple people. But after I switched back to FreeBSD system at work it just felt better. I used Terminal.app on Mac a lot but the xterm(1) at FreeBSD just felt more natural.

More here with details:

https://vermaden.wordpress.com/2018/09/07/my-freebsd-story/

vermadenon Sys Admins and AI

You ask ChatGPT for example this:

show me config for Bareos server with bareos-dir and bareos-fd roles on FreeBSD system and bareos-sd on Linux (as this will be needed to use storage backend to put/get backups from S3 buckets)

... and after it replies you tailor the answer to your needs like:

- show me exact configs for everything

- we will use latest FreeBSD 14-STABLE with PkgBase instead

- we will use free and open source Bareos repo instead of paid one

Etc. Just one of thousands examples.

vermadenon Helldivers 2 PSN account linking update will not be moving forward

TL;DR?

vermadenon A BSD person tries Alpine Linux

The iXsystems do that in TrueNAS SCALE on Debian Linux.

Here:

- https://i.imgur.com/c4WUwb1.png

Unfortunately there is no command like bectl(8) or beadm(8) to manage that.

You have to do that from the WebUI:

- https://i.imgur.com/2JhZ6VB.png

Hope that helps.

vermadenon A BSD person tries Alpine Linux

I have seen what ZFSBootMenu can do and I really like it.

Its just 'crazy' to me that there are ZERO Linux distributions that install as root on ZFS with ZFSBootMenu and enabled ZFS Boot Environments.

None. Zero. Why?

vermadenon Intel used to dominate the U.S. chip industry, now struggling to stay relevant

Intel mostly dominated thank to the monopoly practices.

AMD was better in 2002-2004 on Opteron times ... but Intel just paid 'manufacturers' to use Intel instead in their offering.

Current AMD ZEN/Ryzen is just too strong to make that happen again.

I would say 'fuck intel' but I also like the fact that Intel was the first one to offer open source GPU drivers ...

vermadenon A BSD person tries Alpine Linux

> How baked in?

Like in FreeBSD - to be as available option in installer.

In FreeBSD You just select 'Auto (ZFS)' - then select disk(s) on which it should happen - hit [ENTER] - and it happens.

That level of integration I would expect ... and all that with ZFSBootMenu integration for ZFS Boot Environments.

vermadenon A BSD person tries Alpine Linux

I also liked how the '[ok]' moved to the right after GPU drivers have been loaded :)

- https://rubenerd.com/files/2024/alpine-openrc.png

Of course its OpenRC related and not Alpine.

IMHO it would looked more 'nice' if the '[ok]' stayed where they were - at least till 'login:' prompt - but I know that its just implemented in a way that 'write [ok] after space from right side of the screen' and as resolution increased - it happened what happened.

They could just move the '[ ok ]' or '[fail]' to the left - beginning of the message - that would 'solve' that problem also.