Hacker News Reader

pixelbeat__on Uv is the best thing to happen to the Python ecosystem in a decade

It seems dotslash would complement uv well

https://dotslash-cli.com/docs/

DotSlash to get the interpreter for your platform, and uv to get the dependencies.

Perfect for corporate setups with custom mirrors etc.

pixelbeat__on The Linux Boot Process: From Power Button to Kernel

GRUB is mentioned but not detailed.

Here are some details: https://www.pixelbeat.org/docs/disk/

pixelbeat__on Date bug in Rust-based coreutils affects Ubuntu 25.10 automatic updates

https://github.com/coreutils/coreutils/commit/14d24f7a5

That bring GNU date(1) line coverage from 79.8% to 87.1%

pixelbeat__on Blockdiff: We built our own file format for VM disk snapshots

I see you use flags to determine if a file needs syncing. When we used fiemap within GNU cp we required FIEMAP_FLAG_SYNC to get robust operation.

(We have since removed the fiemap code from cp, and replaced it with LSEEK_DATA, LSEEK_HOLE)

pixelbeat__on The allure of new languages vs. the necessity of problem-solving

The benchmark is against unibyte text. You would get more accurate results by doing `export LC_ALL=C` in your benchmark script

pixelbeat__on Wttr: Console-oriented weather forecast service

Note the terminal -> HTML conversion used to serve wttr.in is based on https://github.com/pixelb/scripts/blob/master/scripts/ansi2h...

pixelbeat__on Dasel: Select, put and delete data from JSON, TOML, YAML, XML and CSV

A similar tool for ini files https://www.pixelbeat.org/programs/crudini/

pixelbeat__on The Elegance of the ASCII Table

I wrote about ASCII and UTF-8 elegance at:

https://www.pixelbeat.org/docs/utf8_programming.html

pixelbeat__on XZ backdoor: "It's RCE, not auth bypass, and gated/unreplayable."

rxzec

pixelbeat__on GNU Coreutils 9.5 Can Yield 10~20% Throughput Boost For cp, mv and cat Commands

The commit with more details on that perf change is: https://github.com/coreutils/coreutils/commit/fcfba90d0d27a1...

A summary of other changes just released in GNU coreutils 9.5 are:

- mv accepts --exchange to swap files

- env accepts --argv0 to override command name

- od supports half precision floating point formats

- timeout fixes various races

- chmod -R avoids symlink replacement attack

For all the release details see: https://lists.gnu.org/archive/html/coreutils-announce/2024-0...

pixelbeat__on Adventures of Linux Userspace at Meta [video]

(Hi Giuseppe!)

Right, the standard centos system binaries are used to provision services and manage hardware.

The internal services linking against the runtime libs you mention, are actually linking against about 2000 built from source packages, and are essentially a separate distro (with a distro in this sense being an ABI compatible set of libs running on a kernel)

pixelbeat__on Adventures of Linux Userspace at Meta [video]

Use ldd on any service binary to see that it's linked against a separate distro. (It's best to use the ldd from the corresponding platform).

There are actually three distros on each host. Current runtime platform, previous runtime platform, centos platform.

pixelbeat__on Adventures of Linux Userspace at Meta [video]

It's worth noting that this discusses the centos based provisioning and hardware management platform.

The actual distribution used to _run_ all Meta backend services is completely separate and built from source (it does share the (non centos) kernel). This is done for flexibility, performance, service isolation reasons

pixelbeat__on DaedalOS – Desktop environment in the browser

Reminds me of my website from 2006 (not entirely reproduced) at https://web.archive.org/web/20060208131821/http://www.pixelb...

pixelbeat__on A peculiarity of the GNU Coreutils version of 'test' and '['

Definitely not

pixelbeat__on Relative shell script includes with realpath on FreeBSD

All the main platforms have realpath(1). It was added to GNU Linux over 10 years ago now, to aid portability, and be a more natural command to access this functionality than readlink(1)

pixelbeat__on A peculiarity of the GNU Coreutils version of 'test' and '['

Yes, macos takes its utils from FreeBSD.

An interesting factoid is that FreeBSD/macos sort(1) was using GNU code until recently, since this is quite tricky to implement. Eventually it was reimplemented for GPL avoidance reasons.

We do consider macos though, and ensure all tests pass on macos for each release

pixelbeat__on Fdupes: Identify or delete duplicate files

czkawka is basically a rust port of my python FSlint tool, which I no longer have time to maintain: https://www.pixelbeat.org/fslint/

pixelbeat__on Base64 Encoding, Explained

Padding is only required if concatenating / streaming encoded data. I.e. when there are padding chars _within_ the encoded stream.

Padding chars at the end (of stream / file / string) can be inferred from the length already processed, and thus are not strictly necessary.

Note how padding is treated is quite subtle, and has resulted in interesting variations in handling as discussed at: https://eprint.iacr.org/2022/361.pdf

pixelbeat__on Decoded: GNU Coreutils (2018)

We generally give credit if at all possible. Even if a patch is extensively adjusted, we'll commit under the original author's name

pixelbeat__on Decoded: GNU Coreutils (2018)

I optimized yes because:

* The fast version is still simple enough

* The general functionality being provided is to output arbitrary data repeatedly, and it can be useful to do this as fast as possible

pixelbeat__on My favorite C compiler flags during development

The gnulib manywarnings module is useful to add as many warnings as appropriate in a portable manner

https://www.gnu.org/software/gnulib/manual/html_node/manywar...

That's used in dev mode for various projects like GNU coreutils etc.

When distros etc. are building though these warnings are not used. It's worth stating as I've seen this issue many times, that -Werror at least should not be used by default for third parties building your code

pixelbeat__on Strlcpy and Strlcat – Consistent, Safe, String Copy and Concatenation (1999) [pdf]

There have been various attempts to improve on strcpy() etc. The current thinking is that strscpy() is the best general interface to this functionality. See the "improving ... strcpy" section at

https://www.pixelbeat.org/programming/gcc/string_buffers.htm...

pixelbeat__on C Strings and my slow descent to madness

String handling in C has many gotchas indeed. Here are some of my notes on the subtleties:

https://www.pixelbeat.org/programming/gcc/string_buffers.htm...

pixelbeat__on What was the point of [ “x$var” = “xval” ]? (2021)

Oh I will reference this from https://www.pixelbeat.org/programming/shell_script_mistakes.... where I also advise against this common archaic idiom

pixelbeat__on Bash retry function with exponential backoff

This technique is useful for running things that normally run quickly, but sometimes need more time.

The technique is used extensively in the GNU coreutils test suite, as detailed in the "performance" section at https://www.pixelbeat.org/docs/coreutils-testing.html

pixelbeat__on Ask HN: What is the most impactful thing you've built?

You're not considering the scale.

We have faster iteration than upstream distros. More flexible. More tuned. I won't give exact details but 1% CPU gives extremely significant monetary savings, and there are at least 15% savings from static linking, PGO, LTO, more appropriate `-march`, more appropriate CPU security sharing considerations, ...

Billions of dollars per year, in essentially electricity and required systems savings (considering the scale of serving 3 billion users a day).

Also devs get access to the latest compilers, language levels, and libs, completely independent of distros, who have a more general compat issue to contend with. Considering there are about 30k tech in Meta this value also multiplys up.