HNHacker News
TopNewBestAskShowJobs

orlp

8,833 karma · joined March 27, 2013

Developer at https://pola.rs/.

Publish a blog at https://orlp.net/blog/.

Other socials:

    http://github.com/orlp/  
    https://stackoverflow.com/users/565635/orlp  
    https://linkedin.com/in/orson-peters/
submissionscomments
orlp··on Amazon confirms 14,000 job losses in corporate division
Passive voice deflects responsibility and agency.

Loss happens, firings are a decision.

orlp··on Valorant's 128-Tick Servers (2020)
No, OSRS is 100 ticks per minute which gives 0.6 second ticks, which rounds to 1.667 ticks per second.
orlp··on SedonaDB: A new geospatial DataFrame library written in Rust
I'm working on implementing extension types in Polars. Stay tuned.
orlp··on From Rust to reality: The hidden journey of fetch_max
Aarch64 does indeed have a proper atomic max, but even on x86-64 you can get a wait-free atomic max as long as you only need to support integers up to 64. In that case you can simply do a `lock or` with 1 << i as your maximum. You can even support larger sizes by using multiple registers, e.g. four 64-bit registers for a u8 maximum.

In most cases it's even better to just store a maximum per thread separately and loop over all threads once to compute the current maximum if you really need it.

orlp··on Pointer Tagging in C++: The Art of Packing Bits into a Pointer
I take it you never wrote code involving atomic pointers. Regardless of memory usage, a lot of platforms only provide single-word atomics (efficiently), making bitpacking crucial for lockfree algorithms.
orlp··on Pointer Tagging in C++: The Art of Packing Bits into a Pointer
> Reliable pointer tagging is not trivial.

It is if you use alignment bits. Not always possible if you don't control the data though.

orlp··on Determination of the fifth Busy Beaver value
Yes. But there is no decider for n-state Turing machines that works regardless of n.
orlp··on Hashed sorting is typically faster than hash tables
Well, years back I released an unstable sort called pdqsort in C++. Then stjepang ported it to the Rust standard library. So at first... nothing. Someone else did it.

A couple years later I was doing my PhD and I spent a lot of time optimizing a stable sort called glidesort. Around the same time Lukas Bergdoll started work on their own and started providing candidate PRs to improve the standard library sort. I reached out to him and we agreed to collaborate instead of compete, and it ended up working out nicely I'd say.

Ultimately I like tinkering with things and making them fast. I actually really like reinventing the wheel, find out why it has the shape that it does, and see if there's anything left to improve.

But it feels a bit sad to do all that work only for it to disappear into the void. It makes me the happiest if people actually use the things I build, and there's no broader path to getting things in people's hands than if it powers the standard library.

orlp··on Hashed sorting is typically faster than hash tables
I (Orson Peters) am also here if anyone has any questions :)
orlp··on ICPC 2025 World Finals Results
As I mentioned in my other comment:

> I'd just like to clarify that I'm not saying this is necessarily the solution the problem writers were looking for, or that it will run within the allocated time. Just that it's a feasible solution.

I don't doubt there's a clever dedicated flow algorithm the problem writers intended instead of the blunt tool which is LP.

orlp··on IRHash: Efficient Multi-Language Compiler Caching by IR-Level Hashing
Every developer I've talked to has had the same experience with compilation caches as me: they're great. Until one day you waste a couple hours of your time chasing a bug caused by a stale cache. From that point on your trust is shattered, and there's always a little voice in the back of your head when debugging something which says "could this be caused by a stale cache?". And you turn it off again for peace of mind.
orlp··on ICPC 2025 World Finals Results
The team manual I referred to when I was in university does in fact contain such a basic simplex LP solver: https://github.com/ludopulles/tcr/blob/master/tcr.pdf (page 22).

I'd just like to clarify that I'm not saying this is necessarily the solution the problem writers were looking for, or that it will run within the allocated time. Just that it's a feasible solution.

orlp··on Polars Cloud and Distributed Polars now available
With all due respect, have you actually used the Polars expression API? We actually strive for composability of simple functions over dedicated methods with tons of options, where possible.

The original comment I responded to was confusing Pandas with Polars, and now your blog post refers to Numpy, but Polars takes a completely different approach to dataframes/data processing than either of these tools.

orlp··on ICPC 2025 World Finals Results
It doesn't seem that hard to solve to me either. It's solvable with basic linear programming.

    1. Add a variable for each node, and a variable for each output edge from stations.
    2. For each reservoir add equality constraints to the sum of incoming edges with the coefficients given in the problem.
    3. For each station add equality constraints between the weighted sum of its inputs (which is 1 for the root station) and its outputs (which are the variables we added).
    4. Add an out_edge >= 0 constraint for each output edge on stations to forbid illegal negative flows.
    5. Add a variable m which is constrained to be less than all the output station variables.
    6. Maximize m.
orlp··on Polars Cloud and Distributed Polars now available
> The creator of duckdb argues that people using pandas are missing out of the 50 years of progress in database research, in the first 5 minutes of his talk here.

That's pandas. Polars builds on much of the same 50 years of progress in database research by offering a lazy DataFrame API which does query optimization, morsel-based columnar execution, predicate pushdown into file I/O, etc, etc.

Disclaimer: I work for Polars on said query execution.

orlp··on I Was Wrong About Data Center Water Consumption
> The width increase (10-100x) completely overwhelms the depth increase (maybe 3-10x), so surface area increases substantially.

No it doesn't.

The only thing that matters (in this oversimplified calculation which only takes into account surface area) is average depth of the freshwater while it is on land. If the reservoir is on average deeper than the rivers the freshwater otherwise would be flowing in, there will be less evaporation per liter of freshwater available for use.

Now a dam also increases the total amount of freshwater that's kept on the land in a steady state situation compared to if the water flowed free into the sea. It would be absurd to count this as "extra evaporation" when this extra freshwater otherwise would've simply be lost when it would flow into the sea instead of being kept in the reservoir.

orlp··on I Was Wrong About Data Center Water Consumption
> On the one hand, a huge dam reservoir does increase the level of water evaporation relative to an undammed river by increasing the amount of water surface area.

That depends entirely on the depth of the river and the depth of the reservoir. If the average depth of the reservoir is deeper than the average depth of a river there is less surface area.

orlp··on Tesla said it didn't have key data in a fatal crash, then a hacker found it
The marketing doesn't even matter. It either needs to be full self driving, or nothing at all. The "semi self-driving but you're still responsible when shit hits the fan" just doesn't work.

Humans are simply incapable of paying attention to a task for long periods if it doesn't involve some kind of interactive feedback. You can't ask someone to watch paint dry while simultaneously expect them to have < 0.5sec reaction time to a sudden impulse three hours into the drying process.

orlp··on Uncertain<T>
Not sure why this is being upvoted as the article is not describing interval arithmetic. It supports all kinds of uncertainty distributions.
orlp··on God created the real numbers
Actually in math it's very common for the more general system to be simpler. Compare for example the prime numbers with the integers, or general groups with finite simple groups and the monster group.
orlp··on macOS dotfiles should not go in –/Library/Application Support
I assume they've made up their mind and are now just tired of discussing it. I don't know why they refuse to even consider an option for it.
orlp··on macOS dotfiles should not go in –/Library/Application Support
I and others have brought this up with the dirs Rust crate maintainer but they refuse to see it this way: https://codeberg.org/dirs/dirs-rs/issues/64. It's very frustrating.

I now use a combination of xdg + known-folders manually:

    [target.'cfg(windows)'.dependencies]
    known-folders = "1.2.0"

    [target.'cfg(not(windows))'.dependencies]
    xdg = "2.5.2"
to get the config directory:

    use anyhow::{Context, Result};

    #[cfg(windows)]
    fn get_config_base_dir() -> Result<PathBuf> {
        use known_folders::{KnownFolder, get_known_folder_path};
        get_known_folder_path(KnownFolder::RoamingAppData).context("unable to get config dir")
    }

    #[cfg(not(windows))]
    fn get_config_base_dir() -> Result<PathBuf> {
        let base_dirs = xdg::BaseDirectories::new().context("unable to get config dir")?;
        Ok(base_dirs.get_config_home())
    }
orlp··on A German ISP changed their DNS to block my website
Are you using a third-party DNS like 1.1.1.1 or 8.8.8.8?
orlp··on 4chan will refuse to pay daily online safety fines, lawyer tells BBC
> will solve 90% of the problem

Remind me again, what the problem they're trying to solve is?

orlp··on Going faster than memcpy
That is something I can agree with, but I can't in good faith just let "it's just a hint, they don't have anything to do with correctness" stand unchallenged.
orlp··on Going faster than memcpy
> Non-temporal instructions don't have anything to do with correctness. They are for cache management; a non-temporal write is a hint to the cache system that you don't expect to read this data (well, address) back soon

I disagree with this statement (taken at face value, I don't necessarily agree with the wording in the OP either). Non-temporal instructions are unordered with respect to normal memory operations, so without a _mm_sfence() after doing your non-temporal writes you're going to get nasty hardware UB.

orlp··on Rotring 600 Ballpoint Pen
My favorite pens are the Frixxion pens. You can erase them, and it actually works well.
orlp··on Itch.io: Update on NSFW Content
This is useless. You can't stop Collective Shout (their campaign almost surely falls under First Amendment rights), and even if you could, 30 minutes later a new group pops up. Plus your message would fall completely on deaf ears for anyone who agrees with Collective Shout.

Bring attention to the fact that payment processors are acting as active censorship of legal content, rather than neutral infrastructure. Emphasize that if they can censor legal content, anything could be next, including but not limited to political donations of a specific party.

orlp··on Top DNS domains seen on the Quad9 recursive resolver array each day
> Wow, that's smart. I was wondering whether there is a way for the bots to generate "unpredictable" domains such that security researchers could not predict them efficiently (even with source code), but the botnet controller can.

There is a fairly simple method which achieves the same advantage for a botnet controller.

1. Use a hash of the current day to derive, for that day, an infinite stream of domain names. This could be something as simple as `to_human_readable_domain(sha256(daily_hash + i))`.

2. A botnet slave attempts to access servers in a diagonal order over (days, domains), starting at the first domain for today and working backwards in days and forwards in domains. An image best describes what I mean by this: https://i.imgur.com/lcEbHwz.png

3. So long as one of those domains is controlled by the botnet operator (which can be verified using a signed response from the server), they can control the botnet.

This means that the botnet operator only needs to purchase one domain every couple of days to keep controlling their botnet, while someone trying to stop them will have to buy thousands and thousands every day.

And when you successfully purchase a domain you can publish the new domain to any connected slaves, so this scheme is only necessary for recruitment into the network, not continued control.

orlp··on FP8 is ~100 tflops faster when the kernel name has "cutlass" in it
Intel's C++ compiler is known to add branches in its generated code checking if the CPU is "GenuineIntel" and if not use a worse routine: https://en.wikipedia.org/wiki/Intel_C%2B%2B_Compiler#Support....
← PreviousPage 3 of 23Next →