Hacker News Reader

Comment by brundolf | Hacker News Reader

Lots of people are writing Rust code who aren't previously versed in the nuances of optimizing bare-metal code. They're (reasonably) expecting amazing performance, because Rust tends to give you that by default compared with higher-level languages. But repeated calls to println!() is one of the biggest and easiest-to-make performance mistakes for those newcomers, so I think it's fair to call it out.

With that said, the OP could definitely have done a better job making it clear that this problem isn't specific to Rust

View on HN

For what it's worth, I run into the same nuances frequently in Python. For example I often want to run

    some_script.py | tee log.txt

But by default, Python will turn off line buffering when it detects it's not talking to a tty. So unless you know to call flush, or to add the right env var or interpreter flag, you're not going to see any early output from tee. This can be super confusing for newer programmers.

It's not Python, it's a generic Unix problem. This is why stdbuf exists.

http://www.pixelbeat.org/programming/stdio_buffering/

> Python will turn off line buffering when it detects it's not talking to a tty

Okay, I’ve been running into a bug in one of my Rust programs and I think this comment has the answer. My program uses a Command [0] to listen to stdin and stdout. Theoretically that Command should terminate when the parent process dies, but if the Command is a Python script that doesn’t include a `print()` with `flush=True`, it doesn’t die. I think you might have found my problem for me.

[0]: https://docs.rs/tokio/1.15.0/tokio/process/struct.Command.ht...

It sounds to me like println! and other I/O macros and functions should buffer I/O by default if the file descriptor is not a terminal. This would be consistent with how other languages’ standard libraries behave.

Developers from other languages, like C, C++, or even Python, would not have expected this particular problem, because it wouldn't have occurred to them that the thing they get from doing a file-open call and that the JSON reader library accepts is a raw file descriptor (for C programmers, the result of 'open' rather than 'fopen') that does unbuffered I/O. In other languages libraries are oriented around buffered I/O structures (a FILE or an iostream).

To me the biggest reason for the surprise is the signature for 'write'

I'm used to a raw stream allowing a few specific operations:

- read/write/peek one byte - write a byte array, but you must explicitly specify length and offset

So seeing a call that simply takes an entire byte string and writes it gives me "buffered I/O vibes". It's hard to quantify, but with how ubiquitous that pattern is I'd be surprised if there are a lot of people who expect similar.

This had been upvoted and now it is downvoted, even though it is obviously true, as the article demonstrates.

I don't think it's fair to call out since no one is putting a println! in hot loops unless they're debugging

Not remotely true. I learned this lesson the hard way a couple years ago while working on a program that did exactly that.

Why were you calling println in a hot loop instead of using a logging library?

Because it was a simple CLI tool that spit out a bunch of output as part of its core functionality.

Reply on news.ycombinator.com