Hacker News Reader

Comment by hmottestad | Hacker News Reader

A good comparison. I couldn’t see that the author has created an issue on the golang GitHub. I assume that this would be considered a bug and that the golang community would be open to a fix.

View on HN

Or not, because it simplifies code, no differences between platforms. And if syscall overhead once per each GB is really a problem you should probably try mmap or some other IO style.

Mmap seems specific to POSIX according to this post[1], so it's not a portable way of using memory mapped files. Since there are already differences between platform capabilities in golang there really shouldn't need to be an artificial limit to how large of a chunk you can read in one operation. In 20 years I'm sure 1 GiB will be considered small.

1: https://stackoverflow.com/questions/52589949/undefined-sysca...

The author saw performance issues with this restriction, which is why they wrote the post.

I would be curious to see the numbers on that.

Hello, author here. Just to clarify, I wasn't saying that this has any perf impact per se. I was trying to see why syscall showed up quite a bit in the profile, when reading a large file, no matter what buffer size I pass in. Rather than spend time finishing the actual work, I spent way too much time trying to understand the reason for the limit.

You should probably ask the author then. There are a bunch of links to various socials on the bottom of the post.

Only that there is no official mmap support in go, due to gc concerns, afaik

Hm? https://pkg.go.dev/golang.org/x/sys/unix#Mmap

Reply on news.ycombinator.com