HNHacker News
TopNewBestAskShowJobs

benhoyt

7,375 karma · joined February 21, 2007

Software engineer and manager at Canonical. Author of GoAWK, a POSIX-compatible AWK interpreter with CSV support (written in Go). Author of Python's os.scandir(). Husband, father, Christian. See my personal website: https://benhoyt.com/
submissionscomments
benhoyt··on Parse, Don't Validate (2019)
I'm not very familiar with functional programming and Haskell in particular. I think I understand the gist of this article, and "use data structures that make illegal states unrepresentable". However, is there a similar article but written with more common languages (C#, C++, Java, Go) in mind? Or is a big part of this concept only relevant for strong functional languages with sum types and pattern matching?
benhoyt··on Cursor's latest “browser experiment” implied success without evidence
Not me personally, but a GitHub user wrote a replacement for Go's regexp library that was "up to 3-3000x+ faster than stdlib": https://github.com/coregx/coregex ... at first I was impressed, so started testing it and reporting bugs, but as soon as I ran my own benchmarks, it all fell apart (https://github.com/coregx/coregex/issues/29). After some mostly-bot updates, that issue was closed. But someone else opened a very similar one recently (https://github.com/coregx/coregex/issues/79) -- same deal, "actually, it's slower than the stdlib in my tests". Basically AI slop with poor tests, poor benchmarks, and way oversold. How he's positioning these projects is the problematic bit, I reckon, not the use of AI.

Same user did a similar thing by creating an AWK interpreter written in Go using LLMs: https://github.com/kolkov/uawk -- as the creator of (I think?) the only AWK interpreter written in Go (https://github.com/benhoyt/goawk), I was curious. It turns out that if there's only one item in the training data (GoAWK), AI likes to copy and paste freely from the original. But again, it's poorly tested and poorly benchmarked.

I just don't see how one can get quality like this, without being realistic about code review, testing, and benchmarking.

benhoyt··on Super-Flat ASTs
That's a good caution. However, traversing a flat AST (iterating a "struct of arrays" rather than a pointer-based tree) is also going to be faster. So the next steps of the compiler, say type checking and code emitting, will also be faster. But how much, or whether it's worth it even then, I'm not sure.
benhoyt··on Archive or Delete?
I use the "baby bear" strategy mentioned in the article. My criteria are something like: archive emails from humans as well as important emails like receipts and invoices; delete advertising emails, newsletters, notification emails, and things that I can just as easily find online.
benhoyt··on Ask HN: What are you working on? (May 2025)
Yes, definitely still got/getting something out of it, thanks. And I'll probably get more out of it when I read up on "how to write a real chess program" for v2, and learn about all the things I didn't think about.
benhoyt··on Dependency injection frameworks add confusion
Yeah, I use this sometimes too (even though Python makes "monkey patching" easy). However, note that it's simpler and clearer to use a default value for the argument:

  def do_foo(sleep=time.sleep):
    for _ in range(10):
      sleep(1)
benhoyt··on Ask HN: What are you working on? (May 2025)
A program that will play chess (written in Go). My 18yo daughter can now beat me at chess (not that I'm any good). I figured if I can't beat her, I'll see if I can write a program to beat her instead. My idea for v1 is that I'd write the algorithm myself, without looking up anything about how to write a chess program (I'm sure such literature abounds). I've just about finished v1; still a few bugs to iron out. To be honest, I didn't find it all that fun, mainly because of all the special cases (all the castling rules and the like).
benhoyt··on Good Writing
> “The ships hung in the sky in much the same way that bricks don’t.”

That is probably my favourite phrase from the whole book. For some reason I find it hilarious. It has stuck in my brain in much the same way that names don't.

benhoyt··on The self-castrated hatmaker who killed John Wilkes Booth (2015)
The guy certainly had ba... uh, courage. Jesus is known for saying striking things, like "If your hand or your foot causes you to stumble, cut it off and throw it away. It is better for you to enter life maimed or crippled than to have two hands or two feet and be thrown into eternal fire." (Matthew 18:8) He was the master of attention-grabbing metaphor. Of course here he means "deal seriously with your sin", not "get out the butcher's knife".
benhoyt··on Ask HN: How do you make a living contributing to and/or creating OSS projects?
I do it by working for Canonical. The vast majority of our projects are open source. Yes, Canonical's application process is very long and thorough, but it's a good company to work for. Those aren't "my" projects, of course; I'm still very much working for an employer, but it's nice that our development effort is in the open and freely available. I also maintain a few of my own open source projects, for which (via GitHub Sponsors) I get about enough for one coffee date with my wife.
benhoyt··on Fly.io outage – resolved
Sure, it's UptimeRobot: https://uptimerobot.com/
benhoyt··on Fly.io outage – resolved
My fly.io-hosted website went down for 5 minutes (6 hours ago), but then came right back up, and has been up ever since. I use a free monitoring service that checks it every 5 minutes, so it's possible it missed another short bit of downtime. But fly.io has been pretty reliable overall for me!
benhoyt··on Go Turns 15
> It's all the design principles of Java that I don't like, but more of it.

I'm curious what you're referring to there. I think of Go as the "anti-Java". Where Java has "org.apache.commons.httpclient", Go has "net/http". Where Java has stuttering like "final Logger logger = LogManager.getLogger();", Go has "logger := log.New()" (and no log4j vulnerablity). A typical "simple" app in Java puts files in "src/com/username/simplewebapp/subdir", typical Go project structure is to put files in just "subdir", or even just in the root dir. And so on.

benhoyt··on Sqlc: Compile SQL to type-safe code
I missed this too. However, I've found you can work around it pretty easily with clauses like CASE WHEN @field != "" THEN column = @field ELSE true END.

Example from the sqlc creator (https://github.com/sqlc-dev/sqlc/discussions/364#discussionc...):

  -- name: FilterFoo :many
  SELECT * FROM foo
  WHERE fk = @fk
    AND (CASE WHEN @is_bar::bool THEN bar = @bar ELSE TRUE END)
    AND (CASE WHEN @lk_bar::bool THEN bar LIKE @bar ELSE TRUE END)
    AND (CASE WHEN @is_baz::bool THEN baz = @baz ELSE TRUE END)
    AND (CASE WHEN @lk_baz::bool THEN baz LIKE @baz ELSE TRUE END)
  ORDER BY
    CASE WHEN @bar_asc::bool THEN bar END asc,
    CASE WHEN @bar_desc::bool THEN bar END desc,
    CASE WHEN @baz_asc::bool THEN baz END asc,
    CASE WHEN @baz_desc::bool THEN baz END desc;
benhoyt··on Parsing Awk Is Tricky
Brian Kernighan sent Gawk maintainer Arnold Robbins an email linking to this blog post with the comment "Hindsight has a lot of benefits, it would appear."

Peter Weinberger (quoted with permission) responded:

> That's interesting, Here's some thoughts/recollections. (remember that human memory is fallible.)

> 1. Using whitespace for string concatenation, in retrospect, was probably not the ideal choice (but '+' would not have worked).

> 2. Syntax choices were in part driven by the desire for our local C programmers to find it familiar.

> 3. As creatures of a specific time and place awk shared with C the (then endearing, now irritating) property of being underspecified.

> I think that collectively we understood YACC reasonably well. We tortured the grammar until the parser came close to doing what we wanted, and then we stopped. The tools then were more primitive, but they did fit in 64K of memory.

Al Aho also replied (quoted with permission):

> Peter's observation about torturing the grammar is apt! As awk grew in its early years, the grammar evolved with it and I remember agonizing to make changes to the grammar to keep it under control (understanding and minimizing the number of yacc-generated parsing-action conflicts) as awk evolved. I found yacc's ability to point out parsing-action conflicts very helpful during awk's development. Good grammar design was very much an art in those days (maybe even today).

It's fun to hear the perspectives of the original AWK creators. I've had some correspondence with Kernighan and Weinberger before, but I think that's the first time I've been on an email thread with all three of A, W, and K.

benhoyt··on How to think in writing
I just finished "On Writing Well" by William Zinsser. Here's a little blurb I just wrote about it:

What a great book! Zinsser uses such crisp, engaging prose. He covers how to write, why to write, and what to write. He uses lots of examples from his own writing and from others -- good and bad. Stories abound. This is not a boring grammar book.

Here's a taste -- he starts the chapter called "The Lead and the Ending" with these stark statements: "The most important sentence in any article is the first one. If it doesn't induce the reader to proceed to the second sentence, your article is dead." Punchy and self-referential.

There's a decent amount about good technique, but it's more about style and voice. The second half of the book covers topics like "Writing in Your Job" (without sounding passive or using buzzwords) and "Writing Family History and Memoir" (that's actually why my brother lent me the book). And it finishes with an inspiring chapter on the craft of writing called "Write as Well as You Can".

If you want to learn to write better, read this book. If you don't, definitely read it.

benhoyt··on Th64: Tiny Hash Function in C
I could be wrong, but I think it just goes over the data once. The first loop increments i by 8, so is processing 64 bits at a time. It looks like the second loop only does the remaining few bytes, if any.
benhoyt··on Starlark Language
Oops, yes. I'd better not post that yet. Thanks. :-)
benhoyt··on Starlark Language
[deleted]
benhoyt··on Scientists should use AI as a tool, not an oracle
> People should use AI as a tool, not an oracle

There, fixed the title.

benhoyt··on [dead]
I love free books, but does the website owner have permission from the author to post this there? The copyright notice says, "Copyright © 2018 John K. Ousterhout. All rights reserved. No part of this book may be reproduced, in any form or by any means, without permission in writing from the author."

As far as this book goes, when I read it a few years ago I kind of appreciated it, but found it a bit too abstract and philosophical to be useful.

benhoyt··on Ok Considered Harmful
I don't find the fixed code "better in every way". The ultra-long names are hard to read, partly because they take up a lot of space and move the important part -- the GetTable() call -- further right:

  var err error
  var leftSideTableExists
  tm.leftTbl, leftSideTableExists, err = rm.left.GetTable(...)
  if err != nil { ... }
  if leftSideTableExists { ... }
Plus, the code explicitly defines the variables with "var" instead of ":=", making it more verbose, and it overwrites tm.leftTbl even if GetTable returns an error (doesn't matter in this case as tm is locally-created, but is often not desired). I'd probably just call it "exists", but if you do want to distinguish left-exists and right-exists, I'd use "leftExists"; "side" and "table" are obvious from the context:

  left, leftExists, err := rm.left.GetTable(...)
  if err != nil { ... }
  if leftExists { ... }
  tm.leftTbl = left
benhoyt··on No Abstractions: our API design principle
From their docs [1] it looks like they do everything using integers: the amounts are integers in the "minor unit" of currency, for example cents if the currency is dollars. So 1000 means $10.00. In languages like JavaScript where everything is a float64, you can still accurately represent integers up to 2^53, which would be $90 trillion.

[1] https://increase.com/documentation/api#transactions

benhoyt··on Go performance from version 1.0 to 1.22
I think those are worthwhile criticisms ... right up until you said "Ruby or Python win over Go by any metric except speed".

As someone who has written a ton of Python and a ton of Go (and likes both languages), I still prefer Python for small scripts due to its terseness and beautiful syntax.

Side note: Go's static typing is so much better than Python's type annotations. Python typing is difficult to use and a mess: it's constantly changing ("improving"), Pyright and MyPy never agree, Pyright doesn't agree with itself between versions, and much more. The bolted-on, late-in-the-game nature of Python's typing really shows.

One place where I have a real side-by-side comparison of the two languages solving the same problem is: I wrote a streaming websocket client with both Go and Python. Go's was so much easier to reason about due to the simplicity and ubiquity of io.Reader and io.Writer. Compare that to the mess that is Python "file-like objects" (https://docs.python.org/3/library/io.html). Should I use BufferedIOBase, or TextIO, or maybe RawIOBase, and what do I override in my subclass? It was quite painful.

My take is that, in addition to its speed, Go wins on its type system, I/O interfaces, concurrency, package management, and test/benchmark tooling. Python wins in its beautiful syntax, [list|dict|set|generator] comprehensions, and quantity of 3rd party libraries.

benhoyt··on An Awk Implementation in C99
I've chatted with Ray (the author of wak -- love the name) a few times about our respective AWK implementations. He's a nice guy and a really thoughtful programmer, and he even found several edge-case bugs in my GoAWK implementation: https://github.com/search?q=repo%3Abenhoyt%2Fgoawk+raygard&t...
benhoyt··on An Awk Implementation in C99
It's a second edition, not a new book -- why did you feel deceived? There's a fair bit of restructuring of the first few chapters, and a whole new chapter on exploratory data analysis (which uses the new CSV support) and a bunch of new stuff about the new Unicode-character support.
benhoyt··on 1BRC merykitty's magic SWAR: 8 lines of code explained in 3k words
Local disk I/O is no longer the bottleneck on modern systems: https://benhoyt.com/writings/io-is-no-longer-the-bottleneck/

In addition, the official 1BRC explicitly evaluated results on a RAM disk to avoid I/O speed entirely: https://github.com/gunnarmorling/1brc?tab=readme-ov-file#eva... "Programs are run from a RAM disk (i.o. the IO overhead for loading the file from disk is not relevant)"

benhoyt··on The One Billion Row Challenge in Go: from 1m45s to 4s in nine solutions
I think that's true with the JVM, but the fastest Java solutions are using GraalVM and its ahead-of-time compilation mode to avoid startup time. In addition, while "go run" might take 149ms to compile and run a program, a compiled Go program starts in just a couple of milliseconds:

  $ time ./t
  Hello, world
  
  real  0m0.003s
benhoyt··on The One Billion Row Challenge in Go: from 1m45s to 4s in nine solutions
Using Go's profiling tool with its "source" view: I used "go tool pprof -http=: cpu.prof", where cpu.prof was generated by "go-1brc -cpuprofile=cpu.prof -revision=9 measurements.txt".
benhoyt··on The One Billion Row Challenge in Go: from 1m45s to 4s in nine solutions
For what it's worth, on my machine the simple/idiomatic "baseline" Java solution (https://github.com/gunnarmorling/1brc/blob/main/src/main/jav...) takes 2m0s, compared to 1m45s for my simple Go version. So Go is a bit better for the naive version.
Page 1 of 27Next →