Hacker News Reader

Comment by SamuelAdams | Hacker News Reader

[flagged]

I think we all understand this to some degree, but working on open source, outside of a few flashy projects, is some of the most thankless work there is. And contributing an immense amount of difficult work (such as perf and correctness improvements across the board) to a repo that you don't own and won't be recognized for is somehow significantly more thankless than that. For whatever reason, people only really care about the creator of a project, and virtually no one else.

For instance, do you know who Junio Hamano is? Oh, he's just a guy who's been maintaining a fairly minor project called Git for the last 15 years. But everyone can connect Linus Torvalds with git, even though he only worked on it consistently for a year or two before leaving it [1].

Also, and I think we all know this too, but working on someone else's codebase kinda sucks. Greenfield is so much more fun. It's a shame, but I'm really not surprised in the slightest.

[1]: https://github.com/git/git/graphs/contributors

As an outsider, getting your code merged into a popular open source project involves a political process of convincing the maintainers that your fix should be addressed, and then convincing them they should merge your code.

Writing a fork involves sitting down at your laptop and coding it out.

Plus of course everything needs rewritten in rust /s.

  $ hyperfine -w 100 -m 1000 -L bin jq,jaq "echo '[1,2,3]' | {bin} '.[1]'"

  Summary
    echo '[1,2,3]' | jaq '.[1]' ran
      1.57 ± 0.15 times faster than echo '[1,2,3]' | jq '.[1]'

Bring on the competition!

As the benchmarks show, jaq is pretty significantly faster than jq.

I've commented before that I expect Rust to be a language that is generally faster than even C or C++ in a way that's hard to capture in small benchmarks, because the borrow checker permits code to be written safely that does less copying that other languages have to do for safety. Given the nature of what jq/jaq does, I wouldn't be surprised that that is some of the effect here. It would be interesting to instrument them up with tools that can track the amount of memory traffic each benchmark does to compare (that is, not memory used but total traffic in and out of RAM); I bet the Rust code shows a lot less.

FWIW, I see no difference. (hyperfine 1.17.0, jq 1.7, jaq 1.2.0)

  $ hyperfine -N -w 100 -m 1000 -L bin jq,jaq "echo '[1,2,3]' | {bin} '.[1]'"
  Benchmark 1: echo '[1,2,3]' | jq '.[1]'
    Time (mean ± σ):       3.4 ms ±   1.7 ms    [User: 0.6 ms, System: 2.6 ms]
    Range (min … max):     0.7 ms …   5.8 ms    1000 runs
 
  Benchmark 2: echo '[1,2,3]' | jaq '.[1]'
    Time (mean ± σ):       3.4 ms ±   1.7 ms    [User: 0.5 ms, System: 2.7 ms]
    Range (min … max):     0.7 ms …   5.8 ms    1000 runs
 
  Summary
    echo '[1,2,3]' | jq '.[1]' ran
      1.00 ± 0.71 times faster than echo '[1,2,3]' | jaq '.[1]'

That would still be a microbenchmark. Given that the benchmarks in the post take on the order of seconds to run, I am assuming they are not microbenchmarks, or at least, much less "micro"benchmarks. I would hope some sort of standard JSON querying benchmarking suite would include some substantial, hundreds-of-kilobyes or more JSON samples in it.

I'm pretty sure you could do this using hardware performance counters, but I never actually tried, so I might be wrong

not going to disagree

I think in this case it's for the completely reasonable reason that he wanted to write it in Rust and asking jq to rewrite their whole project in rust would be obnoxious.

Reply on news.ycombinator.com