Hacker News Reader

wmanleyon Akin’s Laws of Spacecraft Design

I think you've misunderstood the quote. The solution that the great engineer produces is effective, but may not be efficient nor elegant if it doesn't need to be. The point is that solving the problem (and maybe stopping there) is more important than producing something that is fast or beautiful that doesn't solve the problem.

I think there's some overlap with:

> 13. Design is based on requirements. There's no justification for designing something one bit "better" than the requirements dictate.

wmanleyon Far Cry 1.34 source code (2006)

Here it is:

https://github.com/id-Software/Quake-2/blob/372afde46e7defc9...

wmanleyon Far Cry 1.34 source code (2006)

I don’t know about quake, but in quake 2 it consisted of a big list of:

    else if (strcmp(cmd, “somecmd”) == 0)
        somecmd()

I remember reading it at the time and being inspired by the simplicity of it. It didn’t need to be more complicated than it was, and so Carmack didn’t make it more complicated. That was in stark contrast to a lot of the object oriented c++ design stuff that I was also reading at the time.

So much of the quake 2 game source was like that. Beautifully straightforward code.

wmanleyon Time is not a synchronization primitive

> It's a weird example, because it moves listener creation elsewhere, which is not always possible (e.g. consider if the goroutine would be spawning an external program instead of setting up the listener of its own).

Just addressing the case of spawning an external program: if you have control of the external program you’re spawning it can be better to pass the listening socket in using the systemd $LISTEN_PID protocol. It’s easy too implement, language independent and you get systemd integration for free.

wmanleyon A LiveView Is a Process

> Websockets require a central broker to pass messages through when you scale past one process

I believe that this is one of the big advantages of SSE vs. WebSockets. Unlike websockets, an SSE connection is just an HTTP request, just one that has a particular format and is conventionally long-lived. You don't need a broker and you can probably implement it yourself on top of whatever HTTP server library that you're already using. And it already contains functionality for dealing with network unreliability [1].

I've often thought that it would be really good to have better support for SSE in HTTP caching HTTP proxies. Then you could have nginx for example sitting infront of your SSE endpoints serving multiple clients with a single SSE stream to your backend. Just based on the caching headers and a little knowledge of `Last-Event-ID`.

[1]: You can encode any state you require to resume the connection into Last-Event-ID.

wmanleyon Let us serve you, but don't bring us down

The difference between this use case and bitcoin is algorithm mobility. All bitcoin miners must use the same algorithm and get the same result for it to work. This makes it very hard (read slow) to change the algorithm, so there's time and incentive to design, fabricate and deploy ASICs. For client-server PoW the server can choose a new algorithm whenever the old algorithm becomes ineffective.

wmanleyon Database branching: three-way merge for schema changes

wmanleyon How to be a -10x Engineer

I see what you mean: https://regmedia.co.uk/2021/01/05/simple_sabotage_field_manu...

wmanleyon 15k inserts/s with Rust and SQLite (2021)

AFAICS you're doing all your inserts in a single transaction, unlike the article. Using transactions is better for performance, but probably less representative in this specific case.

wmanleyon Europe pushing for lunar time zone

and the minute hand?

wmanleyon Magic the Gathering players lament the loss of 30-year history

Or make plans to anger them in ways that are useful to the gaming company.

As you say: hating invigorates a community

wmanleyon New make --shuffle mode

Very neat.

I'd really like something like this for systemd units. On embedded systems you want to have confidence that any permitted ordering will result in a working system - as there'll be no-one there to restart it if not.

wmanleyon “Expect tests” make test-writing feel like a REPL session

Was discussed recently here: https://news.ycombinator.com/item?id=34350749

wmanleyon What if writing tests was a joyful experience?

Guidelines for tests that reduce maintenance burden rather than add to it:

* Test the externally visible useful properties of your system, rather than implementation details. Related: don’t bother with mocks unless there’s no other way

* Keep your tests DRY: if there’s some invariant you want to check have it appear once in your test code

* Don’t over-test: you can delete tests and checks that fully overlap with other tests

* If you need to over-test make it easy to regenerate the expected results from your implementation (characterisation tests) as espoused by the article. * the amount of testing effort should be proportional to how important the functionality is. Reduce the effort for low impact features

wmanleyon What if writing tests was a joyful experience?

I am of the opinion that the true value of TDD is that tests get written at all. I suspect that in practice the tests tend to get written at roughly the same time as the code, rather than strictly before.

wmanleyon What if writing tests was a joyful experience?

On a related note: I think that deterministic generating something based on a git commit and then later diffing the results across commits is in general a very useful technique. Our builds work similarly. We build our software into (several) container/disk images, then we show the diff between these build artefacts on PRs.

It can help you spot when something changes that you didn’t expect, and vice-versa.

For this to work we store all our builds indexed by git commit sha.

wmanleyon What if writing tests was a joyful experience?

IMO The real value in tests is making sure you don’t break things later, rather than for getting your current code working.

wmanleyon What if writing tests was a joyful experience?

There were other advantages to our new approach too: because all the running of the code is happening server side it is both easier and faster.

Easier because you don’t need to have a complete development environment available on the machine that you’re making the change. You can make a quick change in the GitHub web ui on mobile and see the results. This is particularly important in our case as we have some code that depends on specific hardware to run (CUDA), that isn’t just available on any developer machine.

Faster because the server can run the process in parallel across a farm of machines, which will be faster than running locally.

wmanleyon What if writing tests was a joyful experience?

We used to use a similar approach for testing page objects in stb-tester. It was called stbt auto-selftest (it has since been removed). It would run your page objects against canned screenshots and generate a Python doctest capturing the behaviour of this page objects. You’d then commit the doctest to git and have ci set up to check correctness. See https://david.rothlis.net/pageobjects/characterisation-tests

This was really helpful - we could then see in or pull requests how changes to our page object code would affect the behaviour of that code - just look at the diffs.

It worked ok for a while, but we soon hit limitations with this approach. The big one is the interaction with git. We would get many merge conflicts in these files, particularly when rebasing to reorder commits. Sometimes we’d forget to regenerate the files and have to rerun and rebase after submitting to ci. It worked okish for us as we are sophisticated git users comfortable with rebase and filter-branch, and we’d designed the system so could debug issues effectively, but it wasn’t suitable for wider use.

We’ve moved away from this model to a slightly different one. We still generate this data[^1], but we store the data outside of git in a database indexed by git commit sha. We then display this data and associated diffs in a web-ui. We use GitHub checks to wire it back up to pull requests, so it’s just as visible as when it was committed to git, but more convenient and displayed in a nicer format. You can use the same web-ui to approve changes to this data.

[^1]: though in more structured form than doctests

wmanleyon Ccache – a fast C/C++ compiler cache

I wrote a little bit about this a couple of years ago: https://blog.williammanley.net/2020/05/25/unlock-software-fr...

gittup[1] implements part of this idea, but without the distributed bit. bazel[2] implements the distributed bit (minus trust), but not the distro bit. What's really lacking is momentum around the idea to get a sufficient number of people behind it.

[1]: https://github.com/gittup/gittup [2]: https://bazel.build/

wmanleyon Is Rust stack-efficient yet?

Easier said than done. You can already have:

    unsafe fn new_in_place(out: &mut MaybeUninit<MyType>)

But you require unsafe code both to implement it (to project MaybeUninit) and to call it.

For it to be safe to call the type system would have to encode the invariant that &out is initialised after the call. It becomes even more complicated if the operation can fail.

wmanleyon Is Rust stack-efficient yet?

There's some ugly hacks that may or may not help here:

https://www.reddit.com/r/rust/comments/xxhp3s/perhaps_not_a_...

    r#box!(make_my_elem())

This crate is marked as deprecated because apparently upstream rust optimises its use-case now, but you never know:

https://github.com/kvark/copyless

    Box::alloc().init(make_my_elem())

wmanleyon D3wasm 0.4 – Doom 3 in WASM

I don't think it is the future - but not because of the technology. Things in the browser have proven hard to monetize, unlike app stores/steam/consoles which have been built to enable selling.

wmanleyon D3wasm 0.4 – Doom 3 in WASM

Something I loved about that game was that stopping didn't make you safe - if you stopped for too long the aliens would come and find you. This lead to a intense experience where you had to keep going - even when terrified - because stopping was even scarier.

wmanleyon Think Prometheus, but for logs (not metrics). Simple, efficient, fast log store

I really like the idea of storing logs in parquet format in S3. It's something I've wanted to play with for a long time, but never have had a use-case. I'm also a big fan of Rust.

I imagine you've seen apenwarr's blog post on logging: https://apenwarr.ca/log/20190216

With that in mind I'd like to read more about the architecture. Specifically:

1. Do you have separate components for ingest and querying that can be scaled independently?

2. How dumb is the log ingest path?

wmanleyon Think Prometheus, but for logs (not metrics). Simple, efficient, fast log store

Are you using S3 Select[1]?

[1]: https://aws.amazon.com/blogs/aws/s3-glacier-select/

wmanleyon Let’s make WordPress officially support SQLite

I copy-pasted the quote from "Contributed Code" on that page.

wmanleyon Let’s make WordPress officially support SQLite

SQLite doesn't accept patches:

> In order to keep SQLite completely free and unencumbered by copyright, the project does not accept patches.

https://www.sqlite.org/copyright.html

wmanleyon Unreal Rust

> in rust, bounds checks are only available on debug builds

This is not true - bounds checks are always enforced: https://play.rust-lang.org/?version=stable&mode=release&edit...

Maybe you're confusing it with signed overflow behaviour in Rust or bounds checks in Zig?

wmanleyon Ruff – a fast Python Linter written in Rust

I've not looked at the implementation details of Ruff, but I am of the opinion that lint tools should be based on an architecture like GitHub's stack graphs[1]. This is the technology that powers GitHub's code navigation.

Specifically: analysis is broken up into two phases:

1. one that only depends on the content of a file, and 2. a second one which deals with inter-file dependencies.

The advantage is that you can very effectively cache the results of (1). This makes that phase of the analysis not performance critical. Phase 2 has to be fast, but you can push as much expense into phase 1 as possible.

This provides scalability - both at the level of GitHub where people may view code from many different commits, but also for the live editing case where you may be editing a small number of files in a much larger project. The effects of your small changes may affect many of your dependencies and you want to know the lint errors immediately.

[1]: https://github.blog/2021-12-09-introducing-stack-graphs/