Hacker News Reader

Comment by jacques_chester | Hacker News Reader

My experience is that this is difficult without the system under test being designed to operate deterministically.

So for example: the system clock needs to be replaceable with a fake, or the current time needs to be injected, or perhaps both.

Another example: Go channels. These are a real PITA to surface in a way that allows a test harness to drive them deterministically.

A while ago I built a simulator testing harness for an early version of the Knative autoscaler[0]. I eventually ran aground on the difficulty of keeping up with changes, but it may be rearchitected and repurposed to assist Kubernetes HPA development instead.

FoundationDB's approach is to pretty much bake optional determinism into the language with some macro magic. It makes me wish that were easier to do in other languages.

[0] https://github.com/pivotal/skenario

View on HN

I used to have a similar opinion, that it requires a special language to leverage such power. But recently I've discovered that Rust, with its latest advancement in async/await design together with swappable runtime, can provide a decent story for building determinism tests like described in the video. Yes of course one still needs to mock all the IO as well as clock part, but the deterministic core is already there, and with a carefully designed runtime and surrounding library, we might have pretty good support in Rust for this.

Disclaimer: I'm never a Rust zealot, I do understand Rust has its tradeoffs, and it's not the panacea for every problem. In fact I do have side projects which are perfectly suited and written in Go. I'm just saying Rust's design turns out to be suitable for such a deterministic testing structure, and Rust's target for system programming, can also benefit A LOT from this style of testing.

And also a shameless plug: I do have some initial work exploring this area: https://github.com/xxuejie/diviner. It's still quite rough and a lot of work is needed but I do believe this is something that is worth exploring.

In the FDB simulator case, every event has a future time at which it will fire, and the clock advances to that time when the event is dequeued.

This also has the side benefit of artificially “speeding up time”, which is useful when many actions happen based on long timeouts.

I used the same approach, which I learned from DES textbooks. I wrote up the design here: https://github.com/pivotal/skenario/blob/master/docs/concept...

WebAssembly allows deterministic execution (with caveats: https://github.com/WebAssembly/design/blob/master/Nondetermi...) of any code. I think it is a perfect platform for such type of testing.

Reply on news.ycombinator.com