Hacker News Reader

Comment by fmbb | Hacker News Reader

What is special in starlark that makes it hermetic and deterministic?

Scala is deterministic.

If you call functions that have side effects and nondeterministic behavior you can fall outside the comforts of determinism. But you can stumble upon library functions someone wrote in Starlark that accidentally put you there as well.

The Starlark homepage says

> Hermetic execution - Execution cannot access the file system, network, system clock. It is safe to execute untrusted code.

But the last time I wrote Starlark it was to define build targets in Basel. And executing the builds definitely accessed my file system and the network, otherwise builds would have no results.

View on HN

Bazel splits the build into multiple phases. Starlark only comes into play in the first two, load and analysis. During these phases, Starlark code doesn't have access to the filesystem, except in a few very limited cases like using the glob() function to expand a wildcard to a list of source files. Furthermore, it only generates an abstract graph of build actions. The Bazel engine is responsible for executing this graph in later stages, which might result in non-hermetic things happening but usually not.

Starlark has intentionally limited functionality such as lacking Turing completeness or global variables. This provides guarantees that it can be executed in parallel and will have a finite runtime.

Bazel enforces a hermetic sandbox for Starlark to operate within.

All input files have to be declared, and the build process can only see files that are declared.

Starlark cannot access arbitrary files at runtime, and it deliberately has no APIs for things like system time or random number generation, or global state.

Scala, on the other hand, has no such restrictions. As much as I love Scala, I think it’s an odd choice for a pure, deterministic system. Although perhaps if you use Scala to build a DSL (an area where Scala shines) you could engineer a pure functional sandbox within Scala.

Do you only declare the file names, or also file content and owner and group and everything up front?

Sure it deliberately has a bunch of restrictions in its base form. But in order to use it for anything you have to write custom actions or download and execute custom actions others wrote. And this will mutate your file system and run arbitrary executables.

Input and output files are tracked determistically via content hashing. It cannot reach outside of its sandbox and touch the network or actual filesystem. Those are all (generally pinned) inputs to the system. For a given version of a script and a tuple of input content hashes, you get the exact same output files every time. No way to accidentally iterate over a hashmap, embed a timestamp, or leave the filesystem in a weird state because you were interrupted.

As someone who worked a bit with Bazel actions and uses Scala: I think Bazel cuts down on the area where arbitrary I/O happens. Also, many executables come from pre-defined targets (e.g. java toolchains). One way to deal with I/O is if Mill injected its own implementation of a JVM filesystem/network layer, which I'm pretty sure it can do.

> Scala is deterministic.

I don't see how Scala is any more deterministic than any other language.

That’s what I’m saying. About Starlark.

Starlark does not expose sources of randomness like date.now()

If you're deliberately using that to prove Scala doesn't work for config than sure

It’s about guarantees vs eternal vigilance.

Reply on news.ycombinator.com