Subprocesses are generally bad news

One question though, is it possible to intersperse pure Julia functions but still have them piped, e.g. write custom “shell” commends?

Except when they aren't? I'm thinking of things like the bash exported function bug, an obscure feature nobody thought was there turns evaluating environment variables into RCE. Not to mention different shells may do things slightly differently and you may get caught doing something unexpected.

So why not leave the shell out of the picture entirely? I think execve and friends are much easier to get right than escaping. There are still possible security bugs to consider (eg., if using the -p family such as execvp, who controls PATH? Is anything on PATH writeable?), but the shell won't be among them.

This cult is called engineering reliable systems. If you want to build a system that actually works (instead of mathematically prove that a workable system could exist), you have to take the following into account:

- Describing a complicated system is harder than describing a simple system; there may be spec bugs.

- Implementing a complicated system is harder than implementing a simple system; there are almost certainly implementation bugs, and they are not evenly distributed throughout the entire implementation.

- Code that isn't regularly exercised with full and meaningful coverage is likely to have unexplored bugs, so if you're using third-party code (like a shell), there's little sense in building your system on top of the most obscure parts of it.

- Full and meaningful coverage could mean tests, but is not very likely to, because humans are just not very good at writing good tests and keeping up with them. Humans who write your third-party dependencies (like your /bin/sh) are not subject to your organization's policies about test coverage.

- Code changes, and realistically you have to keep up with changes from external code. Code that matches your expectations today might fail to match them tomorrow, for the same reason that the obscure parts might change because no one is using them.

- You, personally, are fallible. You will have off days. You will forget the edge case you thought of last week. You will come back to your code a year later and not understand the complexity, and then you will be responsible for figuring out the complexity.

- You're not the only person maintaining your code, and even if you had a hiring veto, you're not going to be able to enforce that everyone on your team has the same sense for the same complexities that come naturally to you. Your complex code will get modified by a teammate when you're on vacation, and then you'll get paged for it when you return. It's in your own interest not to set up this situation.

- In general, the people implementing and running your systems are humans, and they have all sorts of weird human incentives: shipping the feature instead of spending tons of time on the edge cases, fixing the broken system in the fastest way possible when it fails at 4:30 on a Friday, not correcting their more-senior teammate's code in reviews if they're not confident they're right, etc. You are much more likely to succeed at engineering around these failure modes than getting humans to stop having human faults.

- If doing the complex thing is inevitable, there's an art to figuring out whether you should do so rarely to minimize the number of mistakes, or frequently so that you get good at it. Both are reasonable options and the better choice depends a lot on the risks.

And even if you want to take the mathematical proof route, you're still subject to incompleteness. I'd much prefer the approach that empirically gives me a high chance of building reliable systems in practice.

In particular, I disagree with the claim "Escaping rules are well-defined (they must be, by necessity)" - there is no necessity that the specification be complete, and there's certainly no necessity that whatever implementation you're using faithfully implements all the rules.