Lisp programmers realized in the 70s that having an artificial division between code and data was frequently problematic, hopefully other languages continue to catch on as well :)
Unfortunately every configuration file format these days seems to add so many features that it inevitably becomes Turing-complete... the trick is finding the balance between expressiveness and halting decidability.
[1] http://www.lispworks.com/documentation/HyperSpec/Body/v_rd_e...
I don't care for the syntax (at all) but I am seriously impressed by the careful design. I've always wondered where you'd need to draw the line between plaintext config files and executable code, and Dhall is much further into "programming language" territory than I would've expected.
*Dhall homepage for anyone else that's interested - https://dhall-lang.org
Sounds a bit like a missing specification for the distinction of load-time and run-time configuration.
Configuration, to me, is something that happens once and not often, and is therefore "something = foo". Update is a run-time thing, and therefore "if something then bar".
Lua covers both of those cases superlatively. Case A, byte code. Case B, same. Just be sure you have "generate byte code" in the right place, C, too ..
It would be nice to be able to statically verify that their submitted configuration "behaves nicely", but the moment you introduce any kind of Turing-complete configuration language all guarantees go out the window because the problem becomes literally undecidable. [And then you try to mitigate it by imposing limits like "kill the parse process if it takes longer than X seconds or uses more than Y megabytes of memory".]
The advantage of, say, a simple .INI file for configuration purposes is that the time required to parse it (and storage space required to remember what's in it) is O(n) where n=size of config file...
EDIT: so take the example Lua file from the original post but change the loop limits from "32" to "16777216":
for i = 0, 16777216 do
for j = 0, 1677216 do
add_body({x = 20 * i, y = 20 * j, mass = -0.1, rad = 2})
end
end
More troubling are loops which consume CPU without hitting memory limits. I would hope Lua provides a way to limit the number of instructions when evaluating expressions. If not then delegating reading the config to a limited subprocess could work at the cost of still more complexity.
Like, I get that there are services that require this level of trust to the end user, but why wouldn't I solve this problem by timing the config load and immediately stopping any config process that takes longer than it should? Its the 21st century, we can still use interrupts. ;)
And whichever you choose, that's great(I've gone on a huge concatenative binge myself), but then to define the application's final protocol, you have to filter it. You have to filter to turn many of these meanings into errors, which leads towards syntax, type-checking, and all the rest.
And to the extent that this is a problem, it mostly reflects the human and philosophical factors. What we believe in determines what meanings we wish to see represented in the protocol; our beliefs are many, varied and often contradictory, with entire organizations and systems of governance designed around reconciliation of these beliefs; therefore extensively structured protocols are necessary to define the "sometimes-errors" that result from our incoherence, our limited notion of reality. And that is producing the gradual ramp up towards complex languages, formats and protocols that address the impossible with the incomprehensible.
But data and protocols that are very truthful usually do not require such a complex structure, or that structure emerges from natural laws. They may be hard to compute, or hard to store - DNA sequences are very large, and modern cryptography is premised on computational difficulty - but the code that deals with these things is not large in expression, and only needs validation along some very particular axes. Regardless the task of discovering and implementing these things is hard, because evaluating the theory is hard, much harder than validating whether a stream of bytes represents a number, a string, or a list.
You're technically correct (the best kind of correct), but jandrese wrote "a complete language", not "a Turing-complete language". You don't need Turing-completeness to create security problems – filesystem access alone does that.
> The former is trivial to solve: add a timeout.
That only solves the termination problem, which is a minor concern anyway. You don't need much time to compromise a system if you have filesystem access. No "bugs in the parser implementation" needed.