JsonLogic
jsonlogic.com
jsonlogic.com
If you are reading this, and you think it is a good idea, please don't follow it as it is not.
Back in 2006-2009 I worked at MobiTV, which we had the same exact way/method of programing our mobile apps, on J2ME/feature phones. The made up language was called CEF which was based on XML. While the idea is sound in paper, This turns really fast into an ugly, debug everywhere type of scenario, where it is hard to both develop and debug properly.
The only way this is ok, is when you have some fully automated UI tool, that stores some kind of state, and retrieves it back, but with no human intervention and some very very limited logic. Otherwise you are in a world of pain. If you try to include real application logic into something like this it is going to hurt your business long term.
It will turn your business logic like some Maven configuration, full of gotchas and more.
However, something I’ve come to learn is that some ideas I thought were bad were actually not bad. They were poorly executed and the exact reasoning why they did not work well in a given application was not well-understood.
So I can see an argument for “this was horrible back when it was done in the early 2000s” simultaneously with “this could actually still be a good idea for some use cases.”
I realize that people need to make decisions based on their personal experience, and I don’t have a good rule-of-thumb for when to try a variation of a “bad” idea.
Sometimes you need the BFH to get to the elegant solution.
In nature, species populations compete with each other and are kept in a dynamic equilibrium. If population size of a species increases, then its predators will have more opportunities to predate, and their population size increases as well. This leads to more predation pressure, which should negate the population increase and decreases opportunities for predators as well. "Predators" is to be understood in a very general way. It can also apply to fungi and other microbial forms of life.
Changes to this equilibrium arise when prey species evolve to resist the predation pressure by predators. And vice versa if the predators improve. Then, ecosystems change, and these changes can be very disruptive.
Agriculture is a rebellion against this dynamic equilibrium because it creates irresistible opportunities for predators. They have to be held in check either by brute force (hunting, pesticides, containment) or by hacking the ecosystem (for lack of a better term) to ensure that helpful predators are around. The former is probably unsustainable in the long term as it is not at all clear that we will forever be able to outsmart the evolutionary potential of predator species. The latter is difficult as well.
Apart from predation, the other challenge to agriculture is that it depletes the soil in the long term. Fertile soil arises as a result of a ecosystem where the decomposers cannot keep up. Most forms of agriculture interrupt such ecosystems and thus deplete the soil in various ways.
To keep this in check, different methods were evolved, such as slash-and-burn, crop rotation and artificial fertilization. But it is not at all sure that we will always be able to stop depletion of our soils just in time to prevent civilizational collapse. Currently, availability of phosphorus is a looming danger because most of it is mined. Once we hit peak phosphorus, we better have solutions in place to get it from somewhere else.
I don't see why serializing close to JSON is any worse than doin so to, say, S-expressions, or Algol-style syntax, or WASM.
If you aren't manually coding it, but generating it, the capabilities matter, the syntax matters a lot less.
Or do you think we are at our peak?
You mean like every single idea/project/etc that ever existed before their invention?
From now on, when someone tells me something is a bad idea, I'll take it mean that I should examine it further.
But what reasons and are they relevant to this use? For instance, I can think of one really good reason (actually several different ones justifying different formats getting used depending on the actual use case, but they can all be summarized the same way when it comes to not using JSON): Shipping an AST between endpoints one of which is a web page and therefore has JSON as its best supported generic object serialization format isn’t the use case.
For people who've spent a lot of time with ANT, you might be familiar with this article from its creator: https://web.archive.org/web/20041217023752///x180.net/Journa...
It was possible and even made sense to use XML as the format for a build tool, however, it got stretched way to far. The idea was applied in use cases that were less and less appropraite, as a result years later you end up with something that is unwieldy and doesn't make sense and, generally, thought of as "terrible".
There are use cases were JsonLogic might make a lot of sense, and for those that is great.
Generally, I'm sceptical as there is high risk of increasing couplings in a way that will make change much harder. This will be particularly true if it is applied to use cases that don't truly warrent this kind of solution.
Of course! JSON is the modern equivalent of XML. In 10 years everybody will mock JSON just like today we mock XML.
- does not entirely make it clear what should go into a tag attribute versus a nested tag
- conversion of those two features into a unified internal datatype can be tricky (for example, this property led to choices that for erlang's xmerl causes a security regression if you are handling user-submitted xml).
- first class support for text inside of a tag is not what you want in 90% of data interchange events, and support for it creates a burden when deserializing.
- it's confusing what should be a text inside a tag versus a tag attribute. Or should that text be parsed into a number? At least JSON is strongly typed, you know what datatype is what just by looking at it.
- because there aren't good "best practices" around it, everyone rolls their XML schemas with their own ideas about what sort of thing should go where.
The major benefit of XML is its obession with schematization means that at least in many case you can grab a schema and figure out an appropriate declarative command to obtain what you want (if you have the right library), but this has basically been conquered in JSONland using JsonSchema/OpenAPI, and stuff like JsonLogic.
JSON's types are pretty anemic compared to the XMLSchema's datatypes[0]. Even it you don't use all of XSD's complex type system, having these basic datatypes is extremely useful and I wish JSON had similar.
The same thing is true if you replace “JSON” and “XMLSchema” with “XML” and “JSON Schema”, respectively.
And also, though less so, if you do the same but replace “XML Schema” with “JSON” instead of “JSON Schema”.
- JSON has just one escape format and all unicode codepoints can be used. XML generally allows only certain unicode codepoints, at some places like element names its further restricted (Quick: Which characters are not allowed? What's the difference between XML 1.0 and 1.1 here?). XML escapes at every place differently. In comments you need to escape less than elsewhere. In attributes you can use newlines but they will get normalized to spaces, you have to hex-encode newlines. There's Cdata.
- JSON doesn't have the entity madness with potential security issues.
- JSON doesn't have namespaces that add a lot of complexity with little benefit
Consider how you'd write these 2 bits of XML in JSON:
<A><B>foo</B><C>bar</C></A>
vs.
<A><B>too</B><C>bar</C><B>uh oh</B></A>
Without a schema, in the first case, how do you determine whether B should be "foo" or ["foo"]? In the second how do you preserve the relative ordering of B #1, C and B #2?
I worked on a project where we had a parse tree that we wanted to serialize out, and for this reason XML was the natural choice.
vs.
{ "a": [ { "b": "too" }, { "c": "bar" }, { "b": "uh oh" } ] }
- Only ever have a single key in each JSON object (treating them as tuples)
- Make all object values arrays.
It makes for some very unnatural looking JSON.
Otherwise it's very clear about the data's format and when there's ordering required
Not to me; its a fairly standard and natural JSON pattern. It’s tue that it use is usually quite limited in any given representation, but usually the thing you are modelling in JSON isn’t general XML with no further knowledge of application domain or logical schema.
This looks like line noise to me. All that punctuation and quoting looks ridiculous. What is the problem with straightforward and trivial to parse things like
[a]
b = foo
c = bar
a.b = foo
a.c = bar {a_list:[{text: too, type: b}, {text: bar, type: c}, {text: uh_oh, type: b}]}}
We wanted a parse tree that could be serialized out to produce the original document byte for byte, but still reflected the grammar.
Things get uglier for JSON in the real world case:
"5+7 #comment"
<sum><integer>5</integer><operation>+</operation><integer>7</integer> #comment</sum>
This was primarily a mechanism to load parsed documents in to code that didn't have access to the recursive descent parser that parsed the document, for either security or performance reasons.
Loading the XML with the appropriate parse flags, doing a pre-order traversal on all the leaf PCDATA, and then printing it out, would regenerate the original document byte for byte.
Perhaps I should go back and implement JSON output and see how I feel about it... thought I'm pretty sure once you introduce a bunch of "text" and "type" fields to the JSON tree it's going to look a lot less readable than XML markup.
There are other fields as well like line and column numbers (output as XML attributes)
I've had to roll my own hack for such cases.
Well, if you are literally developing a model for recasting general XML in JSON, then you know the children of an entity should be a list because duplication os possible and order is signifcant. (Given the close relationships between XML and HTML and between JS and JSON, the obvious starting point for a general solution is to copy the HTML DOM, using only the properties.)
If you are trying not to reserialize XML generally but to solve the same problem as the original XML, then you probably aren’t doing it without a (logical) schema—you know what the object model is that you are trying to stuff into JSON, and you decide based on that.
I can implement a full, strict, compliant JSON parser in an afternoon, maybe even less time. It is simple enough to parse that you can even do it in pure C with relative ease. XML is a lot more complex and a lot of implementations skimped pretty hard on the details. I mean I have fond memories of tinyxml2, and if you take apart any app made in the 2000s you have a good chance of finding reference to it (or on Windows-only apps, MSXML3, or on Linux, libxml...) but it certainly let you do a lot of surprising things. I saw a lot of apps doing XML documents with multiple root nodes. And Apple's plist format is a nightmare. It's so easy to fuck up interpreting XML plist, that Apple did, and it lead to one of the more entertaining privilege escalation bugs in iOS history.
I have some fond memories, but I just don't miss XML. :)
I haven't seen any widely-adopted equivalent in JSON. I've seen some proposals floating around but none of them seem to have really taken off.
We already have this in JSON: add a hashmap key with a prefix. It's at least as robust provided tools do the sensible thing and ignore keys they don't care about. But as with XML, sometimes, they don't, or do, or something.
Maybe I'm paranoid, but I worry someone else will pick the same prefix as me. If you make the prefix something like a domain name, e.g example.com:myExtension, nobody else is going to use the same prefix assuming you own example.com. That avoids the risk, but it can be a lot of typing. Whereas, if the prefix is something short and obviously meaningful (e.g "doc:"), there is a risk someone else will use the same prefix in a conflicting way.
What I liked about XML namespaces is the indirection: the prefixes can be whatever you want, and then what makes them unique is the URIs they are defined to map to in each document.
The one point at which I think it didn't work very well, was when people started using prefixed names in attribute values. You can't tell whether "foo:bar" is a reference to the XML namespace with prefix "foo:", or some unrelated string of "foo:bar", without looking at the schema.
But none of that is really the problem - the problem is that the implementations and parsing are evidently sufficiently difficult that people just don't get them right - hence the hassle.
I would argue also that having indirection like this is just a negative all around - having to look elsewhere in the file to figure out what I'm looking at just creates problems, and in practice it doesn't look like this capability was ever really used - the only times I run into it are when I'm dealing with some vendor-specific implementation (that Python or Go or whatever can't handle so string-templating winds up being easier) and it turns out that what I need to have happen is to match the semantics of example XML exactly to get anything to work.
Which is really the point: the extra complexity in parsing means the probability you're dealing with a broken parser shoots through the roof, at which point you've lost any advantages. Whereas with JSON, the minimum bar is arrays and maps and the basic types work. Stay within those bounds, and whatever namespacing that happens will "just work".
If people use URLs, then DNS gives uniqueness to URLs. Of course each registrant has to enforce uniqueness within their registered domain – for a standards organisation or large software vendor, they often adopt a formal system (spreadsheet, database, naming convention, etc) for tracking which parts of the namespace under their domain name are reserved for which products/standards/etc. Of course, that relies on the vendor/organisation to do it properly – but if (e.g.) Microsoft were to create a mess by letting different Microsoft products use conflicting namespace URIs under microsoft.com, well that would be Microsoft's fault, and not the responsibility of any standard or technology to fix it.
(There is still one issue–what if the domain changes ownership? Some standards, for example ISO 19770, have tried to address this by constructing namespaces based on a domain name with its registration date, so if the domain later expires and is reused by a different organisation they won't clash since they'll have a new registration date.)
> the extra complexity in parsing means the probability you're dealing with a broken parser shoots through the roof
A lot of that is because namespaces were added to XML later, rather than being there from the start; they are effectively an optional feature. So then you have some parsers that support them, some parsers that don't, some parsers where it depends on a flag you have to set (which may default to true or false). I guess trying to add namespaces to JSON now would encounter the same problems. But if someone was to design a new serialisation format from scratch, and had namespace support in it from day 1, I don't think these issues would happen.
In practice not so much.
You will curse namespaces when you encounter some XML with multiple namespaces and your favorite parser starts choking on them.
Can you really? Doesn’t look like anyone else can: http://seriot.ch/parsing_json.php
There are are many parsers which perfectly parse every valid JSON document (like every Perl, Python and PHP parser and many others). They got non-perfect score because they accepted some JSON which standard says they should not have -- like a NaN value or a trailing comma.
While using such parsers can potentially cause problems -- perhaps because your data generator has a bug, and your not-strict-enough parser ignores this bug -- this happens very rarely in practice.
I did do this one actually in an afternoon:
https://github.com/pangbox/rugburn/blob/master/src/json.c
... but I will say that it is intentionally very limited in its API due to its intended use case and desired footprint. (And because it’s designed to compile with OpenWatcom 2 with no libc for only Win32, but those particular constraints are not very interesting for this.)
I would imagine if I did try, I would have a fighting chance at making something properly strict. But I admittedly don’t know how to handle recursion in a stackless manner.
Edit: specifically for C, it would be hard to do it properly unless I had access to an implementation of Clinger and Steele & White. Go would be easier since it has better floating point algorithms with strconv, but relying on C strtod/etc. is probably a bad idea due to variability. That having been said, maybe you could do Clinger in the same afternoon; I’d be worried about edge cases, but simply put, I haven’t tried.
I am not even a programmer by profession. Anyone that can write a recursive descent parser can write a JSON parser.
When you're dealing with complex data, JSON isn't very fun, but XML is still worse.
This annotation structure not only lets us transform the xml into HTML that resembles the physical book via CSS, but also lets us search the content with awareness of what the content is. Simple example, if a search term appears in a title give it more weight.
When a book is done the XML goes off to a printer to become a book, AND into our XML database. A nice feature of which is a lot of our content is in the DB as XML, you ask for a book page and the DB returns the HTML for the page, backend just forwards that HTML to the browser which displays it. ZERO parsing/transformation anywhere once the DB sends it! Just fling the payload at the browser. Super cool at times.
Anyway we all work with these formats differently, some of us never touch the json and xml, some of us only tweak values in config files. Those of us who work with it like that, the only differences between XML and JSON are "how nice are the serialization apis and how fast are they? how big is the payload?"
But when working WITH the format directly there can be more meaningful differences. And the tools that actually exist for the things you are doing may push you towards one format or the other, even if in principle it wouldn't have really mattered.
note: I keep mixing tenses here but this was all in the past, we got bought, don't do any of this anymore!
To clarify: no, I do not believe that JSON is universally better than XML. I believe JSON is better than XML for the majority of use cases where people are currently using JSON today, esp. as an interchange format.
Light-years away from zero-copy serialization formats.
See https://en.wikipedia.org/wiki/Comparison_of_data-serializati...
Random example: https://google.github.io/flatbuffers/md__benchmarks.html
Speed is not everything: being able to self-describe data structures (without the data) enables generation of parsers, documentation and tests.
Streaming support also matters.
(I'm certain one can find a slow JSON serializer out performed by a fast XML serializer, but that's not the point.)
<roof>
<true>
1
</true>
<false>
1
</false>
</roof>
(The really horrible thing was that there were … I don't want to call them "reasons" because that's a strong word, so let's say "cause and effect" that led to this format… and once you came to know them, you "understood", in a sort of horrible sense, the format. But give me a JSON bool any day over it.)
https://www.w3.org/TR/xmlschema-2/#built-in-primitive-dataty...
If you are somehow in an environment where you are working with data nominally encoded according to XML spec, yet unable to leverage any other parts of the standard XML ecosystem, then I would certainly agree that XML is a bad choice!
There are people who actually use JSON for markup... the resulting schemas are usually very bad, and they disappear quickly without even gaining fraction.
When used properly -- XML for markup, JSON for serialization -- each format is pretty nice, and will probably last forever.
Say we have a schema which defines certain data. Let's say it's a "credential" which is a "COVID-19 Back to Work Credential." The credential is valid in the case that someone either 1) has a COVID vaccination or 2) has a negative COVID test in the past 72 hours. That data might look like this:
{
"credentialName": "COVID-19 Back to Work Credential",
"credentialIssuer": "Acme Labs Inc.",
"credentialData": {
"rapidSarsCov2PCRTest": {
"hasTestResult": true,
"testResult": "negative",
"testTime": 1622145851
},
"covid19Vaccination": {
"hasVaccination": false
}
}
}
Optimally, I'd be able to do this within the same schema or file.
With JsonLogic, that might look like this (probably flawed logic, just a demo, not using this in production, may cause your computer to go up in flames, etc.):
{
...
"parsingMatrix": {
"if": [
{
"or": [
{
"===": [
{ "var": "credentialData.rapidSarsCov2PCRTest.testResult" },
"negative"
]
},
{
"===": [
{ "var": "credentialData.covid19Vaccination.hasVaccination" },
true
]
}
]
},
"valid",
"invalid"
]
}
}
To answer your question, what you have is (effectively) code, and just use a programming language for this, call it a function, give it a name, arguments, test suite and register it as a plugin to your platform. Oh, and documentation, change history, etc.
Seriously - the "overhead" more than pays for itself when 100 other yahoos do this nonsense and you end up with data files containing untestable code that breaks when combined with other code, breaks when the platform is changed, etc etc
What is a better way to share this important part of the schema (the matrix) without resorting to letting the "business logic" perform the evaluation? Optimally, any matrix should be able to run against the same business logic for evaluation.
Also, though I understand why you say that, we won't have "100 other yahoos doing this nonsense," and each schema would go through a pretty thorough review process.
If your concern is the sharability, consider making a library or service that provides the business logic.
Well, they _must_ have prior knowledge about it, because they must have to understand the concept of JsonLogic beforehand. So instead, you can just make these systems understand a proper programming language that can produce outputs. Something like https://github.com/dhall-lang/dhall-lang would probably be a much better option for this usecase.
Your problem is that you need the same logic in two places, and you can solve it by centralizing this logic in one place.
Libraries tend to cover broader spectrum of use cases than you'd need. Usually that's okay (eg: URL parsing, string manipulation, HTTP client, etc) but in cases like this the logic schema should be as absolutely minimal as possible.
If you're building an interpreter and need an interchangeable format then you can always generate that out of your own bespoke and very constrained format.
(credentialData.rapidSarsCov2PCRTest.testResult==='negative')
|| credentialData.covid19Vaccination.hasVaccination)
? 'valid' : 'invalid'
It walks through the process of building a Lisp in JSON and the associated parser in JavaScript. It could be a good starting point to build out a cross-platform approach to sending logic over the wire.
It seems weird to me to pass a syntax tree over the network rather than just the syntax. But if the expression is not supposed to be viewed or edited as code at all, but only in some kind of GUI expression builder, then it might make sense.
Lisp is at least human-writable, but if it is supposed to be written by humans why not just use straightforward expression syntax like "a + b".
With all due respect, you're making a classic mistake: presenting the solution to a problem without tracing the problem back to the use case.
Why do you need to include the logic matrix in the schema? I can promise you that the real world use cases where you'd need to this are virtually non-existent. You wouldn't even do this for some extreme case like Mars rover software.
Your business logic should have a single source of truth. The vast majority of the time you can just write an API for other software to synchronize with.
Otherwise just ship new builds of your software package to the client instead of a new schema. If you need to be modular then use modular primitives such as shared libraries.
Again, there's virtually no real world scenario where you need to ship a logic schema (in any format) to a client. You might as well ship a shared library, importable module, a completely new binary, etc, etc.
> Interoperability is the main reason. We need this data and the evaluation matrix to be portable across different "credential wallets" as part of a standard model for matrix evaluation of Verifiable Credentials (see also https://www.w3.org/TR/vc-data-model/).
Representing verification logic in the schema flawed. Just because the W3C publishes a spec doesn't make it good.
So I guess one of the few valid use cases for JsonLogic (or similar) is if your company makes a bad technical decision and you're forced to implement something?
The "verification" required is one of tamper proof, cryptographic safe certificate chain verification, something like the one done using Certificate Authorities, certificate revogation list and the surrounding infrastructure used in TLS.
Encoding anything Turing complete is out-of-spec.
["if", ["or", ["===", ["var", "credentialData.rapidSarsCov2PCRTest.testResult" ], "negative"],
["===", ["var", "credentialData.covid19Vaccination.hasVaccination"], true ] ],
"valid",
"invalid"]
?It's hard to gauge whether or not it's better to go with an existing, possibly inferior standard, or to create a new one, like this...
But yes, does look like other people have grazed this idea as well: https://www.i-programmer.info/programming/javascript/2380-ja....
You could also think about Dhall as a front-end. You could represent the whole credential as a union type. This isn't deeply thought out, but this should give an idea as to how this would look:
let TestResult = < Positive | Negative >
let ScreeningTestDetails =
{ hasTestResult : Bool, testResult : TestResult, testTime : Natural }
let CredentialData =
< Vaccinated | RapidSarsCov2PCRTest : ScreeningTestDetails >
let Credential =
{ credentialName : Text
, credentialIssuer : Text
, credentialData : CredentialData
}
let result
: Credential
= { credentialName = "COVID-19 Back to Work Credential"
, credentialIssuer = "Acme Labs Inc."
, credentialData = CredentialData.Vaccinated
}
let result2
: Credential
= { credentialName = "COVID-19 Back to Work Credential"
, credentialIssuer = "Acme Labs Inc."
, credentialData =
CredentialData.RapidSarsCov2PCRTest
{ hasTestResult = True
, testResult = TestResult.Negative
, testTime = 1622145851
}
}
let prop0 =
λ(x : ScreeningTestDetails) →
assert : greaterThan x.testTime 1522145851 ≡ True
in result2
Edit: added an assertion to model the 72 hour requirement.
{...
parsingMatrix
(if (or (=== (var "credentialData.rapidSarsCov2PCRTest.testresult")
"negative")
(=== (var "credentialData.covid19Vaccination.hasVaccination")
true))
"valid"
"invalid")}This strikes me as nice:
(parsing-matrix
(if (or (= (var "credentialData" "rapidSarsCov2PCRTest" "testResult") "negative")
(= (var "credentialData" "covid19Vaccination" "hasVaccination") "negative"))
"valid"
"invalid"))Here is the thing...There is already a crappy way to describe JSON Schemas[1,2].
It is hard to grok by visual inspection and expressing conditions like "this element can either be a list containing a subset of these values or a hash with keys that belong to the same set" etc make it unwieldy.
The schema ends up being really brittle and hard to modify. 100% green-lit test coverage don't mean a thing when it is soooo easy to end up with a false negative.
Now take that and add logic to it? One has to question the existence of all the programming languages in use if we are going to end up subjecting ourselves to that torture.
[1]: https://json-schema.org/ [2]: https://json-schema.org/draft/2020-12/json-schema-core.html
Json schema validates the data structure, while this seems to be more about writing algorithms in json
That's usually how this type of thing goes wrong. I've seen similar things both work and fail for that reason.
This language made it an explicit goal, though.
I came to the comment section here hoping that the site was actually a joke...
There are limited scopes in which this can work but it seems to be more about the environment or code loader than about the programming language, per se: for example, plugin systems for build tools and servers. The plugin might be isolated by runtime features (like in the JVM) or by processes (as some web servers do, or as Postgres does) but it's not language features that tend to be useful.
Tensorflow models and CSS selectors are both examples of fairly useful plugins that still have limited enough semantics to be sandboxed effectively.
I'm often in the situation where you try to add all kinds of parameters to a query-field in the UI but no matter how hard you try there will always be a power user with the request to do some combination of AND/OR/CONTAINS/NOT with nested parenthesis that is not supported.
After hundred such requests you get sick of it and just want to expose the raw DB-query to the front end, which obviously is a bad idea for security reasons. Having a safe intermediate representation that is sophisticated enough for some logic can be a good idea. Instead of reinventing your own, using this language can be done, as long as it has a mapping into each database query language.
If that power user is properly skilled, authorized and authenticated, why not give them a read-only connection to the DB and let them write their own SQL?
How many times has the UI been nothing but an impediment?
What is really the meaningful difference between this and an s-expression language? “” and {}, really. It’s a different way to serialize the same thing.
So I’d argue a lot of the criticism is valid.
We do, as an industry, tend to run in circles.
Turing completeness.
>So I’d argue a lot of the criticism is valid.
I'd argue they should have paid more attention in college. This is far from a superficial difference.
You might as well call HTML a programming language. Seriously.
>Turing completeness.
S-expressions in and of themselves are obviously not Turing complete. This language could easily have been encoded in S-expressions, just like Lisp can be encoded as JSON. (Like, trivially, as lists with quoted strings instead of atoms, or using whatever object-based syntax.)
There's thousands of ways this could have been encoded. Why that one specifically?
I had only a vague idea of what an AST was, but the highest upvoted comment was clearly saying: This is dumb, we have dedicated formats for turning text into ASTs. ( i.e. a programming language ).
In the past decade+? i have learned what an AST is and have now twice, in a professional setting, prevented an engineer from re-inventing this very concept.
If i had to choose if hackernews is a place where smug snark comments crack down on re-inventing known concepts or a happy place where every idea is a good one i will choose the former.
And once every blue moon we get to laugh and look back at snarky comments such as this: https://news.ycombinator.com/item?id=9224
This is one of the few projects addressing the latter problem space.
JSON is a serialization format. So say you have an in-memory AST, and you want to serialize it: why not use JSON?
If you don't want or need to serialize it, then of course don't use JSON. I can make up plenty of other reasons not to, but they're variations on some other format being better for the application, not JSON being inadequate in any way. I'd probably use JSON myself, and I don't care for it much.
Or are we talking about building up an AST in JSON directly rather than taking some parser output and serializing it? Ok that's... not an AST then, it's a programming language with JSON compatible syntax, and yes I would try and prevent that from getting implemented as well.
Lispers are always pointing out how the parenthesis-and-whitespace notation is just incidental; how s-expressions are really something deeper that isn't bound to a specific syntax, and how this is a strength. The OP is a demonstration of that strength.
0. https://stackoverflow.com/questions/1423081/json-left-out-in...
Citation? I don’t know many people who would call Brendan Eich anti-intellectual for instance.
level = new Expression(['map', ['char', ['tail', 2], 0], {'=': 1, '-': 2}])
body = new Expression(['parse', ['trim', ['body']]])
name = new Expression(['replace', ['trim', ['body', 1]], /[ \t\r\n]+/, ' '])
target = new Expression(['first', ['body', 2], ['body', 3], ''])
For me this is just a temporary throwaway solution while I work on other more critical stuff first. I will replace it with an s-expression based DSL later. Like JSONLogic, it won't be Turing complete.
It just seems to me that JSONLogic is making a huge deal for what's really just a small reusable library.
> JsonLogic has no setters, no loops, no functions or gotos. One rule leads to one decision, with no side effects and deterministic computation time.
It's like eBPF for applications!
The fact that it use JSON instead of your favorite serialization format isn't the thing to be talking about.
And this latter point is why it's just a monumentally terrible idea. I mean.. just look at the Custom Operations wiki[1]. It's honestly just horrible -- horrible -- and may actually even summon Zalgo[2]. This is exactly how we ended up with the XML nightmare of the late 90s/early 2000s. JSON is not meant to be programmable. Please, for the love of all that is holy, stop.
[1] https://github.com/jwadhams/json-logic-js/wiki/Custom-Operat...
Yes! Oh my god, this. I remember how XML ended up being abused to express Turing complete logic in too many instances. Then they even specified an XML format to transform other XML documents, called XSLT, which is itself Turing complete. It really makes my head hurt.
I think why people gravitated away from XML was because it had become a convoluted mess. The JSON specification is terse on purpose, famously enough so to fit on a business card. If you want to abuse JSON to express logic, please keep it as your dirty little secret.
https://github.com/Bestowinc/json-logic-rs
I am currently working on an extension to add functions and more strongly typed operators to avoid the hassle of JS’ type conversion. That extension will probably be released as a separate package, because I am no longer with Bestow.
I honestly thought it was parody around the unnecessary complexity of modern software.
Any sufficiently complicated program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp. (Greenspun's tenth rule)
> JsonLogic has no setters, no loops, no functions or gotos. One rule leads to one decision, with no side effects and deterministic computation time.
LISP is, on the other hand, a programming language. It's rather famous for it.
We did a little write-up about our experience with it here if anybody is interested:
https://blog.routable.com/tablematic-a-tale-of-server-driven...
Be kind. Don't be snarky...
Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something.
My question is if not a solution like this how then SHOULD I be implementing communication of such rules from the frontend to the backend?
I work on and off on a very simple syntax language for things like templates, but would be well suited towards business rules and such too. It was the first parser I wrote outside of school and pretty easy to understand, small and fast too: https://www.npmjs.com/package/jexpr
I even did a library to implement it in go language: https://github.com/diegoholiveira/jsonlogic
- If it's an object, then nothing is going to enforce there only being one key.
- If it's a unary operator, then the value doesn't need to be wrapped in an array, but then how does it distinguish between an unary operator, whose operand is an array, vs. an n-ary operator?
Both of these problems would be solved by having each clause be an array, with the first value being the operator, and the rest of the operands being the parameters. That way having multiple operators isn't even syntactically possible, and having multiple operands is just as simple as the unary operand case.
When I see this sort of thing (which I assume was inspired by MongoDB's own atrocious "query language") I'm always surprised by the fact that many engineers don't seem to be aware of how easy it is to write a parser that is actually designed for the data you're trying to represent. At least "easy" in the context of undertaking a project that is attempting to generalize this exact problem.
JSON is a fine interchange format until you need a specialized runtime to evaluate the data, which is what is happening here. At that point there's zero benefit to using a general purpose interchange format just because it already exists: just write a DSL.
A good place to start: https://www.antlr.org/
I thought I was so clever :) Apparently, engineers have a habit of doing this.
The website lists the ability to easily send it "between front-end and back-end code, and even store it in a database", is there an advantage to it being JSON for any of these goals?
Also, the logic can be embedded in an already-existing JSON file.
This is useful if you occasionally need to look at the file, but usually you don't, so it's not worth putting any more effort into nice tooling.
But I always thought there should be better way for this kind of application. I see a lot of comments about Lisp. My question is: Is Lisp better for above scenario or there is even better way?
One thing that surprised me was the inclusion of map/reduce/cat. Allowing these operators can make runtime exponential in the size of the program. I feel like that is against the spirit of the language.
Some will recognize that these are basically just Lisp s-expressions (as JsonLogic also seems to pretty much be)
"Code as data" indeed
A few other strategies for solving this problem include something along the lines of the Small Clojure Interpreter (1) and Defunctionalization (2, 3)
[1] https://github.com/borkdude/sci
[2] https://blog.sigplan.org/2019/12/30/defunctionalization-ever...
[3] https://www.cis.upenn.edu/~plclub/blog/2020-05-15-Defunction...
>JsonLogic has no setters, no loops, no functions or gotos. One rule leads to one decision, with no side effects and deterministic computation time.
Also it appears that sci is still experimental: "Experimental. Breaking changes are expected to happen at this phase. "
Sure, but one could still implement a Turin-incomplete language in S-expressions. It would be more readable and more succinct, too.
What about regular every day programming languages? When a language is posted to Hacker News we don't laugh them out of the room with "hahah! Why didn't you use S expression syntax for this language??"
Seriously. There are Lisp languages that could replace whatever language you're using, and, there are also Lisp languages/runtimes that you could embed in whatever language you're using, to avoid rewriting it from scratch like this.
The fact that people say "use a lisp" instead of "use Lisp X like everyone else" should tell you something.
It's like the difference between "use SQL" and "use SQLite" or "use Postgres."
Whichever works best for your particular constraints.
> It's like the difference between "use SQL" and "use SQLite" or "use Postgres."
Precisely! "Use Postgres" is a valid answer to the question, "which database should I use?", or "which relational database should I use?". And "use SQL" is a valid recommendation when you see someone reinventing relational data model and reimplementing a relational database without realizing it.
Similarly, "use a Lisp" is a valid recommendation when you see someone reinventing programming using trees from first principles - of which a telltale sign is reinventing s-expressions.
Chez Scheme can be embedded with a C API, is Apache licensed, and crazy fast.
Ephemeral, in-memory instances of SQLite can be incredibly powerful tools for building an extensible business logic engine.
The big takeaway is that SQL is a really great tool for exposing any sort of customizable logic over arbitrary datsets.
The only major caveat with this - You will typically have to perform some degree of transformation over source data in order to achieve a form of normalization well-suited to the writing of business logic (SQL). Assumptions (i.e. denormalizations beyond 3NF [0]) made in your schema here will inevitably hamper or kill the ability of the business to write flexible queries which can satisfy real-world needs. Today, a customer may have multiple accounts. In a few years, maybe we decide it goes the other way too. If you made an assumption, such as nesting these complex types in any way whatsoever, you are probably stuck with a steaming bag of shit that you now have to refactor and retest top-to-bottom. Using relation types to decouple nested complex types is the core tenant in my mind. Identity is foundation, but you can fudge your way around that a little bit.
This one caveat is why a lot of developers look at this idea as a bad one at face-value. It takes a lot of work & iterations with the business stakeholders to get this correct. You can't sit in a silo and expect to completely figure out the abstract nature of business types or learn how they might be related and in what ways. There really is a "correct" and "incorrect" way to go about it if you are being properly academic. Worry about webscale and performance later. Get it correct first. Nothing is too complex for a SQL schema.
If you can satisfy the academic gods of normalization and achieve a stable schema, then you are in for a wonderful ride. Between using views to compose higher-order functions, and UDFs [1] to wire into customized SQL functions, you can satisfy literally any degree of complexity required by the business. The only limit is your ability and discipline around modeling the domain types & relations, and willingness to get your hands dirty with some views and functions. You will find your business experts loving the power they gain by being able to write queries in a domain specific language they inherently understand. Giving them higher-order functions (views) and iterating on that side of the fence is yet another gigantic value play that is invisible if you are looking anywhere other than SQL.
[0]: https://en.wikipedia.org/wiki/Third_normal_form
[1]: https://www.sqlite.org/appfunc.htmlI'm sure there is some reason it's all a horrible idea but almost all my projects these days use React front-ends and Node.js back-ends and a shared library used by both, ideally containing most of the complex logic. This seems like a neat version of that that might be useful in an organization which uses many different languages.
Using JSON to do this would allow schema portability throughout the different languages in our stack, but of course it’s an imperfect solution. Open to hearing any other options (I have looked into YAML as well but that’s a whole other ball of wax)...
Here's your parsingMatrix from your other comment:
{
"if":[
{
"or":[
{
"===":[
{
"var":"credentialData.rapidSarsCov2PCRTest.testResult"
},
"negative"
]
},
{
"===":[
{
"var":"credentialData.covid19Vaccination.hasVaccination"
},
true
]
}
]
},
"valid",
"invalid"
]
}
{ "if": [{"or": [{"===":[{ "var":"credentialData.rapidSarsCov2PCRTest.testResult" }, "negative" ]},
{"===":[{ "var":"credentialData.covid19Vaccination.hasVaccination" }, true ]}]},
"valid",
"invalid"]};
(if (or (= (var "credentialData.rapidSarsCov2PCRTest.testResult") "negative")
(= (var "credentialData.covid19Vaccination.hasVaccination") #t))
"valid"
"invalid")
["if", ["or", ["=", ["var", "credentialData.rapidSarsCov2PCRTest.testResult"], "negative"],
["=", ["var", "credentialData.covid19Vaccination.hasVaccination"], true]],
"valid",
"invalid"]
I guess that last example you demonstrated works, but I fail to see an appreciable advantage of using that over JsonLogic. It seems quite similar aside from being "cleaner," and comes with the disadvantages of not being as well defined a spec as JsonLogic.
Really interesting though!
Might be heavyweight.
I’m working with a system that relies extensively on logic implemented as json structures. We’re not using this lib, I found it to be difficult to follow and more difficult to debug. Ymmv, but I’d definitely think hard before leaning on something like this.
1) You are leveraging a JSON parser to get strings 2) The strings contain tokens that need to be "parsed" 3) You turn this double-parse into an AST
I see this pattern all too often. If you create (or re-use an existing) grammar that is specific to the problem domain then you get a much faster parse, smaller data transfer/storage, and a more human-readable format.
{"*operator*": [...args]}
{"*operator*": single_arg}
In short, you end up using an object and an array for each element. Yet the object is strictly being used as a two-tuple, and objects are a roundly bad idea for this application: the only way you’re ever going to be looking at it is “it’s an object, confirm that it has only one pair in it, then pull the first key out, and pull its value out”. This is quite inefficient in both memory and runtime performance. The only thing it really has going for it is that it guarantees that the key is a string, which is significantly overrated given the other more expensive bits of validation and access costs that it forces.
So the whole thing would be made more efficient (and arguably even more robust) by just changing it to an two-element array:
["*operator*", [...args]]
["*operator*", single_arg]
["*operator*", ...args]
That’s very probably going to be more efficient on fixed-arity operators (which is most of them) and on potentially-unary operators, though it’s possible that it may be more expensive on larger calls if you need to slice the array during evaluation in order to exclude the operator.
So the end result I’ve suggested here is simpler, more consistent, smaller on the wire and in memory, faster, more robust across diverse languages and environments—in short, superior in pretty much every way. The only real place it might not be superior is auto-formatting, where you lose probable separation of arguments from operator (subjectively often desirable) but reduce the indentation level by one (more strongly desirable). It’s also more S-expressiony!
Unlike wasm, this is the wrong vehicle for it. And this implementation's very-limited nature means that anyone who buys into it will need to rip it out when the limited logic inevitably doesn't scale with the rest of their applications.
One thing code versioning doesn’t do well either is executing multiple versions of the same logic. With rules as data, you can store them as timestamped versions and keep applying the correct historical rule against some older business object or view.
So... exactly like isomorphic/universal JavaScript?
What exactly is the practical benefit in this, compared to say, JS functions shared by serialising them as strings or by abstracting them into a common dependency package which can be ran both on front-end and back-end?
This is still a cool project. It issues a lot of effort to build something like this.
But I wouldn't use it for production until this has been around for a while and json gets comments.
That said, I think everything has uses in some circumstances; there are no rules in software that are beyond debate
This way you can ensure the code is a subset of JS (the same subset of JSON-Logic), whilst using all the tools and runtimes of JS.
I'm doing something similar in a side-project of mine as a temporary stop-gap to avoid writing (or generating) a parser for a proper s-expression syntax.
I remember when I was an university student thinking writing java in XML (Spring) was cool. Man I was dumb.
You think of things like Lua in Redis: this is one take on a not-quite-as-capable alternative for logic that can be safely executed in restricted environments (barring errors in the implementation of the parser).
There will be editing/verification requirements: writing any moderately complex logic structure without a designer/test evaluator would be a nightmare, but as a generic way to specify/embed custom logic in a system that is language independent, has effectively no additional parser requirements (given that just about every single language has a JSON parser that is probably already in use in your project) it isn't a completely bad look. S-expression parsers do exist, but I'd wager many more people are familiar with the JSON parser in their language than the S-expression parser available in their language.
I will say: it isn't complete. It lacks clarity about the behavior in the face of missing/invalid operation arguments. Something like
{"<" : ["ham", 42]}
The playground at https://jsonlogic.com/play.html is a start, but before you try handing this to a non-developer, you'd really want a visual editor with node folding of some kind (so you can hide deeply nested structures away while you work on other things), variable completion (so you specify a sample data object and if you're typing in a variable you get completion options) and operator parameter verification (to stop you from writing in incorrect/unsupported values based on the operator).
"Logic" is lazy word for "code" and we saw what happened when Excel "macros" grew up into major applications.
To be safe under maintenance, safe vs security, etc code needs at least these things: - source control - docs (and ideally, examples) - tests (and ideally, CI)
There you can't do e.g. a bare:
if (a)
if (a != null)
In the same way, where in Python you'd do:
a = ""
if not a:
print "this will be printed"
if not (a == "") -- or if not a.isEmpty():
...
Sure, but this stretches the meaning of "truphy" and "falsy", which was invented to describe the extra coercions to true/false.
A language only allowing explicit true and false just has a concept of true and false, not of truthy and falsy.
I mean, you're free to interpret it how you want but nobody else would agree with your definition.
Do you have a citation for this? I’m looking for a source which doesn’t consider true to be truthy, or false to be falsy, and I’m not finding one.
Basically it's very error prone. It's difficult to remember the rules which are usually quite complicated which leads to mistakes.
Much better just to be explicit.
> How do you even have a language without a definition of what is truthy and what is falsy?
You make it so that `if` only works on `bool` types, and don't implicitly convert other types to `bool`. For example if you want to check if string is non-empty, then you have to do `if (the_string != "") {` rather than `if (the_string) {`, which may cause your code to unexpectedly fail if e.g. the string is "false".