Scryer Prolog: A modern Prolog implementation written mostly in Rust
github.com
github.com
Notably, Scryer Prolog is currently the only free Prolog system that aims for full conformance to the Prolog ISO standard, supports constraints such as dif/2, CLP(B) and CLP(ℤ), includes the most recent declarative features such as if_/3 and and the ..._si/1 family of predicates, provides support for sound unification and an alternative execution strategy called SLG resolution, and also implements a very efficient internal representation of lists of characters and partial strings, thus making it the first Prolog system in decades with the potential to efficiently support the use case Prolog was developed for: efficient and convenient text processing. On top of this, it is the first Prolog system ever implemented in Rust, making it especially exciting for fans of this language while also benefiting from many of its innovations.
Here are a few interesting applications of Scryer Prolog:
The precautionary package by David C. Norris, using Scryer Prolog for safety analysis of phase 1 dose-escalation trials in clinical oncology:
https://github.com/dcnorris/precautionary
A Prolog library to connect to PostgreSQL databases by Adrián Arroyo Calle:
https://github.com/aarroyoc/postgresql-prolog
The Retina chainer, targeting EYE Reasoning via Scryer and Trealla by Jos de Roo:
https://josd.github.io/retina/
Thank you Mark and everyone who contributed for all your work on Scryer Prolog, keep up the great work! Some of these innovations have already spread to other Prolog systems, such as Trealla and Tau Prolog, and I hope that Scryer serves as an inspiration for many more systems. Exciting times ahead!
Regarding the name of the project: "Scrying" is the practice of looking into a suitable medium in the hope of detecting significant messages or visions:
https://en.wikipedia.org/wiki/Scrying
What a fitting name for a Prolog system!
Is that a result of the Standard and will Scryer follow those footsteps or are we going to see something closer to Mercury?
I really enjoy your channel and indirectly heard a lot of positive things about Scryer, so of course I'm super interested and would prefer if they don't get too stuck on trying to meet the "status quo". The "status quo" has a lot of room for improvement.
The Prolog ISO standard currently prescribes less than 200 predicates (less than 150 if we discount variants that only differ in the number of arguments), here is a nice overview of all predicates that a conforming system must provide:
http://www.complang.tuwien.ac.at/ulrich/iso-prolog/prologue
The ISO standard also allows conforming extensions that any Prolog system may add as implementation specific features. Notably, we have:
5 Compliance
5.1 Prolog processor
A conforming Prolog processor shall:
a) Correctly prepare for execution Prolog text which
conforms to:
1) the requirements of this part of ISO/IEC 13211,
and
2) the implementation defined and implementation
specific features of the Prolog processor,
etc.
The general trend and intention of modern Prolog systems is clear: A major goal is to add, as conforming extensions, declarative constructs that facilitate more general and readable Prolog code that is also amenable to logical reasoning and automated debugging. However, the extensions that SWI-Prolog provides are not conforming extensions, because they change the meaning of existing, conforming Prolog code in a way that deviates from the ISO standard. The following document provides a good overview of resulting consequences:https://www.complang.tuwien.ac.at/ulrich/iso-prolog/SWI7_and...
Note that it is possible to add infix dot syntax, efficient strings, constraints etc. all while conforming to ISO Prolog! This is the route taken by GNU Prolog, Scryer Prolog, SICStus Prolog, Tau Prolog, Trealla Prolog etc.
You mention you are a contributor to Scryer Prolog. I am curious about the choice of Rust as the implementation language. There are lots of advantages to using Rust but it does not lend itself easily to self referential data structures, linked lists, recursion (tail and non tail recursion) and other things that are found abundantly in Prolog.
I'm wondering if Rust is the easiest language to implement Scryer Prolog in. I would have much preferred using something OCaml (or similar). You would get garbage collection and tail recursion "for free".
What is your opinion on Rust as Scryer's implementation language?
Scryer Prolog compiles Prolog code to abstract machine code, and executes that. Like SICStus Prolog, GNU Prolog etc., Scryer uses the Warren Abstract Machine (WAM) as the target architecture. Therefore, self referential data structures, linked lists, tail calls etc. are not directly mapped as such to Rust data structures and concepts, but occur in a sense only "indirectly" when Rust interprets the virtual machine instructions, where references to data structures appear as pointers to cells in the virtual machine. Similarly, tail calls correspond to certain abstract machine instructions that allow reuse of portions of the virtual local stack.
Currently, Scryer uses a Rust enum to represent heap cells, here is the definition:
https://github.com/mthom/scryer-prolog/blob/ca62e54652ace837...
A more compact representation will use tagged cells that point directly into memory, using for example the modular-bitfield crate:
https://github.com/Robbepop/modular-bitfield
It is my hope that such an improved representation will soon become available in Scryer Prolog, it will likely speed up the system considerably.
On the use of Rust: I contributed only a few hundred lines of Rust code to Scryer Prolog. My feeling when using Rust was that this is a suitable final form of the implementation: Rust seemed so suitable and natural to implement the low-level facilities of Scryer Prolog that I consider it unlikely that any other language will ever warrant a change in this basic layer of the implementation. It is this feeling that I strive for when using programming languages, and I get it also for example when using Emacs Lisp, PostScript and Prolog. OCaml and possibly other languages would definitely also be a good choice, and likely candidates that satisfy this specific criterion! Rust's package manager, the availability of ready-made crates and the easy compilation were also extremely helpful during development. It is my hope that Rust will be especially useful when adding support for multiple threads to Scryer Prolog.
Even if you don’t use it as your principal application language it is a great addition to the development toolbox.
Running in reverse is great for testing, where you can generate test cases that hit interesting variant flows.
For example, I have used it to generate input pension payout requests that would hit the interesting special cases for the bespoke bookkeeping rules for each tenant or subsidiary like “for X, the third dimension is always 930, but for Y they have 930 or 931, but then the fourth dimension is 4,6 or 10 depending on the historical subsidiary of the account origin…”
Another great use of logic programming in the daily toolbox is with respect to compliance documentation where you want to document and verify how data points flow across systems or the variables going into aggregates calculated on the way.
This allows you to answer questions like, “our provider for X data delivered bad data for last month, which customer transactions may have been impacted?”, or “this value looks wrong, where did it come from?”
It also allows you to generate exact documentation for this, or even checking the model vs the actual APIs, schemas or data flow or change notifications.
* There’s also lambda prolog which enables higher order logic (set logic).
I hadn't considered that. I've always done bespoke analyses for this.
Do you have any suggestions for a practical intro for someone that wants to add Prolog to their toolbox? I'd love to learn not just how to use it but also how to know when to use it.
I think you have to learn it and get some experience before you can know when it's useful.
FWIW the only dev I know that had any experience with Prolog told me that it was too slow so his job was to rewrite the program in C. He said it was basically a big loop with ifs and elsifs...
It's quite impressive and fast. It can auto fix all your package.jsons too and is quite fast at it! Being completely new to prolog, I couldn't figure out how to write a not member of list predicate that would work but other than it saved me a days worth of work!
The Polonius reimplementation of the Rust borrowchecker uses Datafrog: https://github.com/rust-lang/polonius/
Prolog uses backtracking search. Backtracking search is quite a good and generic approach. Its not the only approach. Depending on your problem space you can probably totally avoid Prolog.
"Can this user do this thing given these constraints?" or even "Show me all the things this user can do given these constraints"
Our team is solving for this use case. While we focus a lot on abstractions, APIs, etc. that are appropriate for our use case, under the hood everything is driven by an embeddable logic language that we write in Rust, called Polar: https://docs.osohq.com/reference/polar.html
In principle, this would have allowed us to do something akin to property based integration testing (we only tested the fully integrated system, no partial integrations of subsystems or unit testing). The same program could have been used to analyze recordings of real world communications to see if they were correct (to the spec) and where they diverged if not, with only minor modification to the way we interacted with it.
That's actually one of the really nice things about Prolog, you can work in both directions with the same program, you can't do that in any of the mainstream programming languages without libraries (minikanren, for instance, allows for an embedding of this in many languages).
For instance, while trivial, you can use append/3 in Prolog to get the result of appending two lists, or to find all prefixes and suffixes of a list, or to find a particular prefix/suffix given both the combined list and the other (suffix or prefix, respectively).
append([1,2], [3,4], AB). -- AB = [1,2,3,4]
append(A, [3,4], [1,2,3,4]). -- A = [1,2]
append([1,2], B, [1,2,3,4]). -- B = [3,4]
append(A, B, [1,2,3,4]). -- A = [], B = [1,2,3,4];
-- A = [1], B = [2,3,4];
-- ...
By writing one program, perhaps supplying synonyms to get something that reads better, you can get all these options depending on which parts you choose to supply as knowns and which you leave as variables.That is, trying to converge a system from current state to desired state.
If it were written with logic programming and were able to run things in reverse, then it would be able to explain exactly why it is unable to converge the system to the desired state, and what other factors it is waiting on to do so. I think that would greatly enhance observability in a complex system.
I was also in the process of writing a chat bot accepting commands be able to autocompletion that takes authorization into account. So given what the user has typed so far, and what it is authorized to use, it would be able to show the available commands.
When I need to solve a CP problem, I typically do one of the following
* Write a simple model in MiniZinc. This is great for experimentation and testing different solvers to see which kind of solver might be suitable (different CP solvers, MIP solvers, etc)
* For some cases or if I need to integrate with a larger system I write my code using a library directly, such as Gecode, Choco, or or-tools. Using a system directly has the benefit that it is possible to program a custom search heuristic, which I find is often very useful for solving problems in practice.
* Write a custom algorithm. Sometimes the right answer is to just write some code solving the problem directly.
I find models written in a modeling language such as MiniZinc or in a traditional Programming language such as C++, Java, or Python to be easier to read and reason about than Prolog programs that solve constraint problems. That is a personal preference, and varies depending on familiarity.
- FFI to C or Java
- “unique”, i.e. linear type declarations
- modes, i.e. data-flow direction-dependent overloading of predicates
These are my favorite features in Mercury Prolog [1], and I would be delighted to find them in Scryer too.
roo(X, W) :- zut(X, Z), pum(Z, Y), tos(Y, W)
we could write something like roo/2 :- zut . pum . tos roo --> zut, pum, tos.
Note the use of (-->)/2 to define a grammar rule. Use phrase/3 to invoke the DCG. DCGs are currently being drafted for inclusion in the Prolog ISO standard:The only reason I could see ReasonML being called "modern" compared to OCaml would be because of the C-like syntax, but that syntax is very old and would be the opposite of "modern" considering age only.
The main issues are:
- The language is untyped. In some ways you get more power because it it untyped but then it becomes more difficult to build/understand/maintain/refactor larger programs
- Debugging the language is quite difficult. There are some really interesting videos/resources at the "Power of Prolog" [1] that give you tips on how to debug. IIRC many of those tips only apply to "pure" Prolog (a certain class of Prolog programs that don't do things like modify global state etc). Essentially you can keep making your Prolog goals more and more general to home into the bug.
But to answer your question, I am not aware of any "ReasonML" equivalent of Prolog. ReasonML just differs on the "surface" level with OCaml. The internal type system, compilation pipeline etc. is all OCaml. ReasonML as a project makes sense for the OCaml ecosystem because OCaml is typed language and has a sophisticated syntax. Prolog's syntax is SUPER simple. So its not really possible to have too much of a different take on syntax when it comes to Prolog.
In Prolog essentially we have
Head :- Body
This could become transformed to:
if Body then Head
In other words, there is not much room for syntax innovation as there existed in OCaml when ReasonML came out as alternate (surface) syntax.
You have other languages like Mercury which try to be a typed version of Prolog. They are significantly different languages in their own right so I wouldn't classify them in the "ReasonML" equivalent category.
[1] https://www.metalevel.at/prolog by Triska (I highly recommend this resource)
Examples in other languages include adding classes and objects to Pascal with a handful of keywords or adding lambda functions to C++ and to Java with recycled punctuation.
You could add a couple of communication primitives and have Erlang.
But...it’s been done.
I would be interested in becoming used to identify problems that can be solved with Prolog.
The last one is 3. August 2020 and for a while they were getting tagged regularly but now it's been 500 commits with no tag?
Sorry if my question can be answered in the issue tracker I was just curious..
Edit; ah indeed https://github.com/mthom/scryer-prolog/issues/640
I think advertising the language is useful in two contexts only.
- it’s JavaScript (so available on the client end) - it’s a library FOR x_lang.
I mean — a mini-kanren written in Prolog would be droll, as would a REST API framework for NetLogo, but anything that’s useful is useful.
Compiling prolog to abstract code and running it on a VM written in Rust? Yeah its probably close to the language's target use cases of Rust but it lets people see if such functionality is reusable as a library (in this case found out its not) and how its implemented.
I don't know what's with all the Rust hate these days. It's just another language nothing to get fired up about a tag line appended to the title.
https://github.com/mthom/scryer-prolog/blob/master/src/lib/c...
Note that many of these predicates are only wrappers for internal implementations that use the Rust crates. In fact, these existing Rust crates (notably ring) made it especially easy and convenient to add this functionality. library(sgml) is another example of this approach:
https://github.com/mthom/scryer-prolog/blob/master/src/lib/s...
On a general note, it is often an advantage to implement functionality in Prolog, because it helps to improve robustness and efficiency of the underlying engine, and makes changes easier.
For me, SWI Prolog has always been the elephant in the room because there is a ton of good software written in SWI. That said, since it is such an ancient system (but still under active development and supported), I am not surprised if there isn’t cruft/technical debt in its implementation.
No, Docker is not a substitute.
A Prolog meta-interpreter is a Prolog program that interprets Prolog programs. With this facility, you can dynamically extend your programs, by reading code and executing it as if it were part of the original program. But not only that, you can also reason about it, and execute it in different ways than a regular Prolog engine would do it. You can analyze it, and execute only safe parts. You can remove fragments, and show explanations why specific phenomena occur. You can debug it declaratively by constructing generalizations and specializations. You can add features to the language that are not present in its actual implementation, but only in the programs you interpret.
No other programming language provides this ability to such an extreme extent as Prolog. For example, how much C++ code would it take to write an interpreter for a sensible subset of C++? How much Lisp code would it take to interpret the most essential aspects of Lisp? It takes only 3 lines of Prolog code to interpret the essence of Prolog.
Most programmers may currently not "miss" this ability, because what they think of when they think of a "program" is in comparison quite static: The idea that they could interpret external code dynamically as part of their programs is alien to them because the languages they are using either does not allow it, or only allows it in a sense "blindly", by not truly looking into the code and interpreting it in flexible ways, or by making the analysis of given code too hard. Yet, the ability to conveniently adapt and augment the logic of an existing program is a fundamental necessity for truly general and flexible programs that must dynamically take into account changing requirements.
Just: (eval code).
Prolog is a declarative, logic language - we tell the compiler what the goal is, and the compiler works out the steps to take in order to complete that goal.
:- use_module(library(clpfd)).
valid(Num) :-
Num in 100..150,
Num mod 5 #= 0,
label([Num]).
# Python
def valid(num):
return (num in range(100, 151)) and (num % 5 == 0)
Then testing it, `valid(120)` is true, `valid(121)` is false. What the Prolog one does that the Python one doesn't do is if you pass in a placeholder, it can search for numbers that fit the cosntraints and output them, which is cool and a different focus on the problem and what the code exists to do. The Prolog ideal (not always easy to attain) is that you describe the patterns and constraints around the solution and Prolog finds the solution.len(some_list) in Python can tell you the length of a list. length(Somelist, Len) in Prolog can tell you the length of a list, check if a list is the given length, generate a list of a specified length containing placeholders, or generate lists of increasing lengths containing placeholders. All languages can do what all other languages can do with enough effort; Prolog brings to the foreground relations and constraints and patterns instead of state manipulation and variable assignment.
In Jim Grays 1984 article about getting to "1000 transactions per second" (TPS), an "ordinary" system ran at 50 TPS and airlines were pushing above 800 TPS.
So, they had capacity and competence.
Second, airlines are expensive beasts to run so solving hard problems is worth a lot of money.
Third, there are a lot of rules in aviation that suit themselves nicely to combinatorial optimisation.
Think crew scheduling: if there is a delay on an inbound flight the crew may not be allowed to fly the aircraft on its next leg since they would exceed their maximum working hours for the day. But then you need to find another crew and a good rescheduling to perhaps a shorter or earlier flight for the original crew.
And then you have to think about rescheduling passengers to their connecting flights or holding those flights back a bit to allow the passengers to make the connection. And you have twenty minutes to do it before they land.
Handling these "catastrophes" pop up again and again in the daily operations and are well suited for the kind of combinatorial search provided by logic programming in combination with optimisation engines.
There are specialist companies in this area. For example, PDC's Turbo Prolog and Visual Prolog products have been used extensively in aviation since the 90s.
Since Prolog is a logic-based and declarative programming language, if you're dealing with goal reduction or backwards chaining, then Prolog is a excellent fit and will probably be easier to both write and maintain compared to other languages. Theorem proving, automated scheduling and querying data stores are also excellent use cases for Prolog and Prolog-like languages.
For example, "I already have Machines A and B with control system S, but now I want to add Machines C and D from another vendor and I need the control systems to work together, what are my options?"
It would query the configuration rules about products and compatibility with the 'knows' that you tell it and find options for the 'unknowns' and tell you something like, "then you can buy X, Y or Z, but if you chose Z you also need to buy an external temperature control from the Foobar 800 series with the D-bus adapter".
In practice, it is often even more complex.
Logic programming is great for this, and I find that using it in small subsystems of a larger application is a super powerful tool.
For interactive systems, a fast implementation is really important. The CLP aspects of Scryer Prolog would also be very helpful.
For example, it is very useful for e.g. generating test cases to hit every state transition in a complex business process state diagram.
There are numerous examples where Prolog itself was used, for instance, every instance of Windows NT contained a Prolog implementation to handle the details of network configuration.
items = fetch_list_from_db()
for item in items:
if item.match(param):
..
elif item.whatever > param2 || item.override:
..
else:
some_default_behavior()
return custom_sort_thing(results)
In each instance, the logic is critical to business function and frequently evolving, but often underspecified, hard to audit, and each case has its own little missing features.Prolog allows me to very easily state these rules in a way others can understand at a glance and keep them separate from the operational code. Best of it all, it solves for all possible answers (or all inputs for a given answer) without implicitly stating very much looping or recursion - even less so than functional languages in some cases.
EDIT: IMHO one should learn it just to get introduced into the mindset of declarative programming. You can implement a sufficiently fast and complete prolog easily in any language if that allows for a clearer solution to a specific problem than explicit specification of control flow.
Has anyone actually replaced one of the major ones (i.e. Java, C++, Node, C#) in a large production project with it yet? What's holding people back?
Though I would say it’s easier that C++ to deal with and would expect this would help it win a larger share in systems programming. Eventually
I don't think that's true, at least not in a way that matters in practice. Sure, from time to time there's an article about a micro-benchmark in which carefully optimized C++ is 10% faster than carefully optimized Rust. Well, my users don't run micro-benchmarks, they run applications, and with Rust it's much easier and faster to experiment with macro-scale optimizations, which have a real impact on performance.
Case in point, recently I took a single-threaded CLI program and made it multi-threaded by spawning some threads and adding a work queue. There, instant 300% speedup, just add water. Yes, I could have done the same with C++, but it would have taken much longer to make sure I didn't introduce some concurrency bug in my code – and only on the condition that the libraries I was using could be used from multiple threads simultaneously, which is far from certain in the C and C++ ecosystems.
What makes you think this is the case? That's not my impression at all (for reference, I develop software in Rust professionally).