Prolog Web Applications (2016) | Hacker News Reader

Prolog Web Applications (2016)

metalevel.at

104 pointsArneVogel18 comments

Thank you very much for your interest!

Prolog is ideally suited for web applications: HTML and XML documents are readily represented as Prolog terms and can be reasoned about very conveniently and efficiently.

For instance, let's use Scryer Prolog to fetch all article titles from the HN front page.

Put the following in hn.pl:

   :- use_module(library(http/http_open)).
   :- use_module(library(sgml)).
   :- use_module(library(xpath)).
   :- use_module(library(format)).

Then consult the file with:

   $ scryer-prolog hn.pl

and then post:

   ?- http_open("https://news.ycombinator.com", S, []),
      load_html(stream(S), DOM, []),
      xpath(DOM, //a(@class="storylink",text), E),
      portray_clause(E),
      false.

Yielding:

   "Apple announces it will switch to its own processors for future Macs".
   "macOS Big Sur Preview".
   "Most employees of NYT won\x2019\t be required back in physical offices until 2021".
   etc.

Scryer Prolog is a modern Rust-based Prolog system that conforms to the Prolog ISO standard and provides several libraries for web applications: https://github.com/mthom/scryer-prolog. A key attraction of Scryer Prolog is its compact internal representation of strings as lists of characters, making the system ideally suited for the use case Prolog was designed for: Convenient and efficient text analysis, which is also required in many web applications.

Here is a short video about this topic:

https://www.metalevel.at/prolog/videos/web_scraping

Enjoy!

While this was instructive, tying together curl and xpath is not really the sort of thing I'd be worried about when picking Prolog for something like this. Any language looks good when all the heavy lifting is done by other libraries.

How would I run this across multiple cores? Handle exponential backoff for retries? Measure the response time for each submission? Store the result in a database?

As I see it, one noteworthy feature of these libraries is how readily they can be implemented in Prolog itself.

For instance, library(format), library(http/http_open) and library(xpath) that are used in the example I posted are all written in Prolog, using at most a few very basic and general building blocks that are implemented in Rust:

https://github.com/mthom/scryer-prolog/blob/master/src/lib/f...

https://github.com/mthom/scryer-prolog/blob/master/src/lib/h...

https://github.com/mthom/scryer-prolog/blob/master/src/lib/x...

To run the example across multiple cores, we need more support from the underlying Prolog engine, for instance to run multiple threads. In my view, this is a key aspect of designing a Prolog system: deciding what must be provided by the engine, and what is built on top of it. Scryer Prolog is still in its early stages of development. I expect it to provide support for multiple threads and interfaces to external databases in a few years at the earliest. This also depends on how many contributors the project will be able to attract.

When I tried to create a robust, scalable server application with concurrency and whatnot in SWI-Prolog, I realized that I was trying to reinvent the wheel and switched to Erlang/OTP instead.

While Mr. Triska evangelizes for Scryer (which I really hope will succeed in its endeavors), the more mature web framework still is in SWI prolog.

As for the database, this is where prolog shines: prolog is the database. You store data in the internal facts database.

Concurrency is another prolog strength. For example: https://www.swi-prolog.org/pack/list?p=spawn

See also this very nice talk: https://www.google.com/url?sa=t&source=web&rct=j&url=https:/...

As for retries, I have only a blurry idea of what that's about, but from my limited understanding I suspect it's perfectly doable.

I do not know that much about Prolog. Only used it once or twice so far.

Would the internal facts database as storage be as efficient as a typical database, say SQLite or Postgres? What I mean by "efficient" is: Would it consume similar amounts of disk space or memory and would it be as fast in responding to queries with results?

Potentially yes, but current prolog implementations are mostly academic productions with few man-hours invested into them compared to more popular languages. So no, you won't build huge industrial websites on current prolog, but mid-size webapps are certainly within reach as modern prologs are much faster than people usually suspect.

Prolog huge data munging is a thing though. Check the "semantic web" and related for examples.

Hi, yes, I really like Prolog for providing the programming paradigm that is not normally available in a more conventional programming language: I can specify the facts and relations between them, and Prolog will search for a solution for me. That being said, I don't see this functionality to be particularly useful in the context of web programming. Do you have in mind good examples where Prolog will be obviously more powerful and useful than, for example, Ruby, Lisp, and Lua (which can represent HTML elements idiomatically as nested arrays (or classes), s-expressions, and tables, respectively)? Where do I win from using Prolog? (I think that the animal identification game in the linked page, while obviously neat, does not really look like a common web-programming task to me.)

> Ruby, Lisp, and Lua (which can represent HTML elements idiomatically as nested arrays (or classes), s-expressions, and tables, respectively)

Prolog can represent HTML elements idiomatically as nested terms. Unlike in Ruby, Lisp, and Lua you can pattern match and manipulate these terms using unification. I'm surprised you don't acknowledge this as at least as powerful. In fact it's more powerful.

I agree that unification is great, but can you suggest a concrete non-hypothetical example (or better several examples) where this functionality would be useful in the context of web programming?

Sorry, I'm not a web programmer. I am a compiler developer, and I have used Prolog with great success in compiler code: Unification is great for matching patterns in tree-shaped structures like abstract syntax trees representing programs. It should be the same for the abstract syntax trees representing web pages.

I understand if this is too hypothetical for you. The other point still stands, that Prolog's native data model allows you to represent tree-shaped (and DAG-shaped) documents just as naturally as Ruby, Lua, etc.

Is there a decent prolog/SQL bridge? Being able to write applications and queries using SQLite or Postgres database as the fact set for prolog would be very nice.

Wouldn't datalog be more suitable than SQL in this case? And in case the DB you want to query offers SQL only, I imagine there are datalog-SQL bridges available somewhere.

swi has https://www.swi-prolog.org/pldoc/doc_for?object=section(%27p... and there's some more stuff on https://www.swi-prolog.org/pack/list I don't know how great any of them are.

Is there an MVC type of model layer available for prolog?

I would even settle for embedded prolog the same way you can embed SQL. Well I'd prefer the Python connectors to be less compiled.

Perhaps something like https://www.swi-prolog.org/pldoc/doc_for?object=section(%27p...

Thanks, I'll check that out.

Reply on news.ycombinator.com