burakemir

The question is not whether emotions can cause people to be irrational. They can!

Not every case of irrational behavior is caused by emotions though. And when we are making an argument that people are acting against their own interests, it may help to ponder what makes them do so. All the more when we are claiming principles and values that should be accepted by everyone.

"If you don't believe me you are acting irrational / it's because you are emotionally attached" does not seem to be an attitude that gets closer to real causes in a discussion on how to best seek simplicity, but rather a recipe for avoiding discussion or a "thought-terminating cliche."

There must be a better argument for convincing people to let go of code / clean up etc.

Maybe this is also about the desire to create, which of course is also common in engineering. It does not contradict my argument that the cost of maintenance and operations is being ignored eg when one creates things all the time and never removes stuff. And it should be possible to measure or estimate that cost.

Sure, but if we argue that these values and principles should be applicable, then it should also be possible to make an argument why and not blame the irrationality on emotions.

It seems more likely that bounded rationality is at play here, where different parties only know part of the picture (and fail to bring these together and find out what would be best globally.)

While the text touches on many points I would immediately sign, the paragraph starting with "Because engineers are human beings who often form an emotional attachment to their creations, ..." is really out of place.

The cause of complexity is not emotional attachment, these are decisions being made. The decision to add feature after feature and punt on maintenance for example is something that has little to do with emotions. There is a lot of agency that engineers, SWE and SRE alike have in shaping how things are. However there can be good reasons to abandon simplicity. The real trouble here is not psychology but that as a profession we are really bad at measuring and estimating the effective cost of maintenance. Part of that is considering measures to improve simplicity and maintainability as cost that comes without gain and somehow less important than features, and then just accept giant rewrite a few years later. A continuous portion of upkeep would likely be more economical and real engineering has always included an aspect of economy - cost vs benefit.

IMHO the loaded accusation of emotional attachment might be rooted in an "us vs them" attitude (SRE vs software engineering) that should have no place in a sober discussion on the value of simplicity and it diminishes an otherwise great text.

Thanks for the write-up.

Neil Jones' (free) book about partial evaluation deserves a mention here, since it explains how/why it works in some detail.

Query languages are a very cool application of this stuff. I was planning to use wasm for Mangle implementation (a language extending datalog, with its implementation a library for deductive database), following Tiark Rompf and Nada Amin's "a SQL to C compiler in 500 lines of code"... but of course I only have the interpreter for now and started a Rust rewrite instead.

I had done some "research" to find out the meanings of these terms in the past, so it is really nice to see explanations for all of them listed on one page.

My superficial understanding of the space at large is that there are good initiatives and technical solutions / flexible frameworks but progress depends on various non-technical stakeholder organisations (software consumers, eg governments, hospitals, industry) being able to come together to agree on using these.

The complexity is a big obstacle. It is really the same as with software quality standards or digital identity, there is huge untapped potential that comes from a mismatch between what is possible today and what is usable for "society at large." There is still a long way to go.

It is nice to see CBPV on HN. It is a calculus that deal with a fundamental choice in PL of how to approach evaluation. For a different take, here is the first part of a mini series of posts on CBPV that aims at working out the connection to logic ("polarised natural deduction"): https://burakemir.ch/post/cbpv-pt1-small-steps/

Great paper, thanks for sharing! The connection between programming language technology and formal logic, like here for sequent calculus, is one of the most fascinating things.

For anyone who struggles with the references to \lambda-\mu-\tilde{\mu}, the notion that a continuation takes multiple arguments and that "control effects arise naturally", it may be helpful to review the CEK machine first: https://burakemir.ch/post/cek-and-control/

If you are interested, I wrote a thesis in 2007 on "object-oriented pattern matching" available at EPFL which "formalizes" pattern matching in Scala. It did not get into path-dependent types and does not have mechanized proofs, but has a discussion of optimizing translation to lower level code, exhaustiveness checks and user definable patterns (extractors). Like the author of the article, I found Maranget's matrix representation a great basis. This is because algebraic data types come in the form of "sum of products" (or enums with records) and once you have tested one level you want to expand the components of the tuple pattern (columns) and you deal with multiple cases (rows).

I won't claim my thesis is the most readable. I think I would do the presentation very differently today, but going back one's theis topic seems to be one of these impossible things in life if you don't work in academia.

This is a great article describing where web assembly came from, and what promise it holds. It builds up to the pithy quote by Docker founder Solomon Hykes:

"If WASM+WASI existed in 2008, we wouldn't have needed to create Docker. That's how important it is. Webassembly on the server is the future of computing. A standardized system interface was the missing link. Let's hope WASI is up to the task!"

... and ends there. How this WASM-WASI story plays out remains to be seen, but Docker doesn't seem to be the answer. From my (very subjective and biased) point of view, Java gave server programming "extensibility", upload a .jar file without having to restart a server. This convience of updating servlets is something that seems entirely possible with WASM/WASI, but now with safe access to more low-level OS functionality (and maybe on bare metal). I find it an exciting prospect.

You mean datalog or Mangle? Mangle adds a bunch of things that make various things easy to model in the sense that you have entities and connections and data and query that. Maybe one could call this a "knowledge hypergraph".

Datalog is very basic and everyone needs aggregations and structured data so Mangle also supports aggregations, structured data and also some function calls. When you use these it is clearly no longer datalog, but it is easy to see what part is datalog and what part is "more."

To qualify this a little: Mangle does not provide much for mutability (neither the "spec" which is a bit implicit, nor the implementation) so if you want to make a real DB with inserts or an RPC interface you have to code that yourself.

I find having a readable source file and running queries is good for playing around and also may cover many use cases for small DBs with static or slow changing data, or configuration.

Mangle uses seminaive evaluation which is a standard, not very fancy but incremental way of computing fixpoint. I believe Datafun uses the same but there it requires more thinking since it interacts with other language features. I need to reread the paper. There are really many low level ways to optimize incremental fixpoint computation in practice since the key step involves a look up whether a fact has already been computed. However, having evaluation access the FactStore through an interface means a developer can hook up various implementations and that flexibility can matter more than the last bit of performance.

I think it is helpful to see datalog as a formal, conceptual kernel (or "toy programming language" in the famous Alice Book "Foundations of Databases"). When we look at the functional programming languages, we do not usually see is as a dozen of incompatible implementations of lambda calculus.

The kind of standardization that happened for SQL and PROLOG certainly helped spread its use, but it is a very differently world today. Developers can easily do their own take on DBs' data warehouse by serving from memory or existing DBs or files.

If you do not see it as a programming language, but a way to think about computation, then we are in the ballpark of "rule engines:" there are of course innumerable implementations of things that are called rule engines. Like the post, "rules" make knowledge explicit, but we wouldn't even think about the possibility that all the folks who wrote or use these rule engine implementations use datalog syntax. It is more the semantics, structuring the problem as facts and rules, that counts.

Of course having a common syntax helps and matters in getting the message out: that there is a good foundation. But how to add aggregation or user-defined functions is not settled (it is also not settled for SQL, many vendor-specific extensions, syntaxes...) and I think today's business world does not provide much incentives for agreeing and standardizing.

Maybe academia will be able to help over time, by teaching newer generations who will then pick standard syntax for their next PL because they are familiar. For academia to be interested in an applied PL topic, it has to be reasonably formal, derivable from first principles and teachable.

Logic programming offers a good foundation for anything that people call "rule engines." Within logic programming, there is some variation on the degree of declarativeness.

Datalog is arguably the minimal core logic programming, similar to what the lambda calculus achieves for functional programming. Unfortunately, it has been forgotten outside of database and query processing realm. A resurgence has happened in recent years, as PL researchers and also industry have discovered the virtues of datalog (e.g. Flix, DataFun). My own attempt at making this more widely known is here https://github.com/google/mangle, a language from the datalog family and its implementation as a go library.

As the example shows: plain "rules" (or: plain datalog) is rarely enough to capture everything that one wants to express: the question then is, how to combine a pure declarative "kernel" with more general purpose programming (e.g. mapping a list).

PROLOG offered one answer, already in the 1980s, but I fully reject it: the fact that the writing a program in the wrong order with negation and recursion makes it non-terminating is not something we'd want everyone to deal with. Datalog with stratified recursion is somewhat better, as "layers of rules" is a concept that is easy to understand.

In mainstream programming languages, the possibility of writing non-terminating programs also exists, but is rarely an issue. That is why I believe a good combination of declarative and general-purpose has to make it really easy to recognize which parts of a program are in the declarative, terminating, safe kernel and which parts require more attention from the programmer.

I like the part that describes a high level angle of "various ways to achieve security given software development practices."

It makes sense to me that PLs have a role to play and it would be nice if the type system were able to track capabilities.

Controlling access through keys/capabilities can also be useful for memory management. This has been worked out by Crary, Walker, Morrisett "Typed Memory Management in a Calculus of Capabilities" (POPL'99).

If this reminds you of Rust, it should since Rust is a practical implementation of many of these ideas (there were several papers and authors on tracking memory with types).

Back to security: type systems such as the Rust one that are capable of tracking lifetimes can be used for enforcing a capability discipline. An example is GhostCell by Yanovsky, Dang, Jung, Dreyer https://plv.mpi-sws.org/rustbelt/ghostcell subtitled "separating permissions from data"

The permissions in the above references are about read/write access, but one could imagine fine-grained distinction of I/O actions being subject to a similar typing discipline.

Security in type systems has been researched, with Volpano, Smith and others listed here https://en.m.wikipedia.org/wiki/Security_type_system so probably there is more. (edited for typo)

Maybe the view that APIs are little languages can be seen as the view of the postmodern PL community.

Since object orientation brought the possibility of adding little languages to the masses and a few decades later value objects and lambdas and generics where added to all OO languages, a postmodern view may be quite an applicable and useful perspective to deal with this question.

In the end, SQL is what people are familiar with (for better or worse) and what is entrenched in some form in most database, and new versions of the standard are unlikely to change anything fundamental.

I don't believe that adding property graph support to SQL makes much of a difference for SQL and relational databases...

However, the PG query additions (I believe they are close to the Cypher language neo4j) being part of the SQL standard will give a boost to marketing and adoption of PG databases. Vendors can now say "our language follows the SQL standard" instead of referencing cypher and indirectly neo4j. Even if neo4j may have the best intentions, marketing is very important in databases and it is unlikely that vendors could have supported Cypher and thus admit that they are following neo4j rather than leading.

Standards can thus be valuable regardless of whether they are adopted or not and a lot of discussion and collaboration was likely necessary to get to this point re: the PG additions.

In this context, it is also worth mentioning the GQL standard here, which neo4j folks are part of https://en.m.wikipedia.org/wiki/Graph_Query_Language

"... as a joke."

There are two different ways to go about this question:

- asking about the scope of a programming language is asking about the intended programming situations one wants to cover.

- what is a good balance between convenience (sytactic sugar, "features" and abstractions) and asking the programmers to write out things explicitly.

- more subtly: who is using the language, what concepts can be expected to be known to the programmers.

These are of course connected: a particular type of recurring programming situation may lead one to think of patterns and a need to abstract them, whereas to someone who intentionally constrains the scope to a specific area, it can be easier to keep the number and kind of source or "programmer intent" abstractions low.

Often these discussions happen in the context of a general purpose language but there is little consideration that some programming situations are better served by DSLs.

Also a decision on "MIT vs "New Jersey" (worse-is-better) style affects the whole thing, as the notion of correctness is at some level always one of programming situations (a higher level discussion on using the API right way, checking error conditions etc)

Thus is a long-winded way to say: in the end the starting point matters. C++ can be considered a failure language design if you consider it from the readability/cognitive load angle but design choices seem to consistently be about permitting all possible choices. This is why Stroustrup can claim things like "you can write memory safe C++" and not even be wrong. Java can be considered a failure in concision but tool support means that programmers do not pay the cost of writing everything out explicitly.

Haskell and anything in the academic tradition will emphasize abstractions and demand more and more knowledge from users whereas industry tradition will be biased towards low (or lower) cognitive overhead, but if one is willing to specialize, there is an infinite number of nuances of possible points in the design space. We will see more programming languages and come to accept that there will be many that we use at the same time.

So rather than asking how big, maybe we should wonder how to understand the involved trade-offs.

I also think that the talk can be seen as suggesting that open core is the preferable way to make a living from OSS - as long as openness is preserved (service implementations can be exchanged, client can pull out their data etc).

My interpretation of why the talk was mentioned as an argument contra open core: - the talk does have criticism for value extraction that does not preserve openness (as the OP says: promises of interop etc are not kept, lock-in... is it really meaningfully "open"?), and that it is not necessarily sustainable or fair for OSS developers ("who pays the postgres committers"). This latter point certainly gets more nuanced for open core, but can still be relevant e.g. when you depend on some library you are interested in its being maintained, having security vulnerabilities identified and patched etc.

Attempt at a summary (a very good talk!): 1. Open source produces immense value and ironically locks itself out of capturing the value. In the last 20 years, open source has become the oxygen of software development. Cloud providers do what proprietary software vendors did 20 years ago.

2. "IP is free, support for $$" like Red hat is a viable business model but captures only a very small part of the overall value. The much larger part is captured by private sector selling SaaS, Cloud infra vendors. Software has become a commodity, it is user-facing services that capture value today.

3. The value capturing works fine for the companies but little to nothing of that value flows back to the people who write or maintain the software. This needs to be overcome. Openness is important but we need to value it, and governments as biggest buyers of services play a role. If we valued openness more consequently, it would lead to more opportunities for open source developers to capture value by offering user-facing services.

This covers a lot of ground! The author is interested in concurrency, in particular coroutines. They follow a standard approach of PL research to extend a lambda calculus, discussing typing and compilation. This will seem academic but there are certainly many PL researchers who have deep expertise in compiler design and implementation and who also deeply care about communicating these ideas in a precise, formalized manner. To put this in perspective: even after so many years, decades of research, industry and hobby projects turned million of professional users (before thinking about a type system)... the question of exposing concurrency to programmers is far from settled even if it is so important. There are many approaches and different perspectives on what matters more. Formalizing these approaches has likely been done before, but in practical terms we don't care about abstract properties but about performance. That is why I like the article, implementation concerns are important, but the ability to fully specify behavior is also important. Keep it up.

Agree but also want to point out that people usually have a narrow view on "logic programming". Datalog can also be understood with out the top-down evaluation / resolution that is typically associated with prolog, which is why it is known to database researchers and in finite model theory. Prolog is great, but bottom up techniques to evaluate datalog are awesome, too, and would arguably also qualify as logic programming. It is rare to see this acknowledged.

(Mangle was posted recently which I like so I am a bit biased)

Datalog is a formalism like relational algebra but additionally supports recursion.

So it can roughly play the role of SQL. Compiler writers have used it to query their symbol tables and identify patterns for example (seems to be the origin of Soufflé, "static analysis").

I believe the reemergence of datalog is at least in part due to compute and memory being more plentiful so it is more affordable and practical to use a declarative query language. A lot of databases (or the part that is relevant for answering a particular query) comfortably fits in a single machine's memory and an expressive query language is fun to use. There is also an incremental evaluation story.

Another potential reason is that the logical data model makes it easy to represent any kind of data without schema changes and such.

A lot of programs end up either doing SQL or something SQL-like. There are therefore many applications of such datalog like languages.

In many places where people use custom "rule languages" they could use datalog instead. (edited: typo & rule languages)

Mangle is used in an internal application. That application is a staffed project with internal usage.

In mid 2021, I saw an opportunity to apply some ideas on query languages to support development of that particular application. That application was an early idea and a prototype back then.

It was clear that it would need integration with many data sources and that this data would need to be queried in various ways that were not fixed in advance and would evolve. A very common situation...

With the team, we decided to try a novel approach using datalog, and it did indeed help development. Soon they started asking for features, found my bugs, optimized a few things and pushed for design changes.

As the programming language person, I tried to keep internal consistency of the language design and ease of use (static checks!) and wrote most code, yet Mangle certainly would not exist without this team, the application, it's users and management support at various levels.

I insisted on keeping the language as a separate thing since I had these other uses in mind. Open sourcing was therefore not difficult. For the mentioned application, Mangle is merely a mechanism, a library that enables flexible access to data.

So Mangle is not my or anyone's main project, yet many people (including management support at various levels) contributed to it's development.

I am glad you find it interesting! I feel very privileged to work with the larger team of people whose efforts led to this opportunity to do this kind of applied language research.

RDFox using datalog underneath is definitely interesting. Maybe worth making a distinction between products/services and open source projects that can be used as building blocks for such products/services or whatever custom setup one may have.

Thanks for sharing Biscuit, I was collecting examples of authentication policy languages.

Datalog is also the basis for Open Policy Agent https://www.openpolicyagent.org/docs/latest/ , more specifically it's Rego language which is also implemented in go https://github.com/open-policy-agent/opa/tree/main/rego

Yes, there are CTE and recursive queries, and various DBMS also offer views. There are even table-valued functions.

These things are not widespread, and differ by implementation, and the way these are used by clients are copy-and-paste. Something as thoughtful as ZetaSQL https://github.com/google/zetasql does not have mechanisms for structuring (modules, packages, interfaces). SQL will not, cannot evolve into such a direction (or, anything that evolves, will not be recognizable as SQL).

It was a conscious choice to stick the prolog-like syntax because there are numerous teaching resources (e.g. "foundations of databases") and also academic research papers available that all discuss datalog in this syntax.

Like tmptmpgo I consider datalog something different entirely and worthy of study in its own right; a small kernel (or toy) language that by itself is not sufficient for many day-to-day tasks.

It is enough to cover relational algebra with recursion fixpoints. This gives us a handle on SQL queries. What we want is a well-behaved, predicable mix of programming and those relational operations. We have to end this SQL madness.

IMHO, it is undeniably true that Prolog is one particular way of achieving such a mix (and standardized etc), but it is not the predictable, easy-to-use and well-behaved mix we want. We should be able to isolate syntactic fragments where it is easy to ensure safety properties, at the minimum the datalog fragment. That is why having a fragment correspond as close as possible to textbook Datalog was a design constraint for me.

That said, when we talk implementation, there are of course a lot of optimizations that apply equally to Datalog and Prolog, say the magic set transformation.

This is my project! So soon ... I was considering a Show HN but wanted to wait until things like documentation are a bit more complete, but here we are.

Ask me anything.