Fine-Grained Language Composition: A Case Study | Hacker News Reader

Fine-Grained Language Composition: A Case Study

soft-dev.org

43 pointsmatt_d13 comments

This is great. I often miss the asm blocks of C, and think the benefits of mainly working in a higher level language and dropping down to a lower level language for performance reasons have been lost in current times (mobile apps - I’m looking at you). We've also been distracted by very hands-on approaches to composition like language binding and writing interfaces which require so much boilerplate when the interpreter could do the heavy lifting for us.

Don't lecture me about performance when computers today are on the order of 1000 times faster than when I learned programming, and when even non-JIT scripting languages are 100-200 times slower than native code, we still have an order of magnitude speedup over native apps of the 80s. IMHO all that matters today is developer time, which is being squandered by languages like Swift and Rust that encourage a trees over forest view of productivity. My favorite non-mainstream languages are HyperTalk (AppleScript, ActionScript) and MATLAB (Octave) which provide tremendous leverage in few lines of code. Or better yet, code-less functional environments like Excel or FileMaker where you only drop down to a macro language when necessary. Or even better that than, declarative environments like the web that were built with relational data models and markup before AJAX opened the floodgates to callback hell and all the workarounds since (why the web lost WYSIWYG editing is a profound tragedy to me).

This turned into a rant but I simply don’t think code is supposed to be the way it is today. We’ve made a mistake. Yes, it has advanced computer science remarkably. But the opportunity cost of that is that we spend our days writing boilerplate and dealing with nondeterministic distributed computing issues informally rather than working at an appropriate level of abstraction.

Yes, yes, yes! I despair at the lack of options for Rapid Application Development. Web development is a problem that has been solved a million times over; why is it still so hard? (Sure, you could say browsers, scaling, and interactivity all need special consideration.)

I believe there are certain personalities drawn to software development, who are very detail-oriented and love technical challenges, to the detriment of the end result - forest vs trees. Just look at people who want to make games and end up writing a game engine, vs people who jump into Flash and make do with the tools available.

Personally, I would much prefer to just feed in a specification, or set of constraints, to a program, and have it spit out a working program. Program synthesis seems like an interesting area of research, but fairly basic at the moment. I imagine any useful synthesis will be domain-specific - generating a JSON web service/API will be quite different to writing a music visualizer.

Great points, but, rapid website development is a solved problem with templated builders. The problem comes when we need a more complex site/app than something Wix or Weebly or Wordpress can provide.

Then in current practice, it's often back to square one, but it shouldn't be. We have opininated frameworks like Rails and Django that save a lot of time, but only cover the featureset of a CRUD app. There is a lack of opinionated frameworks that work at a higher level, but still below that of drag-and-drop templated website generators.

My first thought is: Neat! Languages with different grammars, syntax, and libraries have different expressivities; my first thought is always compiler implementation in OCaml. I would love, for example, to (effortlessly!) dip into OCaml or Haskell while writing a high-performance C++ program, for example.

I am very intrigued by Lia [0] as an example for how languages can be embedded inside of one another for useful effects. See Will Crichton's great blog post for more [1].

However, I am wondering why Python and PHP were chosen. I can't think of a compelling use for having both languages simultaneously. They are both dynamic scripting languages with similar design objectives.

[0] https://github.com/willcrichton/lia [1] http://notes.willcrichton.net/the-coming-age-of-the-polyglot...

From the article:

> Depending on how one chooses to classify programming languages, Python and PHP can appear similar—most obviously, both are dynamically typed. From our perspective, however, there are a number of tricky differences: PHP has multiple global namespaces which can span multiple files, whereas Python uses one global namespace per file (semantic friction); most of PHP’s core data-structures are immutable, whereas many of Python’s are mutable (semantic and performance friction); and PHP’s sole collection data type is a mapping, whereas Python separates the notion of mappings from that of sequences (semantic and performance friction). As this may suggest, this combination of languages presents a number of design and implementation challenges which have no obvious precedent. We show that PyHyp’s solutions to these challenges allow interesting case studies to be implemented.

Why Python and PHP? There are several reasons, but the major ones were: we had to start somewhere; we had an excellent Python interpreter available to us, as well as a fairly decent PHP interpreter; and PHP and Python turn out to be rather different languages with a number of tricky challenges. We certainly look forward to other people composing together even more distinct languages (e.g. we've also done a composition of Python and Prolog http://goo.gl/p1opSl, though, compared to PyHyp, it is rather simplistic).

I'd love to see Python and R, the top two data analysis languages. Obviously this will only appeal to a subset of users, but language interoperability for data work is a hot issue right now.

Yeah I would love to see this too. Some differences:

- R has lazy evaluation semantics (with caveats), and Python is eagerly evaluated. The ggplot library in Python recently posted to HN sheds some light on these issues.

- R has like 2 or 3 class systems; Python has a C++-like class system (without static typing, but gaining it in Python 3)

- They differ in semantics with respect to closures (Python 3 changed things a bit)

- R's built-in types are all vectorized, but that might be a good thing, so you can use Python semantics for scalars and R semantics for vectors/data frames/matrices, etc ?

- Python has decorators, generators, coroutines, etc.

I tend to write my Python and R in a pretty small common subset, but yeah they are in fact quite different.

A crude R/Python composition would be fairly simple (http://goo.gl/p1opSl shows that a strict and lazy language can be crudely put together pretty easily, though you miss good performance and all the programmer-friendly features of PyHyp). Closures/generators are unlikely to be a big deal (PyHyp has good suggestions for both). Although I don't know much about R's class system(s), I expect that we can probably do OK on those. However, I have no idea how R's vectorised types might be handled -- those could be painful to deal with, or they might just fall out of the hat, and I'd have to know more about them in order to make an informed guess. However, this isn't on our roadmap at the moment, as it doesn't fit in with our current funding, unless anyone wants to change our minds!

Wow. If you can combine Python and Prolog, I can't help but feel you can combine anything. Those are pretty different!

It's possible in OCaml to use a preprocessor (eg. camlp4) to integrate another language at compile time. I did this with PG'OCaml (combining Postgres's SQL & OCaml). At compile time the SQL statements are sent to the DBMS to be "described" (syntax and type checking), and then we insert the necessary glue code to prepare the SQL and convert OCaml values to and from SQL columns and results. The result is type-safe across the two languages, and free of SQL injections. You literally cannot write non-well-formed or badly typed SQL and have the program still compile.

http://pgocaml.forge.ocamlcore.org/

What happens if somebody does an ALTER TABLE changing a column type? I think it's fair to just not deal with that case, but I'm curious if you did more than that.

Then you'll get a runtime error. However if you recompiled before running the program you'd get a compile time error, so it's probably best to recompile your source after any schema change, although bad stuff won't happen if you don't do that.

Reply on news.ycombinator.com