Hacker News Reader

Comment by ajb | Hacker News Reader

It's weird, but the pain of moving to a whole new language seems less than the pain of using one of these language subsets. I guess it might be because of "loss aversion": we dislike losing something we have more the value we put on gaining it. In a new language each newly added feature is a gain, but in a language subset you're always encountering things you would have in the full language. So no matter how good the rational case is, there is a bias against them. I can't think of a successful one actually - anyone?

View on HN

The author seems to speak to that here:

> SPy does something different: on one hand, it removes the dynamic features which make Python "slow", but on the other hand it introduces new features which make it possible to implement and use the same pythonic patterns which we like.

The author seems unusually focused on explaining his new way, which may help it feel more like a new language with a coherent philosophy and unique benefits in its own right.

Not unlike how Crystal isn't really "compiled Ruby" but rather its own language that just looks and feels similar to Ruby.

> in a language subset you're always encountering things you would have in the full language

Exactly. I once pitched the idea of a Python subset (with a different focus, not performance like SPy), and almost every reaction was "will it support <favourite Python feature / favourite library>".

For example, a new language can build its own solution for array math, or maybe that's not something its users need. OTOH many consider a Python subset to be unacceptable if it doesn't specifically support NumPy.

In the end I came to agree with https://pointersgonewild.com/2024/04/20/the-alternative-impl...:

> positioning your project as an alternative implementation of something is a losing proposition

> don't go trying to create a subset of Python [...] Do your own thing. That way, you can evolve your system at your own pace and in your own direction, without being chained by expectations that your language should have to match the performance, feature set, or library ecosystem of another implementation.

I can see the wisdom in this.

As a counter-example, it feels as though Typescript has managed to (largely?) succeed as a subset of Javascript, so maybe the "do your own thing" isn't entirely a lost-cause -- maybe it's just really really difficult?

Typescript is a superset of javascript

Aaah, you are correct -- I had that backwards. All Javascript is indeed valid Typescript, but not all Typescript is valid Javascript.

Appreciated

I think Python is a bit special in this case, as many people tend to use it more like duct tape for connecting different libraries (which often contain compiled and optimized code) rather than a language for doing stuff bottom-up.

One of the standard architectures for complex applications is to put together a scripting language and an systems programming language. You can either look at the scripting language as primary with the systems language used for performance or hardware interfacing or you can look at the systems language being primary and the scripting being used around the edges.

The model of having a closely related scripting and systems language would be an optimization of this and SPy seems like an answer.

[1] AAA games that have a scripting engine on top of C/C++, a finite element solver that lets you set up a problem with Lua and solve it with FORTRAN, etc.

Yes, there are quite many real-world cases of this architecure. But, wouldn't it be better it the same language (more or less) can be used for both? I don't think such a language exists currently, but I think it would be a nice goal.

Oxcaml, of course.

Interesting, I was not aware of OxCaml. (If this is what you mean.) It does seem to tick a few boxes actually. For my taste, the syntax is not as concise / clean as Python, and for me, it is "too functional". It shares with Python (and many high-level languages) the tracing garbage collection, but maybe that is the price to pay for an easy-to-use memory safe language.

In my view, a "better" language would be a simple language as concise as Python, but fully typed (via type inference); memory safe, but without the need of tracing GC. I think memory management should be a mix of Swift and Rust (that is, a mix of reference counting and single ownership with borrowing, where need for speed).

What would be the point of the second language if it was basically the same language as the first?

In general you want a solid engineering language with a focus on correctness, and a second "scripting" language focusing on quick development.

Second if the languages are very similar but not the same it seems like you would see "confusion" errors where people accidently use lang 1 in lang 2's context or vice versa.

Common Lisp

The most interesting pairing I've been thinking about is Rust + RustPython

It's a full python interpreter implemented in Rust. Python has many advantages as a scripting language but the problem is you inherit all the packaging and deployment challenges unlike with Lua for example

Throwing my hands up and moving to Nim was downright easy next to the excessive effort I put into trying out Nuitka, Numba, and PyInstaller for my use case. If you want static compilation, use a language and libraries built with that assumption as a ground rule. The herculean effort of building a half-compatible compiler for a dynamic language seems like a fool's errand, and would be a fun curiosity if so many people hadn't already tried it, especially with Python.

It's so common for something like this to struggle for decades before succeeding.

For instance people have been trying to make a memory safe C for a long time, just recently we got

https://fil-c.org/

which has high compatibility and relatively good performance for that kind of thing. The strength of Python is that so many people are trying things with it.

all of this is well and good if you completely forget that there are billions of lines of Python in prod right now. so your grand epiphany is basically on the level of "let's rewrite it in Rust". i'll let you hold your breath until that rewrite is done (and in the meantime i'll explore workarounds).

I kind of object to this take.

Nobody's talking about porting billions of lines of code, for all we know it's just for personal projects, or a learning experience.

This kind of replies is like killing an idea before it's even started, smells like the sunk cost fallacy.

OTOH I do understand the weight of a currently existing corpus in production, evidence is the ton of COBOL code still running. But still, your reply kind of sucks.

> Nobody's talking about porting billions of lines of code, for all we know it's just for personal projects, or a learning experience.

am i the only person in the room that can read tone? please tell me what is the force of this statement in what i've responded:

> If you want static compilation, use a language and libraries built with that assumption as a ground rule.

is this an imperative only for hobbyists? not sure.

> This kind of replies is like killing an idea before it's even started, smells like the sunk cost fallacy.

there is no idea - that's exactly my whole point. tear it down and build it again is not an idea.

I finished the rewrite in just a few months and have been happily maintaining it for two years and extending it with powerful features it never had before, partially thanks to optimized native binary speed, partially thanks to the ludicrous compatibility of a musl-libc static binary. The rewrite is the backend foundation of a web product stack that is best-in-category.

I didn't choose Nim because I was an evangelist; I had only toyed with it twice before. After much evaluating (and wrestling an ultra-frustrating previous attempt at a Rust rewrite my predecessor had tried), Nim surfaced as the best tool for the specific job. And it still is. It doesn't have to be our entire product stack; it's doing a small subset of very important jobs, doing them well, and is invoked by other languages that do their jobs better as web servers and such. A modular stack helps each language and tool shine where it works best.

Now there are billions of lines of python in production. But it wasn't so long ago that it seemed like the entire industry was going to completely standardise on C++, and if you wanted a picture of your future, it would be grinding away debugging buffer overflows and thread lockups - forever.

I was looking for someone else that had done this, I had the same exact experience.

That said, anyone looking into a completely static typed language that has nice ergonomics, is easy to pick up but has enough depth to keep you busy for weeks on end, and is versatile enough to be used for anything, do yourself a favor and give Nim a try.

https://nim-lang.org/

Nim honestly seems like such a perfect general purpose language, I wish it was invented 10 years ago or that the adoption drastically increases though because the lack of ecosystem makes it tough rn

I would say that Rpython is successful. However, it's primary goal is to be whatever the PyPy developers need to write PyPy.

I totally agree about subsets though, I would much rather have a superset like Cython, but that has it's own challenges for compatibility if you wanted to have a "pure python" version of the same library. Which is why I really like that Cython is using typehints now.

Yeah Rpython escapes this as most people don't write it directly.

Micropython is another example. No-one expects micropython to support numpy. They're just happy to get away from C. But where you can use the full python, you wouldn't use micropython.

Reply on news.ycombinator.com