Tungsten – an expressive, fast programming language | Hacker News Reader

Tungsten – an expressive, fast programming language

tungsten-lang.org

11 pointsjeffpeterson9 comments

So I want to preface my next point by stating actually really like how syntax reads, hitting a sweet spot of leaning just enough into APL's symbol-focused terseness without losing the familiar structure of literally every other popular programming language ever that makes it easy to follow the code. I imagine it could be a great language to pick up for working out an algorithm with pen & paper (the best limus test for how well a language functions as pseudocode imo).

Having said that, I have my doubts about the reduced token claims because of a lack of "begin" and "end" or "{" and "}" delimiters. Surely, if your language is identation based, LLMs will treat significant whitespace as a token too, since they need to keep track of it? Which would result in an extra token per newline compared to using explicit begin/end token?

I suspect that to really optimize for tokens one would have to go the concatenative route, which significant whitespace very much isn't.

Apologies for the slow reply, my lack of karma is throttling how often I can comment.

I appreciate you sharing your thoughts, the whole point of a public preview is to find out how things are received, what’s working, what isn’t.

We more or less agree on the vibe-coded website (especially the copywriting). It’ll improve over time, but this is a side-project, time is limited, and I wanted to release something on 7/4.

Tungsten’s grammar and design are mine.

Tungsten’s implementation, is almost entirely LLM (lowering, escape analysis, SSA), with a good bit of prodding, benchmarking, troubleshooting; in order to have the performance land remotely close to where I knew it could.

While my primary motivation was to build the language that I wish existed, if I can make something that others enjoy or find useful, even better.

To address your other point:

Whitespace is almost certainly getting tokenized whether it’s significant to the language or not. And I would expect most folks are indenting their blocks anyway, even when they are delimited.

I think you’ll find the token reduction comes from more than just dropping delimiters.

I won’t try to litigate the point here, I’ll save that for a longer-form write up that is lacking on the website.

Thanks again for the feedback.

Related (in the repo):

WValue layout https://github.com/tungsten-lang/tungsten/blob/d5f505acca245...

Ruby Lexer (regex-based) https://github.com/tungsten-lang/tungsten/blob/d5f505acca245...

Ruby Lexer (spec) https://github.com/tungsten-lang/tungsten/blob/d5f505acca245...

Thanks for the in-depth reply, appreciate the elaboration on where you did and didn't use LLMs. Looking forward to reading the blog posts and other future updates!

This is fairly impressive, at least at first glance and hits a niche that is interesting. I think a big challenge in finding adoption for a new lang is in the application, or more specifically, when/why/where it's used. This is something I'd love to hear from folks involved in this (if reading) what they think those are and if/how that helped them shape the lang. Good luck!

Thanks for taking a look.

Tungsten is terse and fairly token efficient (20-30% less than Python). I've found it works well for prototyping with LLMs - even though they haven't been trained on it.

I've been using it lately for some scientific exploration, made a flip-graph search for matrix multiplication algorithms (low-rank decompositions of the matmul tensor over GF(2)). It has independently matched the world records for 3x3, 4x4, and 5x5.

Oddly enough, even though it's fresh out the gate, I think it makes a good fit for language implementation (at least as a stage1). I have a few reference lexers, an arena-slabbed AST layout, and a NaN-boxed 64-bit layout in the repo.

The bignum math is nearly as fast as GMP, without GPL license issues. The native regex engine was matching PCRE.

There's a scrubbing feature in the REPL that is pretty fun as well - inspired by a paper and talk from Bret Victor a while back.

Thanks for posting! I'm the creator of Tungsten. Ask me anything.

Can LLMs generate only one style of website? Inter font, dark background, elements that fade in, code in a macOS shell container, neverending lists of features, too many buzzwords.

God they all look the same, and I have permanently associated this style with slop.

That said, a fully vibe coded language created 7 hours ago and published 6. Have you built anything with it in the intervening hour to try it out at least? To make sure it does what it says on the tin. Or you just one shot the thing with Claude and published without review?

I know, right. But I think you might be surprised to find a real language under the AI veneer. It won't be for everyone, but for some folks out there it will map more closely to how they already code in their head.

Either way, it has several features that walk new trails in language design, for better or worse.

The grammar has been marinating for 10+ years, but life kept getting in the way of the implementation.

Hope you'll give it a second look, would be interested to hear your opinion on the language itself.

So just my 2 cents: I also have an immediate "ick" reaction when I see the vibe-coded aesthetic of the website itself, expecting the worst, as well as serious ethical concerns about using LLMs at this point in history. Luckily in this case, it also was (somehow) immediately obvious to me that actual thought was put into the language, so I was willing to take a closer look to see if some genuine effort was put into it.

But be aware that a lot of people won't be willing to do that. Based on my experience in proglang chat servers and fora I'd say that most of the people with the niche interests most relevant to you (deep dives into non-standard ideas for proglang design & implementation) are in that subgroup. Especially the ones with inspiring ideas and knowledge to share.

Beyond expecting honesty about the usage of LLMs I'm not going to say anything about whether you should use them to develop your langoage or not, that's up to you. But I'd strongly advice for avoiding it in the presentation side of the language (website, whatever) at all costs, because it will get in the way of connecting with your potential audience more than it will help.

To give a point of comparison: you are undoubtedly aware of the issue that popular open source projects have with slop PRs? One problem is that it is so much harder to to see where the LLM output ends and where human effort starts. And the last part is where the actual conversation takes place! Having to filter for that is just exhausting and starts to feel disrespectful.

When I'm reading your language I want to read the part of it that is your language, not the LLM generated bits of it. The LLM aestetic and writing style of the website is an extra layer between me and your thoughts, and it's distracting the same way a chain smoker's body odor is distracting to non-smokers despite being oblivious to it themselves. If this is a language you're passionate about, and it looks like it, then I'd say it deserves better than that.

Reply on news.ycombinator.com