Red Programming Language: Static linking support | Hacker News Reader

Red Programming Language: Static linking support

red-lang.org

78 pointsem-bee28 comments

I'm a bit surprised that Red still seems stuck at 32 bits, like it was last time I saw something about it years ago. What is the roadblock that prevents them from building 64-bit binaries? I guess it must be very hard since otherwise it would have already been fixed.

i think the holdup is the lack of self-hosting. probably the rebol compiler they use can only build 32bit binaries so self-hosting and 64bit support go together.

it's on the milestone for 1.0. and according to the roadmap 1.0 is a solid year of work. it's likely not the only thing they want to do for 1.0, but i guess self-hosting is a big chunk

No, even Rebol2 can build 64 bit but the whole compiler needs to be rewritten to produce 64 bit binaries. It’s not limitation but deliberate choice made when 32 bit was still major target. It may seem silly in the hindsight. But it made sense then.

I always love to see Rebol and Red mentioned anywhere, as I spent more than half of my life with these languages, working with them and working on them.

Recently I started working on [Recoil](https://recoil.rblk.eu/) which is inspired by Rebol/Red but targets static memory-safe compiled niche to complement dynamic interpreted nature of Rebol. It’s still in very early phase but it already has some very nice features (at least I consider them very nice) like fully compiler owned `parse`, finite state machine and (lazy) finite state transducer datatypes.

Oh, and built-in Rebol-like scripting language that can run on (not only) ESP32 called F00.

It’s still work in progress but I like it. Maybe someone else can find it interesting too.

> We must free ourselves of the hope that the sea will ever rest. We must learn to sail in high winds. -Aristotle Onassis

Love that quote. More poetic than my quip “Hope is not a strategy”.

https://re-bol.com/short_rebol_examples.r for some examples of the language (from https://redprogramming.com/Home.html)

i'd be curious if red can be used as a commandline shell. the syntax should work well with it. it may just need some wrapper functions to be able to call external programs, pass arguments to them and capture the output.

Not sure why it wouldn't be usable for that:

    -▶ ./red-cli-066
    --== Red 0.6.6 ==--
    Type HELP for starting information.

    >> call/console "echo 123"
    123
    == 0
    >> call/console "pwd"
    /home/cji/portless
    == 0

`call` has a bunch of refinements (toggles or switches appended to the function name with a slash; I'm using /console to redirect output to the parent's stdout), but it's a pretty low-level interface. You definitely could define a few simple helpers and get to a usable Unix-like shell pretty quickly. You'd get native AOT compilation for all your shell scripts for free.

The problem is that you could write those helpers in just about any language, and while Red has an edge over many due to the regular and simple syntax, it's by no means unique in that regard (TCL is an obvious alternative, Lisp-likes are also strong, and even Smalltalk could join the chat if you don't care too much about startup time). And 32-bit-only thing doesn't look good, even if it's not an actual problem in most cases.

In short: it can, but why would you? (Don't get me wrong: I like Red! But with so many other interesting languages (if you're willing to look past TIOBE Top 20), it's hard to justify investing more time into learning Red in particular.)

I think they mean red as a shell scripting/command line language.

All that shell scripting needs is the ability to spawn processes and connect their stdin/out/err together. Or at least, that's what distinguishes "scripting" from "shell scripting". Obviously, you can write a library (I like Python's Plumbum) in almost any language you like that provides this functionality conveniently. So, again: yes, Red can be used for that just as well as any other language (and, arguably, it may be better for this use than many others).

For an interactive shell, you also need a REPL, which Red provides. So if you write that library for Red, you get the interactive shell for free.

Yes, Red has many advantages: it can AOT compile to native, it's homoiconic, it has a built-in Parse dialect (so the library can be really ergonomic), the Red executable is tiny and starts up fast, it has native GUI capabilities (if you're in a Red-based shell and want to view an image, it's trivial to create a GUI window and display it there). I'm not saying Red would be a bad choice. I'm just not sure it would be my choice, given the existence of, e.g., Chicken Scheme or Smalltalk/X.

that was kind of a rhetoric question. i was already pretty sure it would work. so the real question is what is actually needed.

i found the call documentation. that's a good start, but that is pretty much what every other language also provides. what is still needed is a wrapper that allows me to write

    somestr: some command --with arguments

and have that string be populated with the "some" command output. but then we also need to capture stderr and the exit value, so we actually need to capture three return values. next, support for pipes in some form would also be needed.

i have worked with lisp and smalltalk and also TCL so i am aware of course.

with so many other interesting languages, it's hard to justify investing more time into learning Red in particular

i actually find red in particular one of the more interesting languages. alongside lisp and smalltalk. haskell, erlang and ocaml are also interesting, but for shell use i am specifically looking at languages that use whitespace as an argument and value separator and which allow complex expressions to be written on one line, since that happens often in shell commands. that reduces the list of interesting languages quite a lot. if you know any others that you think are worth learning, please share.

btw, your website seems to be down.

Ah, yeah, that makes sense, then.

I tried writing a shell in GNU Smalltalk, but had to mess with the parser - otherwise, the need to parenthesize subexpressions makes it really tedious. I implemented |> as a pseudo-operator that implicitly parenthesized everything before it, which was then removed from the expression. It worked for many simple cases, but wasn't pretty and quickly broke down for more complex ones.

I think the only other language worth investigating in this context is Raku. I avoided it for a long time due to its PERL ancestry, but it's actually a pretty well-designed language, and supports multimethods and overloading of pre-/post-/in- and circumfix operators. The operators support precedence and associativity. If you want an embedded DSL for shell one-liners, I think Raku could deliver. It unfortunately uses the comma for argument separation; you could maybe work around that with quotations and macros, but last I checked, these were experimental and not documented. There was a major effort to move to a better-designed parser that would support AST-based macros, but I'm not sure how far along that is today.

Io would work, though you'd also need to mess with a parser a bit (it's runtime-extensible, though, and also allows circumfix operators and passing unevaluated code as Message objects, which is very helpful in this case). Arguments are normally in parens and comma-separated, but you can simply ignore that. This: `some command --with arguments` is actually a valid Io expression (if `--` is a prefix operator), and since you're not evaluating the code, it doesn't matter if it doesn't make sense semantically. But Io is not very actively maintained (a shame!) It's still a very interesting design. It's described in "Seven Languages in Seven Weeks", if you want a quick intro.

Prolog would probably work, but you'd need to write a metainterpreter - otherwise, threading state in/out of predicates through logic variables would be a nightmare in the CLI. There's also the issue of commas as separators.

You know of TCL, so no need to mention it, though it is a pretty nice fit. A more exotic direction could be concatenative languages: maybe Factor, or Joy, or Kitten. All either experimental or unmaintained, currently. They tend to use whitespace for a separator, and I think they all allow for passing around unevaluated code one way or another.

Looking at the list - Red does look like one of the better candidates, but in normal Red code, all functions know their arity. This means you don't need separators for expressions. It doesn't work that way for shell commands, so you'd probably need to write a Parse grammar, include a separator to allow for variadic commands, and then, after splitting into subcommands, interpret the block command by command. I'm sorry, I'm not too knowledgeable about Red, so I can't give you more details, but that should be the general shape: `shell [ var: cmd1 arg --switch arg2 <some_separator> cmd2 <some_symbol>var ]` where `shell` passes the block to parse, gets a list of commands (including ones that bind results, which would appear as `var_name,command` pair; also, I think path expressions can be used in assignment, so `var/stderr: some command` could be included and handled easily), then walks through the list while setting vars and interpolating them in the following commands. Handling piping wouldn't be much harder than extending the Parse grammar and handling pipelines in the interpreting phase. Finally, you could add refinements on shell for simpler cases (just return exit code, capture all output, etc.) Subshells could be just nested blocks. Thinking about it, it could be a pretty fun way to learn more about Red. Ping me if you end up creating something like that, I'd be very interested to read :)

> btw, your website seems to be down.

I know - I broke my homegrown SSG some time ago and never had the time to fix it :( EDIT: restored some version of the site, 90% of the content still missing.

I had a bit of back-and-forth with ChatGPT about the possible implementations[1] - it shows how little I know about Red, unfortunately, but we came to this representation:

    shell [
        var: [cmd1 arg --switch arg2]
        [cmd2 :var]
    ]

as a general shape of the DSL. The need for a separator goes away (and simplifies the grammar a lot) if you require commands to be nested blocks. :var works for interpolation, because it's loaded as get-word! - trying to cram $var there wouldn't work (that's a syntax for `money!` - only digits are accepted after `$`), but if you want A shell, not BASH specifically, I think you can get away with some syntactic changes. I checked some of the things the model proposed in the REPL, and they mostly seem to work. Subshells and piping would probably require more work in this setup, but it should be doable.

Again: really interesting language and seemingly a good fit for an internal shell DSL. It's also trivial to make files written in this DSL execute as scripts. I might have jumped the gun a little with my "why would you" question at the start :) Personally, I would still seriously consider Io, but that's just a familiarity bias: I spent a year hacking Io internals, while I know little about Red.

[1] https://chatgpt.com/share/6a469cbb-1754-83eb-94b8-14bef67735...

wow, i guess you got hooked :-)

i looked through that chat log. unfortunately over my slow connection it is painful to read. i had to manually copy it one section at a time because it would not load the whole thing at once and kept discarding and reloading sections that are out of view (that's why the slow connection made it painful, anyways, end rant)

i haven't read all of it in detail yet, but i get the gist, and in particular the conclusion. thank you.

i would go for something a little different, possibly a bit more complex DSL:

basically one key syntax feature i would like to have is the ability to run shell commands or red expressions and use their string output as an argument to another shell command:

like in bash:

    cmd1 arg1 $(cmd2 someargs)  arg3 $(cmd3) ...

in red DSL

    shell [cmd1 arg1 [cmd2 someargs] arg3 [cmd3] ...]

the string output that cmd2 someargs and cmd3 return are passed as arguments for cmd1

in addition to that every field in the DSL should also be replaceable with a red expression:

    mycmd: "command"
    shell [:mycmd arg1 :[1 + 3]]

i put the colon there to mark red expressions, but ideally the DSL should figure out on its own whether something is a shell command or a red expression. the two should not be distinct. something along the lines of: let's try if it is red code first, and if not, then it must be shell (since lookups for shell commands are more expensive)

that means we need a different separator for a sequence of shell commands. either ; or a line break. unless the command is nested in a red expression:

    var: [cmd1 arg --switch arg2] [cmd2 :var]

since we know that var: takes one argument it is clear that only one shell expression can follow. whereas

    ls -l [cmd1] [cmd2]

would pass both cmds as arguments to ls. if i didn't want that maybe i would write:

    [ls -l [cmd1]] [cmd2]

    ls -l [cmd1]
    cmd2

or some other separator.

but if we allow any red expression then the first syntax would automatically be supported. we could not get rid of it even if we wanted to. whereas any other separator would require special consideration.

lastly, in an actual shell/repl shell [ ... ] would be wrapped around every line, so i would not have to write it every time.

these are just some thoughts. handling stderr and return values still needs consideration as well as high level data types. one of the limitations of todays shells is that commands can only pass strings, but i would like for commands to be able to return lists, structs, etc. elvish, nushell and others are exploring that space. one approach is that shell commands can return json that can be converted to native objects. some kind of wrapper for that would be interesting.

i am not actively trying to create a new shell, but i am exploring languages for their potential to design a shell language without having to reinvent the whole syntax from scratch.

red's ability to create powerful DSLs without having to parse everything manually is very interesting here. lisp is the only other language with the same ability that i am familiar with. (TCL too, but TCL doesn't have strong datatypes and that disqualifies it for me) . i think even in smalltalk this would be more difficult than red or lisp.

> wow, i guess you got hooked :-)

Maybe a little :D REBOL line of languages has been on my radar for a long time - I've been a PL nerd for 20 years now - but I always found reasons not to engage with them. So it was a bit of a blind spot: I knew some basics, considered it a pretty elegant and powerful design, but the REBOL3 fiasco, the proprietary nature of REBOL2, and the various facts about Red (pivot to it being a "blockchain native" solution was painfull to watch, dependency on REBOL2 for the toolchain/no self-hosting after almost 15 years in development, etc.) made it hard to dive deeper into the language(s). I had some time yesterday, and your comment touched on something I'm already interested in (interactive CLI/universal REPLs), so I took the opportunity to finally learn more about the language.

> i am not actively trying to create a new shell, but i am exploring languages for their potential to design a shell language without having to reinvent the whole syntax from scratch.

As for me, I tried making shells a few times. All those failed, of course, mostly because I wanted to do something a bit different: a text-based computing environment with network transparency, basically a single-player MUD/MOO that would treat the filesystem as a world. I got quite far with Smalltalk - but GNU Smalltalk that I based it on collapsed under me :( I tried Elixir, Io, and Python, and all would require much more work than I was willing to put in. A pity, because I think that would be the best possible environment for LLMs/agents... I'm still thinking about this idea often, and I started working on a new implementation, using Pharo Smalltalk/GToolkit. Parsing is the easier part, actually: it's the liveness of the whole thing (hot reload, adding functionality from within the environment, mapping the environment back to source code, etc.) is the part I had the most problems with, and Smalltalk seems like the most promising platform for implementing it.

The only fiasco with Rebol3, as I see it, is that everybody was asking for the language to be open-sourced, and when it finally happened, almost nobody was willing to take on the responsibility of maintaining it once Carl left, and almost the entire community disappeared - many to the Red camp.

Btw, Rebol is still being developed, without the masses noticing it.

https://rebol.tech/ https://github.com/Oldes/Rebol3/releases

i suspect the community disappeared and moved to red already before rebol3 opened its license.

but if rebol is still being developed then someone (evidently that's you) eventually did take on the responsibility of maintaining it.

i have no clue here. i even only know about red because i met nenad in china. otherwise this whole ecosystem would not be on my radar)

so tell me, what motivated you to pick up rebol3, how does it differ from red and from rebol2 (the repo mentions making rebol3 as usable as rebol2 which implies that as released it was less usable. why did that happen? they could not open everything?)

one difference i see right away is that red is written in red/rebol whereas the original rebol is written in C. being selfhosting is a major feature for a language in my opinion.

As I understand it, there were some investments involved in the development of Rebol2, so it was not possible to release its code. Instead, Carl started Rebol3 as a rewrite. It was first divided into a closed Core library and an open-sourced Host part. Later, both parts were open-sourced, and Carl left.

Rebol3 was always considered an alpha version. In my fork, I've fixed hundreds of bugs and missing features.

And my motivation? I've been using Rebol since its early days. I used it successfully as a scripting language when producing several games. And since I don't make games anymore, I wanted to give something back to this abandoned language, whenever I have the time.

I was also involved in Red (and still follow its development), but for multiple reasons I decided it was better to work alone on the invisible Rebol.

that's awesome. how would you compare rebol3 with red? are there any syntactical differences? or just whats in the libraries? or the architecture, capabilities? as far as i can tell the main difference seems to be that rebol3 is written in C while red is written in rebol2/red itself.

Yes. R3 is written in C and Rebol. Red is so far using Rebol2 to be compiled and Red for mezzanine code.

Main difference is that there is a GUI available in Red. That is not available in R3 yet (at least not in my version). There was a GUI in the Saphirion fork, but it is abandoned for years and I didn't wanted to use it. I would like to provide GUI in the future. Probably as a native extension.

The code is almost same.. although there may be differencies in some function refinements or other details.

Even when Red is compiled to the assembly, it is still slower than interpreted R3 code, because of lack of optimizations. As it depends on its own compiler is also the reason, why there still isn't support fot 64bit arch. To write own compiler is a big task indeed.

And then R3 and Red differs in the number of developers. There may be 3 or 4 people (2 active) working on Red while R3 is worked just by me so far.

your scope of interest seems to have a lot of overlap with mine. though i may be am aiming in a slightly different direction. i am also coming from the MUD world (for my websites i use a platform that has a MUD like architecture underneath :-), and i have a strong interest in a better commandline. i am less focused on exploring languages, that's more of a side quest.

what i really want is a commandline that works more like jupyter notebooks. each command is its own entity, that can be edited and run again. there is an output window/frame separate from the input/command, just like the jupyter. that output can run anything, graphics, an interactive app, like an editor, even a filebrowser. commands can produce structured data that can be interpreted and visualized... (if only i could find funding for this idea :-)

and, yes, i agree, smalltalk has a lot of potential for this. the downside of smalltalk i fear is that it is to bulky for the average user, and this system need to be able to integratable with unix commands written in any language so the interface needs to be agnostic (perhaps json? and another more suitable for binary data, also apps need to be able to run and display output directly to the screen, not depend on pharo to display).

there was a time when i tried using pharo as a desktop. i used a terminal written in smalltalk, and i was looking at finding or building tools a desktop needs (workspace management, a clock, all these small utilities)

if you have something to play around with, i'd be curious to give it a try.

> and have that string be populated with the "some" command output. but then we also need to capture stderr and the exit value, so we actually need to capture three return values.

I haven't used Red, but assuming this part works like Rebol, this works with different refinements. Create empty strings (which are mutable) and pass them in as extra args to populate them:

  stdout: {}
  stderr: {}
  exitcode: call/output/error {some command --with arguments} stdout stderr

That command string is passed to a shell so pipes/etc work within it. Don't remember how it picks the shell so no idea what would happen if Red/Rebol was set as your shell.

Static linking is very essential feature for any such language, I've used it a lot with golang and like how elegantly and easily golang do that. Guess you need to have 64 bit :) of course if you not have any reason to stuck on 32.

i never needed the ability to create a self-contained binary for myself, but when it comes to distributing software to other people it makes a big difference. it was always an issue with languages like python, ruby or pike for example.

so i am very happy to see a high level language get that potential. besides go only common lisp has that ability that i am aware of. others are either to low level (rust/c/c++) or to obscure. not even java does it, although java is common enough that it still keeps distribution easy enough.

to me this means now is a good time to start using red because now i can be sure that i'll be able to build tools that can be shared easily so i won't just be limited to create toys for myself.

64bit is the next big thing on the roadmap. so no need to worry about that. it will happen.

> ability to create a self-contained binary

> so i am very happy to see a high level language get that potential. besides go only common lisp has that ability that i am aware of.

Julia can do it.

Always nice to see Red updates (thanks for sharing)

Being able to bootstrap itself should be the main priority.

And still here.

Reply on news.ycombinator.com