Rye: Homoiconic dynamic programming language with some new ideas | Hacker News Reader

Rye: Homoiconic dynamic programming language with some new ideas

github.com

201 pointsnnx81 comments

The email type seems just a smidge too specialized for me. It could more broadly be an "authority" type (borrowing URI/URN terminology) a la:

authority = [userinfo "@"] host [":" port]

The simple extension of allowing a port makes it a more broadly useful type, since authorities like above appear all over the place. Email is just one protocol after all.

There is also a url type, but it is unclear how general it is (just http/s?). There is a cpath type which corresponds to the path in a URI as well.

URI = scheme ":" ["//" authority] path ["?" query] ["#" fragment]

In Rye typing, authority _could_ be the authority described above, and cpath already covers path. The rest are strings as they don't have generically defined syntax.

Perhaps this is a bit of a quibble but it seems to me like if URIs and their structured components (cpath is already there, and email is so close!) were core types rather than just email and url, it opens up a lot of use cases.

Hm. Interesting proposition. Rye inherited email and URL data types from Rebol. I haven't done that much on them specifically, because they aren't that crucial to the design of the language dynamics, so their implementation certainly should be improved (ant thought about).

The idea of making them more general certainly makes sense. Rye is "end-user" focused so it would maybe still be called email. URI-s right now can have different schemes. Scheme defines the kind of url and Rye has generic functions that dispatch on the kind of first argument. So there is:

open file://readme.txt and open sqlite://db.s3db and open postgres://user@localhost/database "pwd" ...

%path/notes.txt is also Rebolism and it's a shorthand for file://path/notes.txt

Thank you. I will think about this

cpath is currently a little specific, at least in behaviour as it's a context path to a word.

Thank you for making this comment. I will think more about it and certanly reread it when I work on there datatypes next time.

The blog entry titled "Less variables, more flows example vs Python" is strange. ( https://ryelang.blogspot.com/2021/11/less-variables-more-flo... )

The Python version uses intermediate variables so the author of the code is to blame for verbosity, not the language.

This was written in 2021, maybe it's really not that great of a blogpost. I wrote a lot of blogposts from many angles back then, mostly to myself as there weren't many readers, trying to see the language from the outside, to self-motivate, etc ...

I think I wrote a script I needed in Rye and I found it interesting, then I went to Fiver and paid someone to write a script in Python that does the same.

2021 were different times, and I hope I've grown a little too... and now I would ask ChatGPT or Gemini :)

It is a legitimate complaint of the language as there are language-related reasons that motivates it --the conciseness of the language has an impact on readability, but also how convenient it is to debug on separate lines.

The code examples are a tad too verbose but actually better quality than most real-world Python codebases, I would say.

Python is not very verbose but it's not very concise either (especially compared to Lisp families)

ISTM the big difference is that the Python code is doing CGI essentially from scratch, but the rye code appears to be using a CGI library.

The Pyhton side does have import cgi. It also uses req library for request parsing and making a post request. Only for cookies it uses os.environ which is maybe a little unusual, but I didn't specify in fiverr request what libraries to use or not use.

I don't see fewer variables as something good. On the contrary, I find a long "flow" or chain of function calls harder to read or grok.

Variables also help readability, because the name can help you discern what those functions return.

This is a valid argument. There are already a plenty of programming languages where you can do that. You can make temporary variables in Rye too, but it also tries to work well for these chains of expressions or function calls, and I personally prefer this style many times. There is no wrong way, IMO.

If you're the only one reading the code, then sure write it however you like it.

But using long chains of expressions is the same as one-liners or point-free style in Haskell. It saves some typing and also you can skim the code more easily, but only if you're extremely familiar with what's going on there.

I wonder how much you can benefit from this, if you return to the codebase after a 6-month break though. Maybe some people do manage to really memorise these details, but for some of us the effect is more like "wtf is this code doing?"

FWIW Haskell is the only language in which I've been able to return to code that I wrote six months later and still understand it.

It depends on how you wrote it. In general, I found it hard to understand point-free expressions (in the code of others, because I never used them).

IME, point free expressions are exceptionally clear when sufficiently simple, but get muddy fast. The inflection point is probably not too far from

    let rms = sqrt . sum . map (^^ 2)

I'm not trying to change your opinion, I just want to follow my opinion when coding. :)

I prefer more verbs and not too many nouns, and I think our brain is used to reading "sentences" like that.

    get http://example.com |parse-html |find-links |for-each { .if-external { .print-it } }

Is not that dissimilar from instruction we can understand quite well:

    Take a basket, look into it, pick out apples, if the apple is red take it.

If there are complex procedures, I also want to name a temporary result. It's like a divide and conquer strategy. You divide complex expression or instructions to separate definitions and combine those. But I also don't want to divide too much. This on the other hand again obscures IMO.

If I name every internal state, we get this.

    hmtlStr = get("http://example.com)
    html = parse(htmlStr)
    links = findLinks(html)
    for l in links {
      take = isExternal(l)
      if take {
         print(l)
      }
    }

Which I can parse, but it takes more energy to validate that I used the right variable at the right place, not some variable from higher up in the code for example. You probably also don't code like this, but there is some middle ground ...

Just as a thought experiment, don't take me too seriously I will try to "humanize" those instructions:

    Take a basket in your hands.
    Look into the basket in your hands.
    Find apples in looked basket in your hands.
    For every apple.
    Check if the apple is red.
    If it's red, take the apple.

Maybe you can make more balanced version which will be the best of both worlds. Thanks for the discussion :).

I find your first example easy to understand.

It's funny, because I'm watching my experience as I'm reading the code and I feel like variables are like a little break while doing work. When you don't use them, it feels like a constant run.

Also, they give some extra structure, because they name the output of the function. It's true that you have to make extra effort to think of the name, though.

I'll be honest, your example with naming every internal state feels comfortable. Sometimes I do chain a few expressions together, but I avoid chains that are too long. I want to say that more than 3 is too much, but maybe it depends on the specific code.

I don't have examples of my own code handy, so I can see how I did it in the past. Now that I think about it a bit more, I think I'm okay with a longer chain, if it involves transformations of the same structure, but not if it involves destructuring. I perceive the `for` loop as a destructuring.

> Rye is homoiconic, it has no keywords or special forms (everything is a function call, everything is a value)

How does it implement and, or, or if?

Presumably as functions, the same way as excel?

and(x, y) -> bool

or(x, y) -> bool

if(cond, funcTrue, funcFalse) -> void

Yeah I did notice in the examples

    fac: fn { x } { either x = 1 { 1 } { x * fac x - 1 } }
    ; function that calculates factorial

So presumably "either" is the "if" expression.

"if" is the "if" expression - https://ryelang.org/meet_rye/basics/if_either/

Yeah, but "either" is the if-else construct, so the person you replied to isn't really wrong.

    if(and(or(cond1, cond2), cond3), effect1(), effect2())

In most languages, if `cond1` evaluates to true, you would not evaluate `cond2`. If `cond1` and `cond2` evaluate to false, you would not evaluate `cond3`. If all conds evaluate to false, you would not evaluate `effect1()`, and if `cond3` and either `cond1` or `cond2` are true, you would not evaluate `effect2()`

They're not functions because they don't evaluate their arguments before evaluating their body. Their operands are passed verbatim and evaluated explicitly by the body on demand.

In Kernel, for example, we can define these as operatives, which don't evaluate their operands. Assuming we have some primitive operatives `$cond`, `$define!` and `$vau` (the constructor of operatives), and an applicative `eval`:

    ($define! and
        ($vau (lhs rhs) env
            ($cond ((eval lhs env) (eval rhs env)
                   (#t #f)))))
                   
    ($define! or
        ($vau (lhs rhs) env
            ($cond ((eval lhs env) #t)
                   (#t (eval rhs env)))))
                   
    ($define! if
        ($vau (condition consequent antecedent) env
            ($cond ((eval condition env) (eval consequent env))
                   (#t (eval antecedent env)))))

These aren't the definitions Kernel uses in its standard environment. It uses recursive definitions of `$and?` and `$or?` which take arbitrary number of operands, and `$cond` is defined in terms of `$if`, which is primitive:

    ($define! $cond
        ($vau clauses env
            ($if (null? clauses) #inert
                 ($let ((((test . body) . rest) clauses))
                    ($if (eval test env)
                         (apply (wrap $sequence) body env)
                         (apply (wrap $cond) rest env))))))

    ($define! $and?
        ($vau x env
            ($cond ((null? x) #t)
                   ((null? (cdr x)) (eval (car x) env))
                   ((eval (car x) env) (apply (wrap $and?) (cdr x) env))
                   (#t #f))))
                   
    ($define! $or?
        ($vau x env
            ($cond ((null? x) #f)
                   ((null? (cdr x)) (eval (car x) env))
                   ((eval (car x) env) #t)
                   (#t (apply (wrap $or?) (cdr x) env)))))

I don't think that's what Rye is doing.

It's doing if(cond, effect1, effect2) where effect1 and effect2 are functions, and only evaluating the matching effect function. But everything is functions.

technically effect1 and effect2 are so called "blocks of code" in rebol/rye/red/...

Everything being a function is trying to say that every "active word" (a word that does something ... print, first, if, fn, context, extends, ...) is just a function.

This is possible with fexprs or equivalent constructs https://en.wikipedia.org/wiki/Fexpr

It looks like that's how Rye does it as well, blocks can be conditionally evaluated: https://ryelang.org/meet_rye/basics/if_either/

"In REBOL, contrary to Lisps, blocks or lists don’t evaluate by default. For better or for worse, this little difference is what makes REBOL - REBOL." https://ryelang.org/meet_rye/basics/doing_blocks/

It's difficult for a language with this semantics to be made efficient, but efficiency isn't everything.

>programming language based on ideas from Rebol, flavored by Factor, Linux shell and Go

That makes it potential interesting to me.

potentially, autocorrect incorrect :)

Why is it difficult?

Since code-is-data, you can put that exact if statement into several functions.

Since code-is-data, you can swap the if and else clauses around. This applies to all the functions where you've inserted the if statement.

This kind of thing makes it difficult to compile a function to a fast set of machine instructions with the same effect. Not impossible, but difficult.

It's difficult to optimize because evaluation of expressions depends on the dynamic environment, which you don't have ahead-of-time.

You can't even assume `+` means "add the arguments", because `+` may have been bound to something completely different prior to evaluating the expression containing `+`.

Late to the party here. What would be your take on dynamic-by-default, but the ability to fix the env a la Dreams (http://elilabs.com/~rj/dreams/dreams- rep.html) or Zig (comptime)? You can obv. do this in Rye via a static context.

I mean, it's possible in lambda calculus. Anything above that is sugar, right?

Yes, but with supposedly no special forms there cannot be a lambda operator.

Turns out the language does have special forms, which is OK; it's just weird to say there aren't (though it's an understandable goal).

When I worked on 3Lisp (so many decades ago) it became clear to me how many special forms there are (a small number, but more than I thought) and, honestly, how few there really are, so the "benefit" of a 3Lisp turns out to be negligible in practice. Oddly enough I didn't really notice that when writing interpreters because I thought of most special forms as simply compiler hacks ("eventually we can get rid of this").

> it became clear to me how many special forms there are … and … how few there really are …

I'm having trouble parsing this. The two parts there seem to be saying opposite things. Was that an accident, or were you saying that from one point of view it seems to be a lot while from another point of view it doesn't, or something else?

The latter. When writing a somewhat standard implementation people expect redundant special forms (like both if and cond) so there are more than you think. OTOH you can implement some in terms of the others so maybe there aren’t as many as one might thing.

Also, of course, in 3Lisp you can run code in your interpreter and so define new control structures and such. Turns out there aren’t many interesting ones and they have mostly already been thought of.

One new control structure that didn’t need to modify its own interpreter was method combinators. Turns out they mainly useful for unpredictable behavior, except in very simple cases like :before and :after.

I would say Rye does not have special forms; in LISP aiui, lists are evaluated by looking up the definition of first symbol, then evaluating the rest of the list, then apply the rest of the list to the definition. Except when the first symbol is one of a small number; then the list is evaluated fifferently.

Rye's evaluator seems more complicated, but the forms are regular. A block is always evaluated the same way doesn't change how it's evaluated based on what the first element is.

You can have lambdas without special forms in Rye because blocks aren't evaluated eagerly.

Of course I could be way off; I've been having fun this morning poking at Rye for the first time and my Lisp / Scheme exposure is limited to Uni classes eons ago and a resulting allergy to parentheses.

Seeing the meta-circular Rye would tell us for sure :)

> then evaluating the rest of the list, then apply the rest of the list to the definition

True for function calls. But not for the zillions of macros. The "small number", you mention, are the small number of special operators. But there are many more macros. Those get the arg source unevaluated and return a Lisp form, which then is checked again (either by the compiler or at runtime by a source interpreter).

I know well what a FEXPR is (I was a Maclisp maintainer) but they are special forms.

It is interesting to imagine inverting the sense of execution, but as you say it's hard to do much optimization.

They're special forms in a language where functions are something which eagerly evaluates its arguments.

In a language where functions don't do that, they're just functions. Rye appears to be one of those languages. One could quibble about whether that's the right thing to call them, but if we say they're vau or whatever, "everything in Rye is a vau" is still true. I think calling them functions is reasonable though.

In Kernel, we could argue that operatives are the more fundamental type of combiner, and applicatives (aka functions) are the sole special-form, constructed by calling `wrap` on another combiner.

This is how Joy and Factor work as well, so I'd say this is standard fare for concatenative languages.

This took me a minute to get, even after looking at the page about if:

https://ryelang.org/meet_rye/basics/if_either/

The tricky thing about if, and, and or --- the reason you can't implement them as functions in most languages --- is that they need to not evaluate all their arguments immediately. Otherwise:

    // Would print!
    if(false, print("oops!"))

    // Would throw  an error if the key is not present
    and(my_hashmap.has_key("key"), my_hashmap["key"])

The way that ryelang gets around this is that you pass the arguments in a "code block" surrounded by "{}", which delays its evaluation. So you write:

    // Does not print, because *if* never runs its code block arg
    if 0 { print("oops!") }

    // There's no example of *and* anywhere but my guess is you'd write this:
    and { my_hashmap.has_key("key") } { my_hashmap["key"] }

The R language has this feature as well. It's a whole lot of fun to work with.

Likewise in Tcl where blocks can either be sent quoted (using {}) or unquoted (using “”). In the latter, variables and procs will have a round of substitution performed before being passed to the proc.

Rebol (where Rye took this from) is many times associated with Tcl. I've heard good things about it, but haven't really tried it yet.

TCL is kinda similar to Rebol in some ways but in other ways it's the opposite of Rebol, because in TCL everything is a string (although it can ALSO have another type, thanks to clever shenanigans). (You probably knew this!)

I heard this "everything is a string" line many times abot Tcl and it sounded a little unusual, but I havent delved deep enogh in tcl to see what it really meant and brought. I will.

everything has a string rep available. It used to be that every thing was also represented literally by a string. So, for pedagogical purposes, a value 1 would be "1", and to do math, Tcl would do a strtol(val_storage), with the obvious performance implications.

The way things are done now (and have been for a long time), is that values are stored in a

    struct Tcl_Obj{
      int refCount; // objs can be shared
      int myType; // indicates whether currently a long, double, etc
      long longVal;
      double dblVal;
      [...]
      char *stringRep;
      int len;
    }

...in fact, the Tcl_Obj is more sophisticated than this, but for demonstration purposes this is fine.

So "native" (eg: longVal) values are used when appropriate, no marshalling back/forth between strings, but the string rep is always available (can be generated), because that's what Tcl promises: everything is representable as a string. This is what brings the homoiconicity to Tcl - logically it's just passing text tokens around, and emitting text tokens. Internally, again, more sophisticated, but you get the point.

This is a very natural technique in concatenative languages.

As done in Smalltalk.

Smalltalk may be influential, but is now rarely used.

The code block approach is widely applied in two massively used industrial languages though: Ruby and Kotlin. In Kotlin specifically it's one of the very central features.

And Ruby has been strongly influenced by Smalltalk.

Anyway, if you want something perhaps a bit more used, you can check out Pharo. I think that is the most used Smalltalk-like language these days.

> The way that ryelang gets around this is that you pass the arguments in a "code block" surrounded by "{}"

Like Lisp’s QUOTE which has to be a special form.

In the maxima computer algebra system[1] which was ancestrally based on lisp it has a single quote operator[2] which delays evaluation of something and a "double quote" (which acually two single quotes rather than an actual double quote) operator[3] which asks maxima to evaluate some expression immediately rather than leaving it in symbolic form.[4]

[1] https://maxima.sourceforge.io/

[2] https://feb.kuleuven.be/public/u0003131/WBT23/wxMaxima/wxM_i...

[3] https://feb.kuleuven.be/public/u0003131/WBT23/wxMaxima/wxM_i...

[4] although (in my limited experience) maxima sometimes refuses to evaluate anyway even if you do the double quote thing

Is this correct though? Lisp's quote would need some eval or something to evaluate later afaik. More fitting might be a (lambda () ...), a.k.a. lazy evaluation.

The implementation of the if() function would be the one that calls eval() on the true or false code block.

Lambda is a special form

I would imagine it is closer to lambda than quote (though also a special form), since the implementation of if would require that the bindings in the arguments evaluate to their values in the callers environment.

Yes, as samatman said, main reason is that blocks of (code or data there is no difference) don't evaluate so they are passed as function arguments and the function can potentially evaluate them. So if is a function that accepts two arguments, a boolean and a block of code.

loop is a function that also accepts two, integer for number of loops and again a block of code.

Even fn that creates functions is a function that accepts two blocks, first is a list of arguments and second is a block of code. There is no big difference between function fn and function print, they are both builtin functions defined in same manner and there are multiple fns for special cases and you can create your own on "library" level.

In a eagerly evaluating functional language you could always wrap the arguments in a thunk:

So if your built-in functions were implemented in Python, you'd use:

  def if_(cond, t, e):
    if cond:
      return t()
    else:
      return e()

Seems like a good Swiss Army Knife-like addition to the shell script (reminds me of awk as well). It would be interesting to keep it that simple (not another Perl)

Awk was one of early use-cases I wanted to make it useful for. It still has "Ryk" mode, but I haven't tested it in years so I'm not sure if it works right now.

Rebol was known for its small set of moving parts and behaviors but with a lot of depth and flexibility with them. I've added some moving parts with left to right code flow, but I still hope it's a limited set that fits well together. I am adding new behaviors very conservatively now, and I will remove some. Thanks for heads up about Perl! ;)

Congrats, this language looks pretty cool! What advantages does it have over, say, something like Elixir?

It's difficult to compare it directly to Elixir since Elixir is a production ready language built on top or Erlang VM. Elixir certainly has more solid runtime, as Rye is interpreted. But Rye probably has much more malleable runtime that you could maybe better structure and specialize around your problem.

I played around with Rebol many years ago and enjoyed it. This too looks like fun.

The detail about input validation is a really nice one that hopefully the next generation of programming languages all do standard.

It sounds intriguing, but it doesn't look like there are any examples in the readme, and the documentation for the Validation dialect on ryelang.org is completely blank. How does this feature work?

Hi, you can see a smaller example on ryelang.org front page if you ctrl+f "validation".

Here are unit tests / reference for the validation dialect, but they aren't the best source of information: https://ryelang.org/validation.html

Here is an old blogpost with another example: https://ryelang.blogspot.com/2021/01/added-intro-pages-for-h...

It's a dialect with static/builtin rules that can be combined and two rules that accpet blocks of Rye code so they can do anything (check and calc, first checks value for certain condition, second changes / calculates new value from the old)

I've written about validation dialect in previous attempts, but current one "Meet Rye" is still in the process of being written and I will get to this.

It’s definitely an idea which I’ve seen people wish Haskell had. But requires some machinery that’s not quite in that language.

I guess it’s easy to express in a dependently typed language to an extent.

This validation dialect (and SQL dialect are main things) I used in all my web development for years using Rebol. So in Rye I made it part of the base builtins. It's nothing particularly special or complicated and could be improved to cover more cases.

I then had custom functions in Rebol that instead of argument list accepted this validation block and what it returned gor serialized into JSON and returned from the webserver. In case of valiadation error it also got automatically returned and sent back. So you defined such fucntions in a context and got an web accesible API and also you could call them on serverside to render something there (or fill in JSON directly into page that is served).

Function in Rebol (and Rye) also have a docstring and it's accesible within the language so these functions could also self-document. Show their description and validation rules if you called them with &_x=1 (as in explore mode).

Of course I tried to redo all this in Rye and make it even better for this.

Ok ... I got a little carried away ... :)

familiar with rebol, but evaluation rules with op and pipe words gave me headache. would like to know more about context oriented programming, tutorial had nothing in the section, unfortunately

Yes, op and pipe words are the biggest or the most visual addition to Rebol's base idea. Without them if you replace { } with [ ] it's basically just Rebol ... well, with some different details around contexts (rebol's bindology), no refinements, mandatory spacing around tokens, different error handling logic and some other details.

I am still learning to explain or even name things, but various examples of using contexts in different ways are currently what excites me the most. I will write "Meet Rye" further and contexts are one of next subjects to be written about.

> with some different details around contexts (rebol's bindology)

Could you perhaps summarise how contexts work in Rye? On the Ren-C forum we’ve recently been discussing binding a lot, and I’d be interested to learn how Rye does it.

The context in Rye nothing special, but I'm excited about how all it can be used ( and the context details could change a little too in future).

Context is just a namespace of "word: value"-s and an optional link to a parent context. When a word is being searched it looks in the current context and if it's not found it goes up to parent, etc.

One interesting things about contexts are what I called "isolates", a context that is isolated from anything else, but can hold a set of functions specialized for a certain task. Or you can construct and string together custom contexts to create a desired environment for certain code.

When you have a context, you can evaluate code in context, you can call functions in context from outside, you can create functions that are evaluated is a specified context or with a specified parent context, etc ...

You can also move through contexts and list them in Rye console like you would through directories.

I don't know if I answered your question well ... I'm maybe better at showing than explaining. I'm preparing an asciinema demo called "contextplay".

This sounds almost exactly like how environments work in R, then… is that correct?

I've used R for some data visualization in the past, but I'm not sure I was fully aware how R scopes work. The spreadsheet datatype is inspired by dataframes that I first saw in R though.

One difference with Rebol around scopes is that setwords only set values in current context. A function (or any other context) can't change anything outside directly, and also not in subcontext, so there is not context/word: "value".

You can only call functions in sub or parent contexts.

There are few functions that change words in-place and all end with !, they accept lit-word (in Rebol terms) that they should change in place. So this word is seeked in same manner as in general words are, also through parents, but it's very visible and explicit.

> I've used R for some data visualization in the past, but I'm not sure I was fully aware how R scopes work.

You might be interested in this description, then: https://adv-r.hadley.nz/environments.html

I've just skimmed it now but certainly interesting and yes, that namespace and the parent seem just the same. I'm interested in seeing how all they use this. Thank you!

You’re welcome!

About getters.. are they foo? or ?foo ? The examples have it mixed, "Meet Rye" doc has it as ?foo

If you don't come from Rebol it would probably be weird to you that there are many specific word-types in Rye.

    name: Janko ; name: is a set-word - it binds value to a word
    ?print      ; ?print is a get-word - it get's value word is bound to in this case a print builtin function
    :age        ; left leaning set-word (this is get-word in Rebol)
    what?       ; just a regular word
    ...

name get's the value anyway, so we don't need to use ?name but if word is bound to a function just invoking a word will evaluate a function and if we want to return a funtion we use get-word. which has ? in front.

? at the end is just a regular word and a (currently accepted) naming convention where noun? means get-noun. so length? in instead of get-length etc.

Rebol used ? at the end convention for more things, a lot for boolean results, testing of types, like string? and positive? but also for lenght?

For booleans current Rye's naming convention is that we use is-adjective. Rebol used positive? to test if value is positive. We would in this way use is-positive.

The conventions might change if we see that there are ways that make more sense and are also consistent.

Thanks for the explanations. Indeed I don't come from Rebol and the conventions seem like a terra incognita, although I can sense a significant value in them.

From a past romance with Forth I see in general how concatenative languages seem to be underappreciated a bit

Reply on news.ycombinator.com