jcparkyn

Agent loops (particularly coding agents) have a huge amount of repetition, because the entire context is included in every model request. So long as it's at the start of the input and doesn't change, it will be able to hit the KV cache (assuming the model provider actually has the prefix in cache).

This only works because prompt caching is done by matching prefixes, not the entire input.

> If standard defined that left subexpression of addition is fully evaluated before the right expression that wouldn't be allowed.

I'm no expert, but surely this would still be allowed so long as the compiler can prove that incrementing a[0] has no effect on the value of a[1]?

Honestly having looked at the memcached clients available for Java recently, I don't think any of the options could be considered feature-complete. None of the main ones support the meta protocol at all, meaning most of the advanced features aren't possible (and these are things that can't be emulated on the client side).

Hell, the main feature I needed (bulk CAS get) didn't even require the meta protocol or recent memcached features - spymemcached just never bothered to implement it. I ended up abandoning the change I was working on, because the upstream never looked at my PR and it wasn't worth forking over (bigco bureaucracy etc).

There are also quite a few legitimate bugs open for years that haven't had so much as a comment from maintainers.

In the programs I write probably about 80-90% of variables are immutable, and I think this probably corresponds to most other code. Except in certain domains and programming styles, not that much stuff tends to need mutability.

This is why the syntax "encourages" immutability by making it the easiest option (similar to e.g. Rust, F#). On the other hand, if it was an extra keyword nobody would use it (e.g. like Java).

This is a little bit tricky because the parser has to distinguish between:

  for x in arr (something ())
           \                 /-- function call

and

  for x in arr (something ())
               \            /-- loop body

This is consequence of combining "blocks" and "precedence" into the same construct ().

A more fitting example would be to support:

  for x in arr do set z += x;
  for x in arr do something x;

IIRC these both currently require an explicit block in my parser.

This is similar but not quite the same as persistent data structures. In particular:

- We can avoid quite a few allocations in loops by mutating lists/dicts in place if we hold an exclusive reference (and after the first mutation, we always will). Updates to persistent data structures are relatively cheap, but they're a lot more expensive than an in-place update.

- Herd has syntax sugar for directly modifying nested values inside lists/dicts. E.g. `set foo.bar.[0].baz = 1;`.

In practice, is this faster than a different implementation of the same semantics using persistent data structures and a tracing GC? That will depend on your program.

> How to pass a mutable reference into a function, so that it can change the underlying value and the caller can observe these changes?

Just modify the value inside the function and return it, then assign back. This is what the |= syntax is designed for. It's a bit more verbose than passing mutable references to functions but it's actually functionally equivalent.

Herd has some optimisations so that in many cases this won't even require any copies.

> What about concurrent mutable containers?

I've considered adding these, but right now they don't exist in Herd.

This applies everywhere, and it fundamentally wouldn't be possible for just function calls.

> needless cost

Are you comparing to a language with mutable references or a our functional language? A language with mutable references will of course be faster, but this is more intended as an alternative to pure functional languages (since functions are referentially transparent).

In this case, the cost of the indirection is approximately zero (relative to the cost of just doing reference counting), since passing a reference just requires a bump to the refcount. And most of the time the refcount increments are skipped by "moving" instead of copying the reference.

Personally I think local mutability is quite a useful property, which was part of the inspiration for making this instead of just another pure functional language:

- All functions are still referentially transparent, which means we get all the local reasoning benefits of pure functions. - We can mutate local variables inside loops (instead of just shadowing bindings), which makes certain things a lot easier to write (especially for beginners). - Mutating nested fields is super easy: `set foo.bar[0].baz = 1;` (compare this to the equivalent Haskell).

> can you take the address of a variable in some way?

I intentionally didn't add this, mostly because I wanted to explore how far you can get without it (and keep the language simple). Having a "real" pointer as a first class type wouldn't work though, since it would break a lot of the assumptions I use for optimizations.

I did think about two different versions of this but didn't end up adding either:

- Something like `inout` parameters in Swift, which aren't first class pointers. This is really just an alternate syntax for returning multiple values. - A "ref" type, which is essentially a mutable container for an arbitrary value. Passing the container around would share a reference to the same mutable value. This still wouldn't allow modifying values "outside" of the container though.

Nope, it's value semantics (unlike Python), the references are just an internal optimization. Implicit copies happen when a list/dict with more than one reference is mutated.

It's only semantically a pass-by-value, in reality a reference is passed and the data is only copied if needed (i.e. value is mutated while shared).

> Have you benchmarked any real workloads?

Nothing "real", just the synthetic benchmarks in the ./benchmarks dir.

Unnecessary copies are definitely a risk, and there's certain code patterns that are much harder for my interpreter to detect and remove. E.g. the nbodies has lots of modifications to dicts stored in arrays, and is also the only benchmark that loses to Python.

The other big performance limitation with my implementation is just the cost of atomic reference counting, and that's the main tradeoff versus deep cloning to pass between threads. There would definitely be room to improve this with better reference counting optimizations though.

> Use immutable pass by reference. Make a copy only if mutability is requested in the thread.

This is essentially what Herd does. It's only semantically a pass by value, but the same reference counting optimizations still apply.

In fact, Herd's approach is a bit more powerful than this because (in theory) it can remove the copy entirely if the caller doesn't use the old value any more after creating the thread. In practice, my optimizations aren't perfect and the language won't always detect this.

The big downside is that we have to use atomic reference counts for _everything_. From memory this was about a 5-15% performance hit versus non-atomic counters, though the number might be higher if other bottlenecks were removed.

Yes, if you modify a nested dict/list entry, all nodes above it will be cloned. Here's an example:

  x = [1, [2]];
  var y = x;
  set y.[0] = 3; // clones the outer array, keeps a reference to inner array
  set y.[1].[0] = 4; // clones the inner array here. Outer array is now exclusive so it doesn't need another clone.

  var z = x;
  set z.[1].[0] = 4; // clones both arrays at once

Thanks, I updated the post title based on this and another comment. Thanks for the pointer to Euphoria too, looks like an interesting language with a lot of similar ideas.

Not everything in C is pass-by-value. Sure, you can argue that a pointer itself is passed by value, but the data it points to is definitely not.

I'm not sure if I understand the question?

There are two ways to define a variable binding:

    x = 1; // declares x as immutable
    var y = 2; // declares y as mutable

The "default" behaviour (if no keyword is used) is to define a new immutable variable.

Thanks, some interesting reading there that I will check out (I wasn't aware of PCF). Perhaps I should've used more precise wording: "All types are value types".

> Standard ML [...] It has all you dream of and more

The main thing here that's missing in Standard ML (and most other functional languages) is the "mutable" part of "mutable value semantics" - i.e., the ability to modify variables in-place (even nested parts of complex structures) without affecting copies. This is different from "shadowing" a binding with a different value, since it works in loops etc.

> The whole point of names is that they aren't descriptive

I actually agree with this, but that's exactly the opposite of what TFA is arguing.

One of my previous side projects used this idea in the extreme: It's a two-player online word game (scrabble with some twists) but all the state is stored in the URL so it doesn't need a backend.

https://scrobburl.com/ https://github.com/Jcparkyn/scrobburl

Not to mention tracepoints (logging breakpoints), which are functionally the same as printf but don't require compiling/restarting.

> I think if you [...] and have some luck you will succeed no matter what

Ignoring the politics of it, this wording is pretty funny to me.

To be fair, this is the first thing that comes up when I google search for "DuPont connector" (after some sponsored shopping links).

Genuine question: Is it illegal to advertise for a product that doesn't exist, can't be bought, and you have no intention of taking money for?

> Conditionals are common enough that they can justify the indulgence

I think there's another much more important factor that distinguishes conditionals from most other ternary operations (clamp, mix, FMA, etc): The "conditional evaluation" part, which is what makes it hard to replicate with a regular function call.

Agreed, and then there's the time of check/time of use issue with creating a user. Probably not a vulnerability if userService is designed well, but still a bit dubious.

> squash an output into a range

This isn't the primary purpose of the activation function, and in fact it's not even necessary. For example see ReLU (probably the most common activation function), leaky ReLU, or for a sillier example: https://youtu.be/Ae9EKCyI1xU?si=KgjhMrOsFEVo2yCe

> sorts and prints the latest results at a regular interval

Slightly more complicated than that, because you can only sort and print elements once you have _all_ the elements that came before them. Once you add that layer you've got quite a lot more code (and potential for mistakes) than the promises version.

The real answer IMO is how they (don't) integrate with generics, first-class functions, and the type system in general. If you try using a checked function inside a stream() call you'll know exactly what I mean. Yes, it's technically possible to make a "functional interface" with a fixed number of checked exceptions, but in practice it's a huge PITA and most functions just don't allow checked exceptions at all.

Compare to ATDs, where any generic function taking a T can also be used with Result<T> exactly the same. (Not a perfect comparison, but there are lots of other scenarios where ATDs just compose better)