michalsustr

In multi-player you don't have guarantees, but it tends to work well anyway: https://www.science.org/doi/full/10.1126/science.aay2400

Humans are very bad at random number generation as well.

LLMs can do sampling via external tools, but as I wrote in other thread, they can't do this in "token space". I'd be curious to see a demonstration of sampling of a distribution (i.e. some uniform) in the "token space", not via external tool calling. Can you make an LLM sample an integer from 1 to 10, or from any other interval, e.g. 223 to 566, without an external tool?

> It's not that the algorithm is currently not known but it's the nature of the game that deterministic equilibrium strategies don't exist for anything but most trivial games.

Thanks for making this more precise. Generally for imperfect-information games, I agree it's unlikely to have deterministic equilibrium, and I tend to agree in the case of poker -- but I recall there was some paper that showed you can get something like 98% of equilibrium utility in poker subgames, which could make deterministic strategy practical. (Can't find the paper now.)

> I have no idea what you mean by "online search"

Continual resolving done in DeepStack [1]

> or "mechanism to ensure strategy consistency"

Gadget game introduced in [3], used in continual resolving.

> "it's likely mixed between call and a fold"

Being imprecise like this would arguably not result in a super-human play.

> Adding some form of RNG to LLM is trivial as well and already often done (temperature etc.)

But this is in token space. I'd be curious to see a demonstration of sampling of a distribution (i.e. some uniform) in the "token space", not via external tool calling. Can you make an LLM sample an integer from 1 to 10, or from any other interval, e.g. 223 to 566, without an external tool?

> You can have as much training data for poker as you have for chess. Just use a very strong program that approximates the equilibrium and generate it.

You don't need an LLM under such scheme -- you can do a k-NN or some other simple approximation. But any strategy/value approximation would encounter the very same problem DeepStack had to solve with gadget games about strategy inconsistency [5]. During play, you will enter a subgame which is not covered by your training data very quickly, as poker has ~10^160 states.

> The reason both games are hard for LLMs is that they require precision and LLMs are very bad at precision.

How you define "precision" ?

> I am not sure which game is easier to teach an LLM to play well. I would guess poker.

My guess is Chess, because there is more training data and you do not need to construct gadget games or do ReBeL-style randomizations [4] to ensure strategy consistency [5].

[3] https://arxiv.org/pdf/1303.4441

[4] https://dl.acm.org/doi/pdf/10.5555/3495724.3497155

[5] https://arxiv.org/pdf/2006.08740

Agreed. I don't know how fast it would get into the money, but an equilibrium strategy is guaranteed to not lose, in expectation. So as long as the variance doesn't make it to run out of money, over the long run it should collect most of the money in the game.

It would be fun to try!

This is the citation for [0]: Sparsified Linear Programming for Zero-Sum Equilibrium Finding https://arxiv.org/pdf/2006.03451

I agree, but they should come up with the distribution as well.

If you directly give the distribution to the LLM, it is not doing anything interesting. It is just sampling from the strategy you tell it to play.

The tournament measures the cumulative winnings. However, those can be far from the statistical expectation due to the variance of card distribution in poker.

To establish a real winner, you need to play many games:

> As seen in the Claudico match (20), even 80,000 games may not be enough to statistically significantly separate players whose skill differs by a considerable margin [1]

It is possible to reduce the number of required games thanks to variance reduction techniques [1], but I don't think this is what the website does.

To answer the question - "which 'quality' of the LLMs this tournament then actually measures" - since we can't tell the winner reliably, I don't think we can even make particular claims about the LLMs.

However, it could be interesting to analyze the play from a "psychology profile perspective" of dark triad (psychopaths / machiavellians / narcissists). Essentially, these personality types have been observed to prefer some strategies and this can be quantified [2].

[1] DeepStack, https://static1.squarespace.com/static/58a75073e6f2e1c1d5b36...

[2] Generation of Games for Opponent Model Differentiation https://arxiv.org/pdf/2311.16781

I have PhD in algorithmic game theory and worked on poker.

1) There are currently no algorithms that can compute deterministic equilibrium strategies [0]. Therefore, mixed (randomized) strategies must be used for professional-level play or stronger.

2) In practice, strong play has been achieved with: i) online search and ii) a mechanism to ensure strategy consistency. Without ii) an adaptive opponent can learn to exploit inconsistency weaknesses in a repeated play.

3) LLMs do not have a mechanism for sampling from given probability distributions. E.g. if you ask LLM to sample a random number from 1 to 10, it will likely give you 3 or 7, as those are overrepresented in the training data.

Based on these points, it’s not technically feasible for current LLMs to play poker strongly. This is in contrast with Chess, where there is lots more of training data, there exists a deterministic optimal strategy and you do not need to ensure strategy consistency.

[0] There are deterministic approximations for subgames based on linear programming, but require to be fully loaded in memory, which is infeasible for the whole game.

Interserver. But I don’t have personal experience (yet)

The problem is most developers forget a simple thing.

Hardware is the platform: https://www.youtube.com/watch?v=rX0ItVEVjHc&t=2891s

Great! I find apache arrow ipc as the most sensible format I found how to organise stream data. Headers first, so you learn what data you work with, columnar for good simd and compression, deeply nested data structures supported. Might serve as an inspiration.

Are you thinking about adding stream support? I.e something along the lines of i) build up efficient vocabulary up front for the whole data and then ii) compress by chunks, so it can be decompressed by chunks as well. This is important for seeking in data and stream processing.

I'm working on a python library that allows you to put data on GPU as fast as your network/storage allows you to at the hardware limits. No more clunky DataLoaders that are too slow and have inconsistent perf. I achieved stable throughput numbers >8GiB/s (so saturating my PC's nvme SSD), going to test on networked scenarios soon. Works for any python ML library (torch, tensorflow, jax, etc.), both for training/inference, single/multi GPUs on one node (shared/separate queues for DDP), multiple sources of data, seamless transition between training/validation phase (no significant perf drop), arbitrary shape of tensors and lots of data configurations.

Planning to add (de)compression and on-the-fly data augmentations and releasing it to public.

and prof. Demlova too :) nice to see fellows from my alma mater here!

https://math.fel.cvut.cz/en/people/demlova/

Yes exactly, that’s why I posted this. If you (or dear other readers) have been using monoio or other io_uring implementations, can you please share your experiences?

Benchmark is a bit old, but found this library to be the most interesting for thread-per-core setup.

Thank you for sharing, loved the article!

Love the egui layout! Mind sharing which ui components you used? How did you find compiling for wasm compared to native?

I think it’s because of https://en.wikipedia.org/wiki/Address_space_layout_randomiza...

My interpretation is it’s basically a hack because C programmers can’t make their libs reliable/memory safe, and it throws some logs under attacker’s feet.

There was this software called MagicErmine or statifier that could convert the two.

AE (algebraic effect) are very interesting! Great article, thank you.

Reading through, I have some concerns about usability in larger projects, mainly because of "jumping around".

> Algebraic effects can also make designing cleaner APIs easier.

This is debatable. It adds a layer of indirection (which I concede is present in many real non-AE codebases).

My main concern is: When I put a breakpoint in code, how do I figure out where the object I work with was created? With explicit passing, I can go up and down the stack trace, and can find it. But with AE composition, it can be hard to find the instantiation source -- you have to jump around, leading to yo-yo problem [1].

I don't have personal experience with AE, but with python generators, which the article says they are the same (resp. AE can be used to implement generators). Working through large complex generator expressions was very tedious and error-prone in my experience.

> And we can use this to help clean up code that uses one or more context objects.

The functions involved still need to write `can Use Strings` in their signature. From practical point of view, I fail to see the difference between explicitly passing strings and adding the `can Use Strings` signature -- when you want add passing extra context to existing functions, you still need to go to all of them and add the appropriate plumbing.

---

As I understand it, AE on low level is implemented as a longjmp instruction with register handling (so you can resume). Given this, it is likely inevitable that in a code base where you have lots of AE, composing in various ways, you can get to a severe yo-yo problem, and getting really lost in what is the code doing. This is probably not so severe on a single-person project, but in larger teams where you don't have the codebase in your head, this can be huge efficiency problem.

Btw. if someone understands how AE deal with memory allocations for resuming, I'd be very interested in a good link for reading, thank you!

[1]: https://en.wikipedia.org/wiki/Yo-yo_problem

This also called opponent modeling. Used in e.g. poker, so you can maximise your earnings.

https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=oppo...

Did you publish these by any chance?

If you want to build LLM specific to clojure, it could be probably engineered, to add the types as traces for training via synthetic dataset, and provide them from repl at inference time. Sounds like awfully large amount of work for non mainstream language.

I agree. I am under the impression that unlike Rust, there aren’t explicit types required in Clojure. (I don’t know clojure)

So there are examples online, with rust code and types and compiler errors, and how to fix them. But for clojure, the type information is missing and you need to get it from repl.

I understand the point , however I think explicit types are still superior, due to abundance of data in the training phase. It seems to me to be too computationally hard to incorporate a REPL-like interactive interface in the gpu training loop. Since it’s processing large amounts of data you want to keep it simple, without back-and-forth with CPUs that would kill performance. And if you can’t do it at training time, it’s hard to expect for the LLM to do well at inference time.

Well, if you could run clojure purely on gpu/inside the neural net, that might be interesting!

Hi, I made a new crate since I wasnt happy with the existing ones.

I published first version: https://github.com/michalsustr/rust-automata

Happy to get feedback.

Hi, I just made a library which does that!

I published first version: https://github.com/michalsustr/rust-automata

Happy to get feedback.

I published first version: https://github.com/michalsustr/rust-automata

Happy to get feedback.

Looks really neat, might use that in my work!

Well this is timely :) I’m in the middle of writing a library, based on rust-fsm crate, that adds nice support for Mealy automata, with extensions like

- transition handlers

- guards

- clocks

- composition of automata into a system.

The idea is to allow write tooling that will export the automata into UPPAAL and allow for model checking. This way you don’t need to make too much additional effort to ensure your model and implementation match/are up to date, you can run the checker during CI tests to ensure you don’t have code that deadlocks/ some states are always reachable etc.

I plan to post a link here to HN once finished.