Implementing certain things in PyTorch can be fun. I've recently started to make an economic game simulator (where you have players that can buy and sell items, do fighting etc.) and began to outline the design in the usual C#, but then, suddenly, thought - why not make the whole thing run on GPU and support millions of players with tens of millions sim steps per sec? This should make experiment cycles really fast - and that's what simulators are for!

If you don't want to program CUDA kernels yourself, what's the easiest tool for the job? PyTorch, of course. Mutable by default tensors, lots of library operations and generally a pleasure to use.

The programming is really interesting - you start to think in batches - every operation is a parallel operation, it's not 1 player/item doing something, it's a whole lot of them at once - and that's really fast.

I've eventually did run into some problems - for example, ragged/sparse tensors support is not designed for this at all, and since I have variable number of items per player - this is a problem. Also, producing something like logs inside tensors is challenging - reallocation is very expensive and must be done in batches too. There are some other interesting things like launching several operations / CUDA kernels at once.