Also, maybe semi-relevant: I made a small guessing game about the IKEA product catalogue in case someone wants to be lightly entertainment for two minutes [1].
31 karma · joined June 30, 2025
Also, maybe semi-relevant: I made a small guessing game about the IKEA product catalogue in case someone wants to be lightly entertainment for two minutes [1].
- More RAM: bigger models, more intelligence.
- More FLOPs: higher pre-fill (reading large files and long prompts before answering, the so-called "time to first token").
- More RAM bandwidth: higher token generation (speed of output).
So basically Macs (high RAM, okay bandwidth, lowish FLOPs) can run pretty intelligent models at an okay output speed but will take a long time to reply if you give them a lot of context (like code bases). Consumer GPUs have great speed and pre-fill time, but low RAM, so you need multiple if you want to run large intelligent models. Big boy GPUs like the RTX 6000 have everything (which is why they are so expensive).
There are some more nuances like the difference of Metal vs. CUDA, caching, parallelization etc., but the things above should hold true generally.
It has some similarities of a MoE architecture, but instead of choosing experts, it chooses layer routes. Training this NN classifier together with the LLM could condense the required amount of layers for a given intelligence down drastically if it works. If anyone wants to work on this, feel free to send me a message.
I think this is a bit dismissive towards people who create content because they enjoy doing it but also could not do it to this level without the revenue, like many indie Youtubers.
I am overengineering a simulation-based solution to this because I think there are scenarios based on cup shapes and environmental temperatures that allow either answer to be true. This will end up as a blog post I guess.
I'm currently close to the public release. After that, I want to learn some ML techniques to predict Pieter Levels' Hoodmaps classifications from my publically sourced data. It would be cool to have accurate automatic predictions of the places-to-be for every city.