This is a great read. One of the most requested AI features I’ve seen is PPTX slide generation that matches a corporate template and assets perfectly. Claude with skills comes close but not close enough. The amount of fiddling needed to adjust is nearly equal to doing it yourself from scratch. It’s a very tricky standard to work with!
Having built my own voice assistant (https://github.com/acatovic/ova) and having tried many other services and models, I feel the real win is when this is on-device, and by "on-device" I mean being very inexpensive to run on a phone, and not H100. I've now been using Pocket TTS which is super fast, and also Chatterbox and Fish Audio S2 Pro (on the Mac/PC), I feel we are so close, yet so far. The quality is amazing, but can we take this to the next level and make it run on mobile? What would it take?
I feel the single mention of `currentDateTime` against a large number of date mentions (I counted six) in the system prompt gets it confused sometimes when it comes to dates. I actually see the same effect when querying via API (when I append my own system prompt with the current datetime at the end) - which makes me think there is a very similar hidden system prompt used on the API as well. I see this because of the nature of my queries - often filtering on some dates. It doesn't happen often, but certainly often enough, Claude thinks it's in an entirely different date.
This looks great, will try it out. I had a crazy idea of doing a full agentic demo video narration and cutting, so might try your tool with some tweaks.
TTS has come incredible long way, there are so many options. There is Kokoro of course, then there is Pocket TTS which is also a tiny 100M model that allows voice cloning. There is also Chatterbox Turbo, which is bit bigger but also allows for more emotional control of the voice. And then finally there is the Fish Audio S2, which is even bigger but allows even larger and essentially unbounded finegrained control of tone and emotion. And all of these can easily run on your Macbook.
I find it astounding that ppl still comment “it’s still behind” or “it’s not the best model”. Everything is about the harness. Even the big AI labs are focusing on managing agents - sandboxes, memory, context, skills, loops. With the right harness GLM 5.2 can do no wrong.
I have the opposite experience with this, and I personally also default to simplest solutions at least as a baseline. However it’s important to distinguish between simple solutions that approximate the problem very well, to simple solutions that work in limited context or with heavy sacrifice in assumptions because those will hurt you in the long run.
I mean it's always nice to play around with sLLM finetuning, but for practical purposes I would always start with a lazy learner using embeddings (something like a small Stella model), pre-embed the topics/categories, embed the question, perform a kNN using cosine distance. You can use an LLM to "expand" the topics before embedding to make them more contextual. This is usually super fast and super simple and gives you a nice baseline. Then I would add a classification head after embedding layer (with maybe some dropout + 2-3 MLP layers) and train my own classifier, and compare that to lazy learner. Only after that would I start finetuning an LLM.
Neko Health has been doing this now for a few years. What I heard is that ultimately it doesn’t solve much (other than them privately collecting all your data) because there are lot of false positives and these false positives are deferred to the general healthcare system, which is a major bottleneck.
Can this be used for ML/AI projects as well? I'm thinking for version controlling LoRa finetunes, finetuning data (which can consist of text, images and audio), safetensors, etc?
As an amateur watercolour artist (shameless plug: https://www.instagram.com/p/DBlKG5cMPxa) I have to say the feeling your made with this wash is gorgeous. Back in the analogue world - paper grain and type/brand has a lot to do with it. Watercolour is really about unpredictability - it's about taking advantage of this unpredictability in terms of how the water travels down the grain and the impact that it makes, combined with light/shadow and "confidence" the artist brings with the brush. So of course it's never going to be truly transferrable digitally, but I still love the work you put into this.
I keep seeing these "sovereign" LMs time and time again. In Sweden we had GPT-SW3 (https://www.ai.se/en/project/gpt-sw3) and same story there. Instead of burning money on "sovereign" claims, national research labs should instead focus on building on top of solid baselines (like Qwen/Kimi) and finetuning frontier models with real agentic utility that can be applied across actual use cases and can be widely used by its people, basically for free. Nations should mirror what Cursor has done with Composer 2.5 for example.
This looks beautiful and I'm sorry the current state of affairs has made you not want to publish the code, I would love to play around with it. Regarding your decision to build - I feel you, I've had the same happen to me for everything from charting libs to various web components.
As an aside, I really like your web page - simple and clean with images and demos, no bloat.
What an awesome story. Not too many stories about Aussies out there, but what Han brothers are doing with Unsloth in AI, and stories like this one, makes this fellow Aussie super proud!
Any particular reason for BM25? Why not just a table of contents or index structure (json, md, whatever) that is updated automatically and fed in context at query time? I know bag of words is great for speed but even at 1000s of documents, the index can be quite cheap and will maximise precision
As someone who's been working in legaltech space where MS Word add-in chatbot was a killer feature, this is brutal. And in their demo they are hammering on the legal case (redline chat).
Not on the same extreme level, but I know that some coffee machines use a tiny CNN based model locally/embedded. There is a small super cheap camera integrated in the coffee machine, and the model does three things: (1) classifies the container type in order to select type of coffee, (2) image segmentation - to determine where the cup/hole is placed, (3) regression - to determine the volume and regulate how much coffee to pour.
This is beautifully written and visualised, well done! The KL divergence comparisons between original and different quantisation levels is on-point. I'm not sure people realize how powerful quantisation methods are and what they've done for democratising local AI. And there are some great players out there like Unsloth and Pruna.
This is awesome, well done. Been doing lot of work with voice assistants, if you can replicate voice cloning Qwen3-TTS into this small factor, you will be absolute legends!
Great work! Kind of reminds me of ell (https://github.com/MadcowD/ell), which had this concept of treating prompts as small individual programs and you can pipe them together. Not sure if that particular tool is being maintained anymore, but your Axe tool caters to that audience of small short-lived composable AI agents.