npodbielski

What a messy thing to do. Maybe fine for a project but a desktop? No.

Yeah, I did the same. Now I am happily running Flash Next.

For that price? It is better to buy DGX Spark or Strix Halo 50% more RAM is not worth about 5% more performance for 3 times the price.

I have no idea why but I was thinking about Optimus Prime and Megatron when I was clicking the link. I was a bit disappointed.

That is good too though.

When I changed the number of draft tokens to 3 in both, it helped and they Draft is actually performing a bit better:

- draft: 67.17

- MTP: 64.18

Why they used those examples? Seems strange.

Which was not he point because I was testing their solution for MPT and it was just funny addition. But of course in internet you always will find some 'well akchually' person straight from the meme.

Well I tested it on 7900XTX with the same prompts and their draft model gave me about 30t/s. Their own snippet of code with regular MTP model gave me 60t/s.

Also model with their draft answered incorrectly. With MTP it answered correctly.

Question was: "Does MikroTik CRS312-4C+8XG-RM have combo ports?". The answer is Yes.

I am using Flash Next for few weeks and it is very capable model. I just wish there would a way to have faster prefill because reloading longer sections of session sometimes can take even 2h. I stopped using Qwen 3.8 27B completely on my Strix Halo.

Yes, for fun I tried OVH AI Endpoint and they do not have cache read at all. They bill you every time you send a prompt regardless if you are hit cache or not. One agent session was like 80M input and 300K output and I paid 30$ for that. Or rather I interrupted it and let my local Qwen finish it because cost was getting radicoulous.

what would be the usecase? sharing account with kids? it is easier to just install something else and register on throwaway email?

Or two gorgon halos?

Looks like really great tool to generate some graphs and diagrams for static file blogs.

Wow. I just read couple, but... That seems terrible! People do that?

On the other hand my own father believes now he is an alien from outer space because someone generated stupid youtube video...

I am running it on 32GB and I did not saw model loosing it context even after 4-5 compactions in pi. I am running sessions for few days sometimes. I think it looped once, but loop police extension stopped it. The only problem I have know is how pi compaction works, which is forcing full prefill which takes time and it is erroring a lot. I wrote my own compaction that should remove full prefil but it does not work. But this is the only problem with this setup and it is more problem with pi then the model. I much more prefer it to use Qwen then paid models: Claude forces me to do reauth every other day and codex models either are too costly or not capable enough.

I bought several cameras and they unvr early this year. It works great but I have no AI features because you have to buy they hardware just to accept some license (!). I mean... OK but no.

If they would allow me to send them email saying: "I hereby promise I wont record my neighbours having little tete-a-tete."

Sure I understand liability. But requiring consumer to buy another product to fully utilize they other product? Sorry but no. They have really cool hardware and nice UI but policy like that... no.

What about Raspberry Pi is not open?

As you said: everything works on llama.cpp Why it does not work on vllm? Of course you can say that it is AMD fault but there was an issue of abysmal performance of models on Strix Halo, that is open for half a year (https://github.com/vllm-project/vllm/issues/34579#issuecomme...) and nothing is happening there. They do not care about those use cases. Seems like they are going with bit players that will run vllm inside datacenters racks. Hobbyists does not matter.

And it fails on rocm of course. This engine is such a hassle on AMD.

Anybody was able to run this model in a server?

If it is not hard, with an experience of this guy, he could came up with a better example? It is your own words so I am sure you will agree? He did alright job so he could exercise gray matter a bit and came up with some API for saving customer data and then test it his way? I mean... This would be more interesting and "real world" so I am sure it would be worth to write longer blog post about it!

It spits out password in logs according to gif. No thank you.

What I do not like about that kind of example is its abstractiveness. Yes sure you can argue about testing `doSomething()` and it all falls apart when there is an actual business scenario to test.

It is. I am running it on R9700

On the other hand I am running this model to write some tests for my hobby project for two days now and it is able to deduce and fix errors and bugs that Qwen 3.6 was not able to. Yes, it thinks a lot but this makes reasoning about problem much better. Also it did not run it self into a loop once even which is a problem with Q4 even with dense models.

On the other hand it maybe do too much i.e. I asked "how we could test it?" and instead of answering it just actually wrote tests. But it was the same with Qwen 3.6.

This is not true. As author points out his domain is invalid unless you install something. So the whole thing is misleading.

What a terrible website to open on your phone. Text and image is clipped, it jumps up and down, loads for several seconds, refreshes entire page several times... I opened it and have no idea what this hardware supposed tonbe.

so it is just llamafile with few additional application in bundle?

Yes, the best way to land yourself a good gig is to have a friend in the company already and will put a good word.

In my case I would say they are comparable but moe models are looping and getting lost a lot more than dense models.

On the other hand having 90t/s with any local model is nice and Pi with loop police extension can prevent looping a lot.

How people do such nice animation? Is there is some nice program you can use? I looked and I could not find anything that seemed really easy to use. Llms can sometimes generate something usefull and sometimes something attrocious. I remember few months back article where someone create animation of how 3.5" 1.4MB disk is build, somebody asked about this in the comment and author said it was done mostly by hand. While this is impressive it would be cool to be able to draw some animation for work prestnation that looks nice but also so I do not have to spend 3 days crafting something like that. Would be nice to draw few of them in my blog too. Any recommendations?