Show HN: Samosa Chat - Run Qwen3.6-35B-A3B Locally on a 16 GB Mac

github.com

6 points·dwa3592··9 comments

Very interesting. I am always curious on llama.cpp and vllm. All the best for samosa inference engine. One reason I am not running local models on my Mac now a days, is My Mac book pro is getting heated more. Otherwise would love to run couple (at least 1 to 3) of local models continuously.

>>One reason I am not running local models on my Mac now a days, is My Mac book pro is getting heated more

Samosa was optimized for keeping mac's temp in check. This is one of the important reasons for creating this - no excessive wear/tear to the machine. https://github.com/deepanwadhwa/samosa-chat#the-three-princi...

maybe give it a try and let me know if your macbook is getting heated while running this?

Actually quite cool. My mac is not getting heated. will check more. Memory pressure is very low. I see it (memory used) just increased from 22GB to 27Gb(on 48GB pro), while using somosa command(again a very basic test). Not really measured tokens/s, but I see it's reasonable.

Thanks for testing. Actually if you open the samosa app; it will show you memory consumption, tokens/sec etc. Let me know what you think.

Great, yes with samosa app can see details, it is 6.52tok/sec, 4.05GB memory. Use case, I have been waiting was to run some agents in my Mac continuously with local models, but without my Mac getting heated. Seems I can experiment with samosa inference engine.

+1 for pairing Samosa with Chāt :D :D

i was pleasantly surprised to find out it hadn't been done until now.

This is pretty sick. 7tokens/second is not fast but still seems useful.

any way to run samosa in opencode?

Reply on news.ycombinator.com