mythz

Had Cancelled my Claude sub after they banned OAuth in external tools, but just renewed it today after seeing their principled stance on AI ethics - they matter more when they hurt profits, happy to support them as a Customer whilst they keep them.

Servo isn't a JS engine. Do you mean why didn't they abandon their mission statement of developing a truly independent browser engine from scratch, abandon their C++ code base they spent the last 5 years building, accept a regression hit on WPT test coverage, so they can start hacking on a completely different complex foreign code-base they have no experience in, that another team is already developing?

I consider HuggingFace more "Open AI" than OpenAI - one of the few quiet heroes (along with Chinese OSS) helping bring on-premise AI to the masses.

I'm old enough to remember when traffic was expensive, so I've no idea how they've managed to offer free hosting for so many models. Hopefully it's backed by a sustainable business model, as the ecosystem would be meaningfully worse without them.

We still need good value hardware to run Kimi/GLM in-house, but at least we've got the weights and distribution sorted.

Sure he's still one of the top players, but he's not as strong this year and OP is suggesting he still has an edge against the GOAT, who this year:

- Has won Freestyle WC

- Has won SCC

- Has won 2x Titled Tuesday's

- Has won a Freestyle Friday

Hikaru can snipe a win off Magnus here and there, but I don't think there's any time control or format where he could win a long series of chess matches against Magnus.

Hikaru is either in a slump or his skill is starting to age: hasn't won Titled Tuesday since November, hasn't won Freestyle Friday this year, came last in Speed Chess Championship, etc.

We'll see how well he does in Candidates this year to see if he's still a top contender. Although I do believe this is his last chance to fight for the world title.

BearSSL by Thomas Pornin is always worth checking in on, not sure what the current status is but looks like it received a commit last year.

[1] https://bearssl.org

Using a coding plan, haven't noticed any throttling and very happy with the performance. They publish the quotas for each of their plans on their website [1]:

- $10/mo: 100 prompts / 5 hours

- $20/mo: 300 prompts / 5 hours

- $50/mo: 1000 prompts / 5 hours

[1] https://platform.minimax.io/docs/guides/pricing-coding-plan

Really looked forward to this release as MiniMax M2.1 is currently my most used model thanks to it being fast, cheap and excellent at tool calling. Whilst I still use Antigravity + Claude for development, I reach for MiniMax first in my AI workflows, GLM for code tasks and Kimi K2.5 when deep English analysis is needed.

Not self-hosting yet, but I prefer using Chinese OSS models for AI workflows because of the potential to self-host in future if needed. Also using it to power my openclaw assistant since IMO it has the best balance of speed, quality and cost:

> It costs just $1 to run the model continuously for an hour at 100 tokens/sec. At 50 tokens/sec, the cost drops to $0.30.

Not VRAM? What performance are people getting running GLM or Kimi on DDR5?

> Doing inference with a Mac Mini to save money is more or less holding it wrong.

No one's running these large models on a Mac Mini.

> Of course if you buy some overpriced Apple hardware it’s going to take years to break even.

Great, where can I find cheaper hardware that can run GLM 5's 745B or Kimi K2.5 1T models? Currently it requires 2x M3 Ultras (1TB VRAM) to run Kimi K2.5 at 24 tok/s [1] What are the better value alternatives?

[1] https://x.com/alexocheema/status/2016404573917683754

Did the napkin math on M3 Ultra ROI when DeepSeek V3 launched: at $0.70/2M tokens and 30 tps, a $10K M3 Ultra would take ~30 years of non-stop inference to break even - without even factoring in electricity. Clearly people aren't self-hosting to save money.

I've got a lite GLM sub $72/yr which would require 138 years to burn through the $10K M3 Ultra sticker price. Even GLM's highest cost Max tier (20x lite) at $720/yr would buy you ~14 years.

Not feasible for Large models, it takes 2x M3 512GB Ultra's to run the full Kimi K2.5 model at a respectable 24 tok/s. Hopefully the M5 Ultra will can improve on that.

Lets not miss that MiniMax M2.5 [1] is also available today in their Chat UI [2].

I've got subs for both and whilst GLM is better at coding, I end up using MiniMax a lot more as my general purpose fast workhorse thanks to its speed and excellent tool calling support.

[1] https://news.ycombinator.com/item?id=46974878

[2] https://agent.minimax.io

Yep the Qwen team has been churning out models for basically everything, and lets not sleep the big blue whale that started it all which is rumored to have a 1M context drop coming soon [1]

[1] https://www.reddit.com/r/LocalLLaMA/comments/1r1snhv/deepsee...

Not concerned with electricity cost - I have solar + battery with excess supply where most goes back to the grid for $0 compensation (AU special).

But I did the napkin math on M3 Ultra ROI when DeepSeek V3 launched: at $0.70/2M tokens and 30 tps, a $10K M3 Ultra would take ~30 years of non-stop inference to break even - without even factoring in electricity. You clearly don't self-host to save money. You do it to own your intelligence, keep your privacy, and not be reliant on a persistent internet connection.

We're getting high quality drops for the perfect trifecta of leading models from the Chinese models with the same day release of GLM-5 and Kimi K2.5 1T model drop a few days ago.

Despite having many great options, I end up making use of all 3. MiniMax is my fast workhorse for tool calling and getting quick responses. GLM for all coding tasks whilst Kimi K2.5 1T model has deep knowledge for everything else, with an Opus-level command of the english language. There's been many times where I've preferred Kimi K2.5 over Opus.

They announced it on twitter [1]:

> A new model is now available on http://chat.z.ai.

Looks like that's all they can handle atm:

> User traffic has increased tenfold in a very short time. We’re currently scaling to handle the load.

[1] https://x.com/Zai_org/status/2021564343029203032

It's looking like we'll have Chinese OSS to thank for being able to host our own intelligence, free from the whims of proprietary megacorps.

I know it doesn't make financial sense to self-host given how cheap OSS inference APIs are now, but it's comforting not being beholden to anyone or requiring a persistent internet connection for on-premise intelligence.

Didn't expect to go back to macOS but they're basically the only feasible consumer option for running large models locally.

Not sure how it works in other OS's but in Omarchy [1] you hold down `Super + Ctrl + X` to start recording and release it to stop, while it's recording you'll see a red voice recording icon in the top bar so it's clear when its recording.

Although as llms-py is a local web App I had to build my own visual indicator [2] which also displays a red microphone next to the prompt when it's recording. It also supports both Tap On/Off and hold down for recording modes. When using voxtype I'm just using the tool for transcription (i.e. not Omarchy OS-wide dictation feature) like:

$ voxtype transcribe /path/to/audio.wav

If you're interested the Python source code to support multiple voice transcription backends is at: [3]

[1] https://learn.omacom.io/2/the-omarchy-manual/107/ai

[2] https://llmspy.org/docs/features/voice-input

[3] https://github.com/ServiceStack/llms/blob/main/llms/extensio...

Big fan of Salvatore's voxtral.c and flux2.c projects - hope they continue to get optimized as it'd be great to have lean options without external deps. Unfortunately it's currently too slow for real-world use (AMD 7800X3D/Blas) when adding Voice Input support to llms-py [1].

In the end Omarchy's new support for voxtype.io provided the nicest UX, followed by Whisper.cpp, and despite being slower, OpenAI's Whisper is still a solid local transcription option.

Also very impressed with both the performance and price of Mistral's new Voxtral Transcription API [2] - really fast/instant and really cheap ($0.003/min), IMO best option in CPU/disk-constrained environments.

[1] https://llmspy.org/docs/features/voice-input

[2] https://docs.mistral.ai/models/voxtral-mini-transcribe-26-02

I've shipped an Omarchy MCP Server that lets AI Assistants manage your Omarchy desktop themes - switch wallpapers, change color schemes, toggle dark mode and more, all from natural language:

https://llmspy.org/docs/mcp/omarchy_mcp

Domain bought too early, Clawdbot (fka Moltbot) is now OpenClaw: https://openclaw.ai

EOL of Windows 10 forced me to, but I'm not mad - Desktop Linux is Great!

It's definitely the superior OS for modern development and general system admin, WSL/Docker always felt like an uncanny valley kludge.

doesn't work, looks like the link or SVG was cropped.

MCP allows AI Models that doesn't support Image generation the ability to generate images/audio via tool calling.

But you can just select the Image Generation model you prefer to use directly [1]. Currently supports Google, Open AI, OpenRouter, Chutes, Z.ai and Nvidia.

I tried Replicate's MCP, but it looks like everything but generate images which I didn't understand, surely image generation would be its most sought after feature?

[1] https://llmspy.org/docs/v3#image-generation-support

MCP support is available via the fast_mcp extension: https://llmspy.org/docs/mcp/fast_mcp

I use llms .py as a personal assistant and MCP is required to access tools available via MCP.

MCP is a great way to make features available to AI assistants, here's a couple I've created after enabling MCP support:

- https://llmspy.org/docs/mcp/gemini_gen_mcp - Give AI Agents ability to generate Nano Banana Images or generate TTS audio

- https://llmspy.org/docs/mcp/omarchy_mcp - Manage Omarchy Desktop Themes with natural language

I will say there's a noticable delay in using MCP vs tools, where I ended up porting Anthropic's node filesystem MCP to Python [1] to speed up common AI Assistant tasks, so their not ideal for frequent access of small tasks, but are great for long running tasks like Image/Audio generation.

[1] https://github.com/ServiceStack/llms/blob/main/llms/extensio...

I wouldn't use Claude API Key pricing, but I also wouldn't get a Claude Max sub unless it was the only AI tool I used.

Antigravity / Google AI Pro is much better value, been using it as my primary IDE assistant for a couple months and have yet to hit a quota limit on my $16/mo sub (annual pricing) which also includes a tonne of other AI perks inc. Nano Banana, TTS, NotebookLM, storage, etc.

No need to use Anthropic's premium models for tool calling when Gemini/MiniMax are better value models that still perform well.

I still have a Claude Pro plan, but I use it much less than Antigravity and thanks to Anthropic axing their sub usage, I no longer use it outside of CC.

Couldn't think of a better title, do you have any suggestions?

No custom state machine or agent, it's only a copy of Anthropic's 3 computer use tools: run_bash, edit, computer.

https://github.com/ServiceStack/llms/tree/main/llms/extensio...

It's run in the same process, there's no long agent loops, everything's encapsulated within a single message thread.

Yep, but it only supports GitHub OAuth. i.e. Content is either saved under no user (anonymous) or the authenticated GitHub User.

https://llmspy.org/docs/deployment/github-oauth