Ollama started out as the clean "just run a model locally" story, but now it’s branching into paid web search, which muddies the open-source appeal. Windows ML goes the other direction: deep OS integration, but at the cost of tying your stack tightly to the Windows ecosystem, very reminiscent of DirectX.

Now, the real question is whether vLLM/ONNX or just running straight on CUDA/ROCm are the only alternatives or we are all trading one vendor lock-in with another.