Hacker News Reader

Pretty sure the whole reason Ollama uses raw hashes everywhere is to avoid copying the whole NN gigabytes every time.

Maybe I am doing something wrong! When I change parameters on the modelfile, the whole thing is copied. You can't just edit the file as far as I know, you have to create another 38GB monster to change num_ctx to a reasonable number.

The parameters (prompt, etc.) should be set only in the new modelfile (passed to `ollama create`), using a FROM referencing the previous ollama model. Parameters in a Modelfile override the hard-coded parameters from the GGUF itself (which are sometimes buggy); in fact from elsewhere in the thread it sounds like Mimo is missing proper stop tokens, or maybe templates in general; I'm not an expert).

This will show a separate entry in `ollama list` but only copy the Modelfile not the GGUF.

Alternatively, if you use the API, you can override parameters "temporarily". Some UIs let you do this easily, at least for common parameters.

Reply on news.ycombinator.com