Hacker News Reader

Comment by strangescript | Hacker News Reader

Everyone is trying to say its better than the biggest closed models. It feels like it has parity, but its not the clear winner.

But, its free and open and the quant models are insane. My anecdotal test is running models on a 2012 mac book pro using CPU inference and a tiny amount of RAM.

The 1.5B model is still snappy, and answered the strawberry question on the first try with some minor prompt engineering (telling it to count out each letter).

This would have been unthinkable last year. Truly a watershed moment.

View on HN

* Yes I am aware I am not running R1, and I am running a distilled version of it.

If you have experience with tiny ~1B param models, its still head and shoulders above anything that has come before. IMO there have not been any other quantized/distilled/etc models as good at this size. It would not exist without the original R1 model work.

you’re probably running it on ollama.

ollama is doing the pretty unethical thing of lying about whether you are running r1, most of the models they have labeled r1 are actually entirely different models

Are you referring to the distilled models?

yes, they are not r1

Can you explain what you mean by this?

For example, the model named "deepseek-r1:8b" by ollama is not a deepseek r1 model. It is actually a fine tune of Meta's Llama 8b, fine tuned on data generated by deepseek r1.

If you’re referring to what I think you’re referring to, those distilled models are from deepseek and not ollama https://github.com/deepseek-ai/DeepSeek-R1

the choice on naming convention is ollama's, DS did not upload to huggingface that way

aren't the smaller param models all just Qwen/Llama trained on R1 600bn?

yes, this is all ollamas fault

Yeah I don’t understand why

ollama is stating there's a difference: https://ollama.com/library/deepseek-r1

"including six dense models distilled from DeepSeek-R1 based on Llama and Qwen. "

people just don't read? not sure there's reason to criticize ollama here.

i’ve seen so many people make this misunderstanding, huggingface clearly differentiates the model, and from the cli that isn’t visible

you don't mind me asking how are you running locally?

I'd love to be able to tinker with running my own local models especially if it's as good as what you're seeing.

https://ollama.com/

How much memory do you have? I'm trying to figure out which is the best model to run on 48GB (unified memory).

32B works well (I have 48GB Macbook Pro M3)

you’re not running r1 dude.

e: no clue why i’m downvoted for this

You are probably being downvoted because your comment is not very helpful, and also a bit rude (ending with "dude").

It would be more helpful to provide some information on why you think this person is not using R1. For example:

You are not using DeepSeek-R1, but a much smaller LLM that was merely fine-tuned with data taken from R1, in a process called "distillation". DeepSeek-R1 is huge (671B parameters), and is not something one can expect to run on their laptop.

Is this text AI-generated?

Probably. It's helpful tho, isn't it?

I actually wrote it myself.

I set a personal goal in trying to be more helpful, and after two years of effort, this is what comes out naturally. The most helpful thing that I do is probably not posting senseless things.

I do sometimes ask ChatGPT to revise my comments though (not for these two).

You have reached chatgpt level helpfulness - congrats!

Wasn't a value judgement

Reply on news.ycombinator.com