DeepSeek gives Europe's tech firms a chance to catch up
reuters.com
reuters.com
Trying to get it to generate some pretty simple verilog code with extensive prompting.
It seems really bad?
Like specify what the module interface should be in the prompt and it ignores it and makes something up bad. Utterly rubbish code beyond that. Specify a calculation to be performed yet it calculates something very different.
What am I missing? Why is everyone so excited? Seems significantly worse to me than llama. Both o1-mini and claude haiku imperfect, sure, but way ahead. Both follow the same prompt and get the interface and calculation as specified. Am I doing it all wrong somehow (more than likely)?
After fixing up my open-webui install I tried "testing 1 2 3, testing. Respond with ok if you see this." Deepseek-r1:8b started trying to prove a number theory result.
Is there a chance this thing is heavily optimised for benchmarking not actual use?
> Probably better for them to misunderstand and run 7b than run 671b. [...] if you don't like how things are done on Ollama, you can run your own object registry, like HF does.
I use Ollama for prototyping and then move what I can to a vLLM set up
I think the main point that turned me off was how they have their custom way of storing weights/metadata on disk, which makes it too complicated to share models between applications, I much prefer to be able to use the same weights across all applications I use, as some of them end up being like 50GB.
I ended up using llama.cpp directly (since I am a developer) for prototyping and recommending LM Studio for people who want to run local models but aren't developers.
But again, if you find Ollama useful, I don't think there is any reasons for dropping it immediately.
ollama has their own way of releasing their models.
when you download r1 you get 7b.
this is due to not everyone is able to run 671b.
if its missleading then more likely due to user not reading.
I'm not super convinced by their argument to blame users for not reading, but after all it is their project so.If you want the actual "lighter version" of the model the usual way, i.e. third-party quants, there's a bunch of "dynamic quants" of the bona fide (non-distilled) R1 here: https://unsloth.ai/blog/deepseekr1-dynamic. The smallest of them is just able to barely run on a beefy desktop, at less than 1 token per second.
DS has more or less been ignored for a very long time before this.
You think Ollama is purposefully using misleading naming because they're mad about DeepSeek? What benefit would there be for Ollama to be misleading in this way?
There is no benefit I think.
How do you get accurate information on the template structure?
Not that I don't believe you (I do, and I think I've seen them correct this before too), but you happen to have specific examples when this happened?
More recently deepseek 2 had a space after the assistant turn, causing issues with output quality and language https://www.reddit.com/r/LocalLLaMA/comments/1dko6rp/if_your...
Sadly, Ollama has a bit of a confusing messaging about this, and it isn't super obvious you're not actually testing the model that "comes close to GPT-4o" or whatever the tagline is, but instead testing basically completely different models. I think this can explain the mismatch in expectation vs reality here.
Verilog is relatively niche as far as programming languages go, so I’m not surprised that you’d have trouble getting good output generally. You can only train the model on so much stuff, and there is probably limited high quality training data for verilog. It’s possible the model planners just decided not to prioritize this data in the training set. 8b sized models will especially struggle to have enough knowledge about niche topics to reason over it. Anything that small is really just a language tool for NLP tasks unless it’s trained specifically to do something.
All that said, your comment does illustrate a misunderstanding with the “thinking” models. They always output a long monologue on what to say, for anything, even “hello”. It’s a different skill to prompt and steer them in the right direction. Again, small models will be worse at everything, even being directed in the right direction.
TLDR: I think you need to find a new model, or at least try the “full” version through the web app or API first.
the mind model behind it very different then that of "normal" programming languages, so less reuse of learned knowledge from other places ("knowledge" for a lack of better wording)
For such small models I would recommend specialized models only. Like Deep Seek Coder. But I think that one is lagging behind the state of art now.
Seems the explanation is the deepseek-r1 models I was using are not, in fact, deepseek-r1. Thanks all for the heads up.
They're also hyper sensitive to what I'd describe as geometric congruency between input and output: your input has to be able to be decompressed into final form with basically zero IQ spent on it, as if the input were zipped version of as yet existing output that the LLM simply macro-expanded.
R1 is just an improved LLM, nothing groundbreaking in those specific areas. Common limitations of LLMs still apply.
IMO, the layman's model of LLM should be more of predictive text than AI. It's a super fast keyboard that types faster than, not better than, your fingers.
I'm not sure where you get this generalization from. It seems like most local models you can run locally today on consumer hardware are kind of at that level, at least in my experience. But then you have things like o1 "pro mode" which pretty much allowed me to program things I couldn't before, and no other LLM until o1 could actually help me do.
Like everything. You do not even mention your machine spec, so I'm assume you just pick the ones that fit, which probably the quant versions.
Quant versions of the "small" models do not perform that well. Not the way you expected them to be.
Deepseek 8b is based on Meta's Llama3
I would say 70b doesn't worth the cost in comparison to 32b except coding... but if you want a model for coding, then you should try a model specifically trained for that, like Deepseek-Coder or Mistral's Codestral.
Deepseek-r1 is considered a general use AI. It would do good enough at many topics but won't excel on everything.
You can't be serious...I heard to open an account your are supposed to send them a fax... :-)
The UK will need to be an AI hotspot if we want to continue being a financial centre.
Not sure why AI would help there.
Founders are more likely to want to tackle smaller problems with a more secure business model.
Investors are much less likely to back high-risk/high-reward speculative business propositions.
Government will insist that we need to go slow and steady and prioritise goals like inclusivity and safety over performance.
The public will be sceptical of any large AI company, especially if they're pushing regulatory boundaries, and demand government intervention.
Every job posting I see seems to be some variation on the theme of pumping some representation of money around or retail/ecomm/HR management.
You are you going to have to support this statement a little bit better :-)
The only one close to UK in Europe is probably Switzerland and France but they too are mostly focused on research in universities rather than pushing out commercial products the way the US is exceeding at. Everyone else is not even in the game.
Otherwise, Europe is doomed to doing pointless, prematurely dead-ended one-offs.
He moved from Britain originally due to the difficulty in getting his research funded.
So the issue isn't one of intellectual capital - and while it's obviously the case that well place monetary capital is an issue - it's not clear to me what the real underlying issue is.
Perhaps Europe needs a tech/industrial revolution again - where the power shifts from the old guard to the new. Perhaps too many people in charge in Europe are from a certain class that studied history at university.
I'm not sure the problem is an understanding of economics, it's an understanding of how the real world works, and how to make it better - they are often too easily swayed by big lobby groups with vested interests.
Science and engineering isn't even a category for occupational background of UK MPs in the following report.
https://www.smith-institute.org.uk/wp-content/uploads/2015/1...
https://people.idsia.ch/~juergen/physics-nobel-2024-plagiari...
Now you could argue that the people in the 60's and 70's didn't have the compute available to make non-toy networks, and it was only applying the same techniques on bigger datasets with more compute that was the real difference.
Sure - but that happens all the time in science - every innovation is building on the shoulders of others and the assignment of the Nobel prize is as a result often rather arbitrary.
Also don't underestimate the value of reducing to practice - the difference between coming up with an idea and actually making it work in practice.
And if I remember correctly, PhDs in the UK are kind of weird compared to the US. Your thesis has to be research that you haven't published yet.
Plus, we first have to prioritize solving more urgent and important topics like affordable housing (WHEN?!), the collapsing pension and welfare systems which is a ticking timebomb, cheap energy, collapsing demographic (see affordable housing), illegal immigration, Putin's war next door, the rise of the right wing (see illegal immigration) before jumping into another pissing contest with the US and China on something that's not gonna help fix the pressing issues we have right fucking now. I don't see how we can recover from this downward spiral when I look at the inactions of our politicians who are just kicking the can down the road and blaming the EU and other countries of the union for their own systemic failures.
Winning the AI race might sound cool but it might also be similar to winning the race to the moon: a cool flex but not super useful to the general population if they can't afford a place to live or getting healthcare in a timely manner. Until ChatGPT can wipe your retired old ass in a care home I doubt many people will see AI investments as being a top priority.
Imho death spiral could be turned by providing enough affordable housing. That would be really long term goal, but the democracies do not have long term goals - the time after election ist time before election.
When a new citizen is born, there’s 18+ years for the required housing supply for that person to be created. When a new citizen is imported, they need housing TODAY. It’s just not a sustainable model on a continuous basis, but no one wants to hear that.
Germany has plenty of applied research organizations, from universities (e.g. RWTH) to things like Fraunhofer. The funding schemes behind these organizations are horrible and I would argue that in many ways, they are machines to burn up potential. Even with all this, Germany has been doing okay on the publicly funded AI research front, but that is irrelevant. The US isn't leading because of publicly funded AI effort, but because of privately funded AI effort.
Housing affordability is 100% because we let people tell their neighbors they cannot build as much housing as they want to, where they need to build it more slowly. There's no middle ground, there are no acceptable structures of land use law if you want affordability. If you get to tell your neighbor how much housing they can produce, bureaucracy will form around that and it will drive up the price of housing.
They CAN solve housing because, like you said, they're the ones causing it. They just don't want to because the housing bubble is making a lot of people rich.
The changes you see - like allowing ADUs - are inherently very limited impact so that they look good to those advocating for more housing without actually lowering prices.
No, only 3 above 3 trillion in market cap: Apple, Google, NVIDIA
US is also 3x the population of Germany.
Let us remind that the European Commission, announced with great pomp and circumstance last year that they were proudly the first ones to regulate AI: https://digital-strategy.ec.europa.eu/en/policies/regulatory...
Let's think about it for a minute. Eu was already behind the race, and they were proud for actually creating even more barriers for their businesses and researchers to catch up in the AI race.
Europe will have a change (in AI and other areas) if they get rid of most bureaucrats in Brussels. That's it. Otherwise, what expects us, is a long, slow decline into obscurantism and irrelevancy.
You know, like the kind of quality system you need in place if you make food for human consumption, produce light bulbs, or any of a myriad of other production processes. Somehow the people doing catering at my employer's canteen manage to comply with that, but it's too complicated for tech bros.
Regulation for building a house in Europe are also totally valid, and what kind of person wouldn't follow them, right?
But then you need to send a pre project for approval that takes 6 months (and pay for it), then during construction you need to get a local government worker to check the progress several times and see if the rules are being followed (and pay for it) and after you finish the house, you need to wait up to 12 months for a government official to come inspect the house and declare that your house follows all their rules (and pay for it) and you are finally allowed to live in it.
So no, let's not try and declare that these rules are obvious, and great, and we need them and what kind of people wouldn't want to follow them? When in fact, these rules mean that at every single step, you are going to wait for the government to bless what you tell them you want to do and then to make you wait again while they check if you did what they allowed you to do.
P.S. do you even have any idea of what kind of hurdles small and big companies in Europe have to go every time they need to do something just because of personal data protection rules?
Also consider this: petrochem companies would have an easier time if they were allowed to dump waste into rivers instead of having to jump hurdles to process it properly. That doesn't mean that environmental regulation is bad and we should do away with it to let them innovate more.
"The bureaucracy is expanding to meet the needs of the expanding bureaucracy"
This is somewhat misleading, because OpenAI price is for uncached and DeepSeek price is for cached. DeepSeek uncached price is $0.14.
I think most of the work done in AI here is focused on current applications of AI not on the development of new AI.
The AI race is not about innovation, it's about speculation. I do believe the tech holds promises, but as it stands now, the primary goal of AI is to attract capital. The benefits of AI are going straight up, siphoned by tech billionaires, and I don't see many improvements in the lives of my American friends.
Does it make sense to burn lot of money now? Or more to wait the technology and field to mature and then buy it for commodity prices. Think back to solar and wind power for example.
It is just the beginning.
Lower than the usual DS price? Or than OAI? Article is a little ambiguous.
Would be surprised if DS is offering 1/5 rates
Please wait a bit until I've leveled up to the capitalist class. I'll be there any day now. Thank you.
Huh? Can you mention any examples of this, specifically in tech?
AFAIA outsourcing is done largely by western countries, including the US, _to_ eastern countries, including parts of Europe, Russia, China, India, etc. This is an obvious cost cutting measure, since developers there are cheaper and the talent pool is large. This has been going on for decades. Hell, the H-1B visa is made to bring those developers in, while still undercutting their salaries compared to US employees.
Outsourcing is much less prevalent in the EU, let alone "all tech"...
> overregulation
As opposed to no regulation? The US and China are not role models for how tech companies should exist and operate in society. Whatever "innovations" are being stifled by regulations in the EU is for good reasons. Big Tech has way too much power and influence to the detriment of society. At least the EU is making an effort to draw some boundaries, and if you ask me, it's not nearly enough.
> allowing to be taken over by culture of lazy (and lazy cultures)
Yes, how dare those lazy europeans have a sensible work-life balance!
I suppose this is the plus side of picking "unstructured human language" as your API. If everything is a chatbot, then the vendor lock-in is minimal.