SiFive Rolls Out RISC-V Cores Aimed at Generative AI and ML

Pytorch XLA is such a pain to use. And once you go TPU you need the same energy to switch back, so you can’t quickly test out how it performs on your problem.

Typically Google resources go to TF before PyTorch, no?

The Groq AI chip startup has solved this problem. They don't use hand written kernels at all, instead they use a compiler, and they have the top speed in the world on LLaMA2-70B, 240tokens/s.

https://www.youtube.com/@GroqInc/videos

Other interesting Groq tidbits - their models are deterministic, the whole system up to thousands of chips runs in sync on the same clock, memory access and network are directly controlled without any caches or intermediaries so they also run deterministically.

That speeds up communication and allows automatic synchronisation across thousands of chips running as one single large chip. The compiler does all the orchestration/optimisation. They can predict the exact performance of an architecture from compile time.

What makes Groq different is that they started from the compiler, and only later designed the hardware.

Mid-journey is a good example.

It does a bunch of useful things already. Arguably most users of Copilot see it as having a positive contribution or they would not pay the price. Generative Image models are pretty good at fixing photos and creating vector art. Chatbots, even though they hallucinate, are consistently polite, helpful and efficient. They are good at sparking new ideas, playing fantasy games, debugging and rubberducking. Maybe they are also good therapists, when prompted right. They are universally used for summarisation, text based question answering, and extracting data in JSON.

> Like what? What's unthinkable tasks are so useful thanks to AI?

Erasing strangers or unwanted items in the background on pictures taken by my phone locally from my Phone's photo app was unthinkable a few years since you needed Photoshop on a PC for that. I assume the TPU is doing heavy lifting on image segmentation since the UX is pretty much touch-to-erase.

I asked SD to generate me a photo of Putin eating Turkish food and 30s later had a photo of Putin eating something that looks a lot like Turkish food. If I kept trying or threw more processing time at it, I'd have gotten something nicer. Probably.

https://i.imgur.com/y5BfxBF.jpg

Two years ago this immensely useful feat would have been impossible to accomplish in 30 seconds.

On a more serious note, ChatGPT has made made me a much more productive person. Recent example where it helped me a lot in getting an understanding of a new topic: https://news.ycombinator.com/item?id=37622335 - Even when it's making a fair amount of mistakes, it can be like a personal tutor who points you into the right directions when you're stumbling around in an unfamiliar field. Sort of like a much smarter search engine.

Granted, useful to me doesn't mean useful to you. But there's also people to whom search engines aren't useful, because they don't know how or when to use them.

Can you elaborate on the solar panel part? I haven’t heard about that before.

Unavoidable tradeoff? Fundamental tension between modularity and efficiency.

Unless you want to make application specific processors that's just a reality. We are gone include things like crypto even if the computer never uses any crypto.

The thing is, for some things you can get 100x improvement. Even if not used often its gone make a difference.

But for them most part, things that are part of RISC-V RVA23 profile make sense to be on a standard PC or Server.

Asked differently what isn't really necessary that is in RISC-V? One could make arguments for a simpler vector extensions for example. And a few other tiny things you can argue about.

Overall RISC-V RVA23 is a reasonable and feature complete set for a modern processor.

x86 on the other-hand has quite a few things that are not really needed if you redesigned it.

>They also want to add a bunch of custom extensions to make up for the lack of the standard C extension.

Was this mentioned anywhere or just a guess?

>It seems they are trying to slap a riscv decoder with a minimum amount of work.

This is interesting, and if true I would just add that they would likely sell both cores – one RISC-V and one ARM – at least in the short term. Proof is that they have already taped out and anounced thier NUVIA based Snapdragon X desktop/laptop SoC. This would further incentivise compatibilty between ARM and RISC-V as this wouldn't be a one time port like Apple did with PA Semi. I would also add that this is all speculation.

> It seems they are trying to slap a riscv decoder with a minimum amount of work.

This doesn’t make sense to me. I thought that one of the selling points of the C extension was that it was a minimal amount of work to add to the decoder.