Tenstorrent engineers talk open-sourced bare-metal stack | Hacker News Reader

Tenstorrent engineers talk open-sourced bare-metal stack

eetimes.com

178 pointsJoachimS16 comments

The Metalium repo is here: https://github.com/tenstorrent-metal/tt-metal/

There's also an "unboxing video" from Dr Ian Cuttress and Jasmina Vasiljevic from a few months ago: https://www.youtube.com/watch?v=WJpJkvNw9Ts . It's a bit out of date because a lot of the stuff that was promised as "coming soon" has actually happened.

I personally think making this open source is huge and am cautiously optimistic that the hardware has potential for general purpose parallel computing, so I've ordered a card.

Very recently we have also opensourced BUDA, top-down software stack for running ML models on Tenstorrent Hardware: https://github.com/tenstorrent/tt-buda

Metalium being the bottom-up software stack giving open access to Tenstorrent Hardware.

Very interesting, this should broaden the case for Tenstorrent well beyond the usual kind of "AI" and into traditional HPC workloads, which will` tend to involve quite a bit of custom code and heterogenous compute where every "core" is not working on the same thing.

Yeah, I'm not an ML guy, but I was looking at the matmult.h header here, as well as the stuff in eltwise_unary and thinking there'd definitely be some more general purpose uses for this stuff.

I wonder what the memory transfer throughput/latency is from host to card and back

https://github.com/tenstorrent-metal/tt-metal/blob/main/tt_m...

Based on the name I assumed this was gonna be about a torrent site I hadnt heard of.

Transcutaneous electrical nerve stimulation torrents.

My thoughts went towards a bare-metal torrent server, suited for seedboxes.

> “We want to open [Metalium] to the point that everything is 100% transparent,” she said. “We will click ‘open’ on our Github repo, and the world will be able to see everything that we’re doing internally.

That's would be massive. Will that also apply to firmware?

Firmware can sometimes get a bit complicated due to third party IP used and whether that can be opensourced, but within those constraints this is the ambition.

I.e things like memory controller setup and link training is often constrained by the IP vendors on whether you can share it in source form.

Good news in general though -- there isn't a lot of firmware on these cards in the path of the workloads, most of it is around system management and thermals/power.

Terminology-wise some of pieces in the code base are called firmware, but are loaded at runtime and built as part of the usual build process. It's more of a linked-with-the-workload libraries than resident FW. (There's some of both though).

So right now it’s only for model inference is it and not for training

Good question. I have been compiling an analysis of the hardware market with the idea of analyzing a cloud market for LLMs. Feel free to comment: https://docs.google.com/document/d/1nehciIpaKj3-sEta5Jx2q9in...

I am mainly interested in the training side because it is the more difficult one.

Yes, Grayskull is Tenstorrent's entry-level devkit for inference only. Future generation of chips to feature training.

Correct, working on enabling training on next iterations

Didnt Tenstorrent pivot out of ML?

Curious why you think that?

The memory is too small to be useful nowadays.

Reply on news.ycombinator.com