Live demo: 1T CoE model with 330T/s on SambaNova Suite

One of the coolest things about this demo is that the token generation runs on a single node using just 8 RDUs (SambaNova’s chip).

Disclosure: I work at SambaNova and was part of the team that made this