Etched Is Making the Biggest Bet in AI | Hacker News Reader

Etched Is Making the Biggest Bet in AI

etched.com

45 pointsCompuIves17 comments

How will these companies replace the software? Don't they need something similar to CUDA?

(I work at Etched.) You need something as complex as CUDA only to support general-purpose programmability; Sohu is built for one thing and one thing only: transformers. So while we certainly need a software stack to harness the chip, it’s much easier to do so, and even easier then to adapt existing LLM serving tools (vLLM, etc.) to use this stack.

> Sohu is built for one thing and one thing only: transformers

Thanks for clarifying this. Could you clarify whether your chip supports the transformer architecture in general, or only specific models for e.g. Llama 70B? In case of the latter, would your ASIC have to be reprogrammed for each model?

Transformers in general. There’s no reprogramming of the ASIC needed, just applying a different sequence of layers, and that’s exactly what our software stack is meant to support.

"In five years, AI models became smarter than humans on most standardized tests."

This seems to be a novel definition of "smarter" - one could also argue that the printed answer key for a standardized test is smarter than most humans.

Wow.. I wonder how this compares to groq?

I made a request to access their developer cloud. Anyone have any idea when they start processing those requests and how many slots they might have?

Was curious about this as well, seems like the closest comparison today is to Groq, wondering how it holds up.

There are a couple now. Groq , sambanova and tenstorrent come to mind.

The more the merrier. If we do need power plants worth of juice then it may as well be dedicated custom hardware to optimise this to the max

Thanks for telling me about Sambanova. Too bad they don't have an API. The speed is absolutely incredible. They actually have a demo where it just keeps re-running the chat completion as you are typing your query.

>Too bad they don't have an API.

Think they do, but I couldn't sign up...the verify email just never arrived and eventually got fed up.

Where is the API sign up?

A coworker of mine recently got access to Groq. Not sure how many slots there are but we have a pretty strict rate limit for what thats worth

I have access to grow, was asking about sambanova

Etched has an interesting approach, I am not convinced in their public info because it is vague. Could be great or could be bullshit. Well certainly some marketing.

> Specialized chips are inevitable

I mean, are they? Seems like the industry would prefer these things to become commodities, especially if it helps with portability and reproducibility.

The specialized chips may very well become commoditized over time. It is the way things go if the market is large and competitive enough - and that there is sufficient competitive advantage over the existing types of devices (GPUs in this case). For a rather recent example, see Bitcoin mining ASICs.

Of course GPUs (and CPUs, and pre-transformer/LLM type NPU/TPUs) are also going to respond to better suit LLM workloads. So we may see a convergence in capabilities.

What is for sure is that the LLM chip architecture is not yet settled. Currently available chips are mostly designed pre LLM craze, with slight adaptations. And have large areas of potential improvement, at least 10-100x (maybe 1000x) in power efficiency. And power efficiency is a key element (in addition to up from hardware investment). Which is the opportunity this startup (and others) have identified.

Given that Moore’s law is slowing down, given that circuit design is getting more automated and given the rise of chiplet architecture, I expect that you will have more specialized chips.

Reply on news.ycombinator.com