Machine Learning for Computer Architecture | Hacker News Reader

Machine Learning for Computer Architecture

ai.googleblog.com

97 pointstmfi21 comments

Are there available AI accelerators on the market at the same level of Google's? I worry we are being locked out of compute capacity. I worry these corporations did a hostile acquisition of all the talent and critical infrastructure and we are left out. And new competitors have such an uphill battle it will be almost impossible to catch up.

You already needed a corporation just to create a regular, but state of the art general purpose processor. You're not going to do what Intel or Samsung does in your garage. Sure, with something like RISC-V you can design a processor and implement it in an FPGA, but that's going to run a lot slower than a processor that is a full-custom design.

There are some smallish competitors in the AI accelerator space (Groq and Graphcore come to mind) that are trying to innovate architecturally, but I suspect the door is closing for these companies for the reasons you suggest. The ones with some good ideas will likely be bought out by the likes of Google and Microsoft.

There are big players capable of opening it up like Philips, Sony, IBM, Microsoft, and several more.

Or even if, in a weird twist of fate, China embraces an open hardware alliance.

But it's looking very, very bad as things are right now.

China is already a big player in the RISC-V space and will likely dominate there. Ironically, this is at least partially due to the policies of the previous US Administration which is driving China to adopt the open RISC-V ISA so that they're not frozen out of US technology. But still, these are large players with a lot of money (gov funding in China's case).

> There are big players capable of opening it up like Philips, Sony, IBM, Microsoft

You forget Apple which with the M1 has proven that they can even outdo Intel. But again, Apple has a lot of $Billions to throw around. Microsoft seems to possibly also be making some processor moves, but we'll see how that works out (and again, $Billions to throw around)

Apple is definitevely part of the FAANG pseudo-oligopoly. And they are even less open than Google. They all buy up any competition who might disrupt the market. It's very sad.

Even OpenAI turned out to be propietary and practically bought out by Microsoft. And that's mostly software.

> Are there available AI accelerators on the market at the same level as Google's?

Not sure if they're quite at the same level (hard to measure apples against apples and all that), but there's a few companies in the space - namely, Groq, Cerebras, Tenstorrent, and Untether. Besides that, both major FPGA vendors have ML inference IP available.

I'd also bet other FAANG-ish companies are trying, besides just Google, but I would expect anything to come out of them to also be compute-as-a-service like Google's hardware.

Nvidia is another big player, as is Tesla. One nice thing about California law is that employees can freely move to another company and work on similar things (no strong non compete enforcement, IANAL). So they can work at Google and design the good stuff and then move to other places and keep going. Tesla’s chip designer came from Apple for example.

> And new competitors have such an uphill battle it will be almost impossible to catch up.

That's almost all of the science, engineering, industry and bureaucracy of modern civilization in general, not just the chip industry. Experts and organizations are more and more piling atop an impossibly high, increasingly disorganized and fragmented tower of knowledge without leaving formal markers and scaffolds to lead those who will come after them. How much of the knowledge that goes into manufacturing a modern car exists in a single resource called "How to build a 2020 car" ? hell scratch that, how much resources remotely detail how to build a single decent 90s-grade engine ?

The fact that this is google is only tangentially related as it's only a subcase of the general problem that corporations hoard knowledge and research into proprietary "solutions" and "intellectual property": this would still be nearly just as dangerous if those building the chips were a bunch of highly-funded university researchers who know each other by name and are separated from the rest of humanity by billions of dollars of funding and literal lifetimes of knowledge that they're adding to without teaching to others. Not that I blame them (the words might sound as if I am), this is a far greater mess than any single one cause.

It's an astonishingly underappreciated problem of modern civilization how black-boxy and fragmented it's institutions and knowledge have become.

> In “Apollo: Transferable Architecture Exploration”, we present the progress of our research on ML-driven design of custom accelerators.

AI is now designing better hardware to run AI which will be able to design even better hardware to run AI on...

When the AI realizes its purpose is to make a better AI.

Just to calm your nerves: place and route algorithms are often classical AI anyway, the things that is changing is what still is and isn't referred to as "AI". The cool thing about this work is that it is looking at higher level parameters about layout instead of the kind of heuristic algorithms you see in APR. Would be cool to see this in tools one day.

Skynet is coming ...

But since it only speeds up part of the design process, they can’t build the hardware much faster than before and cycle time is very slow. If you’re going to worry about this at all, you should be a lot more worried about AI-designed software.

"However, the manifold of architecture search generally contains many points for which there is no feasible mapping from software to hardware."

I'm having some trouble understanding how this manifests itself. Can someone help me with this by e.g. providing a toy example?

It means the ML algorithm can propose designs that do well on the objective function (e.g. improved runtime), but can't actually be constructed. They give the example of designs that have more memory than can actually fit on the chip.

Yes I understand that, but if that is what is meant I find the wording to be somewhat strange. They mention not being able to find a "feasible mapping from software to hardware", and later on "some of the constraints may not be properly formulated into the optimization, and so the compiler may not find a feasible software mapping for the target hardware".

So the problem is that there is no software mapping, which I understand to be the mapping of compiler instructions to the underlying hardware. It looks like I'm missing something. Is this the same as saying that the hardware design is not feasible?

I imagine they have a basic design in verilog with various tunable parameters (memory size, clock speed, how many instructions to issue at once).

They also have a way to run that hardware in a simulator and see how quickly it could train some network.

The ML optimization problem is to come up with a bunch of constants which performs well, but also compiles into a manufacturable chip. Clearly setting the clock speed to 9999Ghz isn't that...

But what I don't understand is: they claim their approch side-steps the "unfeasible" configs, which is and would be a major achievement, however I don't see how the unfeasibility is captured in their evaluation function, which measures mostly runtime and area, and none of them give negative clear negative rewards to unfeasibility since for example, as you noticed, unbuildable configs would return high runtime... Area might correlate negatively, but at that point I don't see how some methods work (eg evolutionary algorithm) and others really don't... Did you understand that part?

Can anyone say the singularity is upon us?

No. Why not value an achievement as is. "AI"/Computer assisted chip design is great.

Reply on news.ycombinator.com