Hacker News Reader

Comment by p1esk | Hacker News Reader

Wait, you haven't even built a prototype? How can you possibly know if your chip will even work, let alone be better than any existing GPU?

I'm sure you're aware that since Mead's retina chip there have been dozens of attempts to build NN chips, both analog and digital, and very few of them got further than the simulation stage (ETANN or ANNA chips come to mind), and no one managed to produce a commercially successful product.

Nvidia Tegra X1 claims to have 1Tops @10W for 16 bit precision, and the cost is probably under $100. They can probably double that performance if they drop precision to 8 bit. That's what they ship today, and next year they will release the Pascal version, which will undoubtedly will be bigger, faster, and more efficient. What makes you sure you can compete with them?

View on HN

Actually, I can give you a better answer...

An ASIC is always going to be at least 10x better than a CPU/GPU for performing the same algorithm. The question isn't whether or not an ASIC can beat NVIDIA, the question is whether the target market is large enough to support an ASIC company.

At Isocline we assume that this market IS big enough to support an ASIC. Our competition is not NVIDIA, it's the future all-digital ASIC company that can do the same thing, but without all of the whiz-bang technology. If we have to, we could probably fall-back to be that all-digital company, but I'd prefer to maintain our technology advantage.

NVIDIA's advantage is flexibility, there's always going to be a lot of demand for that.

An ASIC is always going to be at least 10x better than a CPU/GPU for performing the same algorithm.

In theory, this has always been the case. Yet every single neural net ASIC built in the last 25 years has failed in the marketplace, for the same reason - "silicon steamroller". Invariably, when the ASIC was ready to ship, which was almost always much later than was hoped for, general purpose chips have caught up in performance.

I'm not attacking your startup in particular. I'm just pointing out the history behind the field of specialized neural hardware.

p.s. Your competition is Nvidia (or Intel, or Xilinx, etc), because they are well known, big players, who produce reliable products, with huge development infrastructure and expertise. Nvidia specifically has been focusing on deep learning applications, they are already targeting computer vision for cars with their mobile GPUs. If I'm Ford or Toyota, who would I consider for partnership when I need chips potentially making life or death decisions on the road? If your technology really works (big "if", because you haven't built anything yet), then your best hope is one of those big players acquires you.

These are all good questions/points

History is something we need to contend with, not just for neural networks, but also for analog computing which has a similarly troubled past.

For NN history, there has not actually been a market for NN accelerators until recently. You can see this because:

1. No NN algorithm was worth accelerating until AlexNet came along in 2012

2. What commercial products even use NN now? Currently it is mostly just voice recognition which is processed server-side.

Right now we are not attempting to go after any markets that a GPU would be sufficient for the reasons you mention; we're sticking to products that can only work with our technology. By the time we went after an overlapping market our credibility would be established and that wouldn't be an issue.

Yes, the market for NN based products is still in its infancy. It can explode if Apple or Samsung decide to do image or voice processing locally on a smartphone, by using a coprocessor/accelerator chip alongside with CPU/GPU. It could make sense considering the expense (power, time, bandwidth costs) of sending every image off to a datacenter for processing.

I'm curious, have you considered using analog weights (e.g. floating gate transistors, or DRAM capacitors)? This could reduce multiplication from 32 transistors to just one!

Analog weights can save a lot of delay/power/cost if you can implement them right, easier said than done

I guess you'll have to wait and see

Reply on news.ycombinator.com