The future of deep learning is photonic | Hacker News Reader

The future of deep learning is photonic

spectrum.ieee.org

137 pointsAnon8444 comments

Bumped into this company some years ago that use light to perform 'random projection', which can be used to approximate matrix multiplication. Which is used a lot in the AI space (reduction of dimension etc)

How? They do the projection quite literally: with DLP projectors that project on CCD through a 'dirty' medium, for randomness. At current DLP resolutions and refresh rates, every pixel being one bit, I'll let you imagine how much data you can push through, using only 30W. The whole thing is sold as an appliance, a sort of co-processor that can do just one operation (but a really complex one)

They released their first commercial product a couple of months ago, hope it works out for them as the idea sounds pretty amazing.

edit: found their website https://lighton.ai/

Yeah, LightOn has been doing this for years already, it's kind of strange that there was no mention of them in the article. I know them because their offices are close to the research center where I work (in Paris, France). If I'm not wrong they were planning to offer their optical processor as a cloud service too.

fyi the founder Igor Carron runs a pretty nice academic blog on compressive sensing (a bit less academic in recent months)

https://nuit-blanche.blogspot.com/

Also hosts the advanced matrix factorization jungle website which is a nice browse if you're interested in mf techniques.

https://sites.google.com/site/igorcarron2/matrixfactorizatio...

his presentation on LightOn is pretty interesting https://www.youtube.com/watch?v=0QtY4_UJF0w&ab_channel=Light...

You might be well informed in compressive sensing:

What's the status of the compressive sensing research, and commercialization if there is any?

I remember that blog from when I was studying penalty/regularization methods over a decade ago. Was a great source of interesting papers and ideas.

It looks like their rack mounted 2U sized device does up to 1900 dense matrix mps.

Would you happen to know how that would compare to a typical (I guess mid-range) server for AI? I'd like to get a sense of the magnitude of improvement offered.

IMO these sorts of methods become interesting the day they are demonstrated working in a conventional AI framework on a commercially important graph like Resnet or BERT. Everything else follows from a killer demo like that.

The benefit is obvious if it works, but it also entails embracing non-deterministic results by design because it's analog. It's also a major shock to the tech stack. It took a half decade for GPUs to be noticed. This is a much more traumatic transition, no?

Yeah, I just came here to see if there was any discussion about this. I end up implementing stuff out of research papers pretty much exclusively. If I could train more efficiently, I would happily. But if I have to build something from scratch using an unconventional paradigm with no large body of research behind it, the value to me is negative, because before I even start I would have to convince my customers they can rely on it, that it matches state of the art, etc. Then I'd have to work with a much smaller set of tools built around it, including pretrained models with ultimately millions of dollars of R&D tied up in them. It's good to see new research, but this is not at all immediately helpful for applied work. Show me ResNet implemented this way, with a comparison of speedup and benchmarks on imagenet, and my opinion will change entirely.

On the other hand, will there be dramatic improvements on the software side using improved algorithms? It seems like deep learning research is currently more focused on making things possible than making them efficient.

You might want to watch Nvidia's presentation on the recent MLPerf competition. There's a tremendous amount of engineering going into making this stuff more efficient on current hardware.

https://developer.nvidia.com/blog/mlperf-v1-0-training-bench...

Most of the effort is in fine tuning the existing methods, which is important, but also leads to reinforcing the status quo in terms of algorithms. In other words, transistor scaling gets a sizable fraction of optimization, and the viable algorithms are ones most similar to dense matrix multiplication. If you unfix the algorithms (e.g., analog or very sparse models), you’ll fall off this optimization efforts and your model will eat dust.

Which is why you need a killer demo. New ideas are great, pure research rocks. But new ways to solve real world problems derived from that research is the killer product IMO.

See an unnamed AI ASIC company proudly announcing they have achieved better perf/$ without achieving better perf for how not to do this.

It’s a Catch 22. New algorithms stink because conventional hardware stinks at running them and vice versa.

Before AlexNet and GPU-optimized kernels, deep learning took too long to train large models, so large models were not used, and other ML methods seemed more promising.

I agree a killer product is what you want, but I think the incentives are wrong for a startup to get both algorithms/hardware right before funding runs out. Just getting a working compiler is out of reach of many of these startups.

We agree that the incentives for startups are completely broken right now. But until that environment changes, the bar is pretty high for disrupting current methods.

Ah… I love the new buzzword people will be throwing around soon enough. It sounds great I have to admit

“Photonic AI”. Wow.

Pretty soon Microsoft will introduce us to the P# language and have intro courses for those that don't want to deal with all that math stuff, with free credits for Azure.

The neurons in our head are analog so I'm not surprised that folks are prodding there.

I really don't like the emphasis here on circuit element sizes. If a computer the size of an elephant can train a large neural network more cheaply than 1000s of gpus why should we care?

Wasn't aware that Luminous was an SNN application. That's...far less interesting, given we can't even get electronic SNN's to do anything useful.

I think spiking neural networks are still pretty interesting, because they're the only type that we have hard evidence of being able to produce general intelligence.

I agree, though, that if they're just doing 'conventional' neural network based machine learning, it's hyperbolic to call it 'the future of deep learning'. Anything these can do, by the sounds of it, GPUs can do a bit slower.

Of course, 'quantity has a quality all of its own' and we wouldn't be doing things like GPT3 without GPUs... but then again the current approach of just throwing a bigger network at a problem seems to me to be a way to overfit your training data while pretending you're not.

Photonic in-memory ML https://www.rmit.edu.au/news/media-releases-and-expert-comme...

I often wonder if you could use different wavelengths of light to do the same. Kind of how you could mix yellow and blue to make green.

This future always seems to be just around the corner for the past decade or so. Feels like vaporware to me.

Deep learning is here, today. Not really vapourware.

AlexNet came out 2012, less less than 10 years ago.

Going from AlexNet to, say, self driving cars in under ten years sounds insane to me. This is only the beginning.

Good point I had not connected the two but that’s pretty significant!

Google has been working on the self-driving car for 12 years.

https://en.m.wikipedia.org/wiki/Waymo

They had cars on the road in 2009:

“San Francisco was one of the first cities where we tested our self-driving cars, dating back to 2009 when we traveled everything from Lombard Street to the Golden Gate Bridge. Now that we have the world’s first fleet of fully self-driving cars running in Arizona, the hilly and foggy streets of San Francisco will give our cars even more practice in different terrains and environments.”

Deep learning has a lot of problems. Energy cost is not one of them.

I have said this many times and way considered a fool. Electricity is so bad compared with light.

Part of my day job is optics design. There are some tradeoffs. One of them is that the structures of IC's are approaching a couple orders of magnitude smaller than the wavelengths of visible light. To illustrate this point, we have to use something other than visible light to make chips. Getting light through smaller structures becomes quite lossy. A way to make up for the size penalty might be to exploit increases in speed, but the structures for making high speed optics (such as femtosecond lasers) remain bulky.

I'm certainly not trying to nay-say it, and continued research is worthwhile. The history has been that when a new thing is announced that will beat silicon, by the time that thing comes to fruition, silicon has caught up through just grinding incremental improvement.

Optics can solve some interconnection issues, since it isn't confined to 2-dimensional (or "2-1/2" dimensional) structures.

If you could hit terahertz clock speeds something with only 100k transistors and hundreds of nanometers or larger feature sizes could kill modern CPUs… provided you could keep it fed with data from fast enough I/O of course.

Why is visible-light wavelength a limitation? Energy requirements?

Just technology. Visible light can use materials that are relatively straightforward: Glass, pure silica, and some other things can be made transparent to visible light. Silicon is friendly as a detector material. Getting materials, detectors, and light sources, to work at shorter and shorter wavelengths is a technological hurdle. Not insurmountable, but hard.

Loss is a main issue. It’s minimum at particular wavelengths

Dude silicon absorbs below 900nm. We are talking about IR here.

Just to clarify, so we're on the same page, silicon is the most common detector material for visible and UV. It absorbs a photon by kicking an electron into the conduction band, which is then swept to an electrical terminal and registered as a voltage or current.

Silicon transmits IR, and can be used for optics and waveguides. So there's two sides to silicon, with the transition in that regime of about 900 nm.

Actually, silicon photodiodes work down to about 1050 or even 1100 nm, but at a rapid loss of sensitivity and temperature stability. On the UV end, the silicon might be perfectly friendly, but the layers of stuff on top of the silicon become problematic. Silicon is also used for higher energy particles, just due to the havoc caused by absorbing an X or Gamma ray.

True. But short wavelengths are inappropriate for these kinds of devices generally. ~1300nm is the sweet spot for silicon photonics.

Yeah, that's actually a hassle for miniaturizing optics because at any given wavelength, you can make optics out of silicon, or detectors, but not both. The IR detector materials all have their tradeoffs, of course.

Why should we care about size? As far as I understand it real estate isn't a limiting cost for data centers

Ultimately because of cost and scalability. Photonic components are analog devices not digital so they do not have a “quantum” size that scales with process node. The size is the same in any node generally. It is possible to make the devices smaller by making them plasmonic but at the expense of loss. Since these are analog signal chains the signal to noise ratio at the end of the lossy device cascade becomes a significant limiting factor.

For scalability, the larger the die size the more it costs. The yield also becomes a problem as the die area increases due to random defects. I’d don’t understand the visible light comment since silicon absorbs below 900nm (visible and NIR).

One issue is that the speed of light is 30 cm per nanosecond. This limits the size of processors that can run at, say, GHz speeds. It's also a limitation of wires, which is one reason why the original Cray supercomputer had such a funky geometry.

When I was in grad school, the nuclear physics lab had racks of cables of different lengths that were conveniently labeled in nanoseconds.

Analog computers seem to be very well fit for running neural networks (that's since the beginning, this is not the fist time normal computers are beaten), but they are a really bad choice for mostly everything else.

In all likelihood, we will find many more usages for optical processors, but I really don't think they will ever be your main CPU.

Analog computers can't easily be mutliplexed and the integration density of memory + other compute is nowhere near as high as current SRAM/DRAM. This might change with memristive crossbars, but that still doesn't solve the structural part of the problem, since most deep learning workloads are nowadays structurally very far from a feed-forward perceptron and dynamic execution etc. are absent from analog approaches.

Modern architectures are becoming simpler, usually just stacks of transformers, and they are good for text, vision, audio, or almost any task without special modifications. That means if they could implement an optical transformer that is 100x more efficient/faster everyone will want to have it.

Reply on news.ycombinator.com