IBM analog AI chip could give the Nvidia H100 a run for its money
techradar.com
techradar.com
IBM blog post: https://research.ibm.com/blog/analog-ai-chip-low-power
Also I'd like to mention that those chips are for inference only. You still need GPUs for training.
Imagine a good inference engine right in a phone.
Modern IBM isn't what it once was.
Edit: to clarify i mean on the software side. I don't know enough about the hardware side to comment.
essentially they develop and license technologies around the chip making process
I just feel like IBM has a track record recently of doing great research but not actually manufacturing anything. Kind of like Google.
Some of the research does end in such products, that are outside HN radar, because IBM isn't cool, yet many HN startups won't achieve a legacy like it and still be pumping money like IBM, even for boring business.
More like outside people's radar because IBM only sells to big companies.
I’m sure if I wanted to move to the team it would’ve been possible to play with z/OS.
It is a matter of being on the right team.
That doesn't make them competitive in new markets, though.
I won't be around in 100 years either but I am sure IBM won't survive for the same reason I'm sure the sun will rise: all rigorous analysis today indicates it.
They just haven't done a whole lot lately, and most of what they have done is support people locked into their legacy solutions.
The chip in the article gains its advantages from doing the compute directly in the memory cells. It saves power by moving information around less.
(Quick google found this fun hobbyist article of a perceptron built with op-amps: https://www.nutsvolts.com/magazine/article/the_perceptron_ci...)
While there is a bunch of research for modeling organic brains, IMHO it is primarily driven by neuroscience trying to understand how humans work, and not directly applicable to making computation more efficient.
Then you can kind of walk it to "but what if it kinda worked like [my understanding of how the brain works], would it be more efficient?", but it's not really clear what conversation we're having at that point.
Considering IBM likely isn't going to crater in stock like some riskier bet might, and even if it's just to benefit from initial hype, I don't see a negative in moving chips I have laying around to big blue even though I very rarely do trades nowadays.
So about about 1000x away from being useful? Most current models are billions not millions.
It's not as if we said wow, all you need is attention, and then Nvidia built some GPUs for it.
Cutting energy costs by an order of magnitude could be enough motivation, but you'll still need to demonstrate competitive model performance.
https://en.m.wikipedia.org/wiki/Foveon_X3_sensor
There is a book called The Silicon Eye about the history, and not coincidentally Carver Mead also wrote a book on neuromorphic analog neural network implementation back in the 80s.
From what I recall there are analog feedback stages between adjacent cells, and that was a ongoing theme throughout the work of the teams around Mead. His neuromorphic book is almost entirely about doing that to sound and images.
As an aside the Foveon cameras are worth experiencing. They are amazingly slow, the colours go wrong in less than perfect lighting (the stacked filter) but the edges on objects in resulting images have a definition that you only realize bayer filters completely destroy when they are gone.
Of course today the actual resolution of more conventional sensors dramatically exceeds the best foveons to the point where this advantage is nullified.
If you only mean for old digital cameras, then sure. But you're not seeing bayer artifacts in any modern camera. Rolling shutter artifacts, on the other hand..
https://www.digitalcameraworld.com/news/sigma-will-never-giv...
Back to topic, I believe this type of tech (analog/physical computing, processing and memory in the same unit) will become very significant on a 10+ year horizon. Hinton has been hinting at some collaborations recently
The point is you might think that if you were blissfully unaware of anything else. Even if you're not you shouldn't, because edge directed upscaling is imagining information into the sensor data that isn't actually there.
The two main gotchas of Foveon sensors in its days(so far) were while SIGMA maintained that Foveon pixels are worth 3x Bayer, the sensor resolutions were consistently 1/3rd or less than competitors to begin with, and also that the sensitivity was atrocious. ISO800 was pushing it for early models and recognizable colors at ISO3200 in later models was an achievement in its own(they lose chroma before luma). Fujifilm did better than that in films!
SIGMA Corporation in Japan was virtually the only user of Foveon sensor, in their DP and SD series cameras. Photos taken with e.g. dp2 Quattro in 50mm, DP1x in 28mm are easily found on Flickr. To me a Foveon image looks like it's taken through a piece of radiation shielding glass and later color corrected, which I'm guessing to be close to literal description as the color components are indeed captured through preceding luma and first chroma layers.
The "Quattro" generation Foveon used 2x2 subpixels for the top layer. My impression then was it was a last-ditch effort to mitigate poor sensitivity, but it's interesting how it was arguably an only analog convolutional network processor in a consumer product, from a different perspective.
I do with that at least _tech journalism_ outlets would develop a habit of asking at least one followup question when a PR says that something is like the human brain. Having analog values between zero and one is not at all particular to the human brain, or any brain, where we have spikes occurring at varying rates, not scalar activation levels. This chip could equally well be described as having components which mimic a dimmer switch, or perhaps a gas gauge that varies between empty and full. Yes, that's cool. No, it's not really brain-like.
At conferences I used to go to, they'd just phack the hyperparameters of an ensemble model on the benchmark sets, and put out a press release saying they were state of the art. They were mostly ignored by the academics who went to the talks on the actual novel work.