Second version of HoloLens HPU will incorporate AI coprocessor | Hacker News Reader

Second version of HoloLens HPU will incorporate AI coprocessor

microsoft.com

142 pointsitamarb87 comments

Serious question: what is an AI coprocessor technically? Some machine learnt models burned on a chip? Or some kind of a neural net with updatable weights?

It is not well documented by anyone. However, the expectation is that it is a matrix or convolution coprocessor, as this is a common operation in deep neural networks (for both inference and training). For instance, NVIDIA says they are supporting 4x4 convolutions with the tensor unit.

One example is: https://arxiv.org/abs/1704.04760

There are many potential designs for these things, but the first gen TPU is one that works, is in production, and has been described in a paper. But you have to differentiate if you mean an inference engine, or something that can also do training. For HoloLens, it's probably going to be an inference unit, which means it'll possibly look something like a TPU, perhaps with more specific hardware support optimized for convolutions (which are very important for visual processing DNNs these days), as the NVidia tensor units are.

Probably also enough high speed memory to store the weights without needing to go to RAM.

I was in the audience at CVPR when it was presented. They were doing semantic segmentation using resnet-18, so I'm guessing it speeds up convolutions and some linear algebra during inference. I'm guessing it won't be used for training.

The AI coprocessor is probably the first processor designed directly by the marketing department...

Oh man, that ship has sailed!

According to the linked article, this coprocessor seems particularly focused on Deep Neural Networks (DNN), so it does sound like a updatable weight neural network evaluator.

A whole butt ton of GPU-style FMA and low precision float multiply ALUs would be my guess

How low precision can you get and still have it be useful?

Google's TPU proves that 8 bits is still good.

Note that the TPU used 8 bit integer math, not even floating point.

1-bit, binary neural networks work.

I used a Hololens and it was awesome (it fit over my glasses). Inference on Hololens could make you do so much more. Maybe you could use it to take scans of parts from a CNC or the CNC tool and create a system that could tell you if a similar piece will fail in the future. From what I have seen compared to Google Glass I think it could be possible for richer interactions and also it uses both eyes. But, I haven't heard any Enterprise users use Hololens.

We got a couple at my company and I've built some protoypes for it, and threw up a little blog for it. The cool thing is the HoloToolkit which seems to be maintained by some xbox developers, and scripts all the functionality you need to get off the ground.

The hard part has been coming up with prototypes that go beyond a cool experience, and solve some customer's problem. It doesn't recognize what it's looking at, as much as it's just recognizing the shape of what it's looking at. You have to write the functionality to then determine what it is. Though the HoloToolkit does have scripts that will recognize walls, floors, tables. But it falls really short of the wild ideas my co-workers come up with. There are some example projects out there that will show you how to determine faces, which is cool. But the best ideas most people have, the beta Skype app already knocks it out of the park.

What is it about the beta Skype app that is so compelling?

Here is my explanation:

The Skype call will use the forward facing camera of the Hololens user. For the Hololens user the view of who you are calling will be presented in front of you as you move around. Also, people on the Skype call can interact and draw on objects in front of you.

See https://www.microsoft.com/en-us/hololens/apps/skype.

The best possible business apps, including the Thyssenkrupp promo, revolved around conferencing with people in real time in an augmented reality space. Someone can help you while viewing what you're seeing, and interact with you using holograms, files from your OneDrive, and of course video chat. The Skype demo is by far the best implementation I've seen of this, and it's designed really well imo. The other user does have to download a skype plugin, but then they are able to interact with your space.

I've yet to test it with two devices, but there's a lot of shared experience functionality in the HoloToolkit.

I haven't used it, but I feel fairly certain that the killer app for VR/AR is going to be eye contact. Skype with eye contact would be a major leap forward. Eye contact lets you negotiate speaking/listening handoff in a natural way, instead of using awkward procedural norms.

The use cases are pretty varied. We're using some for a medical study looking to see if the HL can help lessen pain through distraction (tested against normal distractions like iPads). The HL is nice, as it lessens incidents of simulator sickness. We're also trying to get the thing to be used to scan the homes and backyards of long-term stay patients so that they can 'go back home' for a little bit while hooked into the hospital. Honestly, in our experiences, it seems like the best use of VR is 'live' events like concerts, sports games, minecraft, etc.

As an aside, you know you are a real adult when you are given the HL for the first time ever, full of 3-d potential and amazing sound and all the coolest things your 10 year-old-self could think of, and you immediately dive into the wireless and firewall settings.

The virtual home sounds more appropriate for VR than AR or am I missing something about the in hospital scenario - with AR you'd still see all the hospital just with some bright holograms trying to be "in front"

Turn the lights off?

haha, if only you could.

HL needs light for environment tracking. The best way I've found is to use sleep mask to block the light while letting cameras "see"

No it doesn't. Surface mapping is done via IR.

Turns out the patients like having the lights on, so do the nurses, so they don't trip over anything.

Pro-Tip: Want to change medicine and help the world? Find a way to get rid of the wires and tubes. It is really amazing how many wires and tubes there are in a room these days. Yeah, wireless charging and all that, but often that'll mess up a pacemaker (maybe) and so it's avoided. Also, bandwidth issues with a whole building of wireless signals. It's not easy, but it would help more so than a cancer vaccine.

You know, I've never been able to pin down what the difference is between AR and VR, everyone has a different answer. Basically, yes, we'd use the non HL ones for 'home' visits while in the hospital, if the kids can handle the nausea (they tend to be on a lot of meds, things get complicated). Again, the 'hacker ethos' doesn't really work in a hospital, moving fast and breaking things is a no-go with the Hippocratic oath.

The way it's been explained to me:

VR -> Virtual Reality is where the entire landscape is computer generated, the viewer does NOT see anything other than what the computer generates.

AR -> Augmented Reality is where the viewer is able to see around them and the computer projects images to the glass surface, at the right size and color, so that they appear to be in the world around the viewer

One of the best Enterprise uses I've heard for hololens is measuring spaces. The voices of VR podcast described a company that installs stair lift. Before theyd have to have multiple people come and take lots of measurements before the install. Now it's just one person who can get all measurements in less than 15 minutes.

The article mentions a demonstration of hand segmentation made at a conference, here's a short video of it:

https://mspoweruser.com/microsofts-next-version-hololens-wil...

Any idea who makes it? It's probably close in architecture to GPUs, the kind of thing Nvidia and AMD are positioned to capitalise on.

I suspect it will have more in common with mobile GPUs than the big iron ASICs that come from NVIDIA or AMD.

IMHO Its most likely a variant of the AMD GPU or APU, because it would take some major differentiation before nVidia would do a largely custom core processor, even for Microsoft.

nVidia's Tegra Xavier SoC is under development, with the Volta GPU architecture that would include the Tensor core which is specifically intended for deep learning and AI applications.

That's doubltful. AMD is happy to provide bespoke chips for big customers (like the XBox) but what you want for inference is to multiply entire int8 matrixies in hardware dataflow. The silicon providing float32 support in an AMD GPU isn't needed and their dataflows only extend to vectors rather than matrices. In the case where the matrix you're multiplying is much bigger than the execution hardware you can throw at it this doesn't make a big difference in efficiently using adders. But using vectors rather than matrices is hugely wasteful of register read ports if you're always doing matrix operations and don't also have to excel at operations which are only vectors.

Maybe:

http://www.general-vision.com/ ???

No idea, actually - but they are one of a handful of companies selling such a part.

I was thinking it might be closer in spirit to AMD's APU which combines graphics and compute onto the same chip.

They used customizable DSPs from Tensilica. I wonder if this is based on the same technology.

You're right, Microsoft revealed last year in a Hot Chips presentation that HPU v1 is a Cadence Tensilica DSP with custom instructions [1]. Given that, I'd bet that HPU v2's neural net core is Cadence's "Vision C5" [2].

[1] http://www.tomshardware.com/news/microsoft-hololens-hpu-arch...

[2] https://www.cadence.com/content/cadence-www/global/en_US/hom...

I'm guessing it's close to the tpu design, ie a systolic matrix multiplier.

Other possible candidates include INT8 DSPs

Altera, an Intel subsidiary. [1]

1. https://www.bloomberg.com/news/articles/2017-07-24/quest-for...

Microsoft are using Intel/Altera FPGAs in the datacenter, and not likely for mass produced devices like this.

While it is possible they are using FPGAs for the Hololens Processing Unit, I seriously doubt it. The FPGA needed to exceed the performance of a high-end mobile GPU (OpenCL/CUDA for Deep Learning/AI) would have an excessive power consumption for a mobile device.

Another factor is that higher end FPGAs are expensive ($200+ wholesale) and typically cost thousands each, and not suited to a mass produced consumer device.

As for who made this HPU, I'd say AMD helped design it and GlobalFoundries (or maybe TSMC) are doing the production.

I have a friend at Intel who works in the AR division and their comment to me is that in one way or another all AR/MR glasses are running Intel silicon. I asked them specifically, does that include, Hololens, Apple's Project, and a couple others specifically, and they answered in the affirmative. YMMV though.

Its possible it could be Intel silicon, just not a Altera FPGA.

My money is on Movidius:

https://newsroom.intel.com/news/intel-democratizes-deep-lear...

Side note in case anyone at MS happens to read this, but the only actual content on this page that appears above the fold for me (viewing on a Surface Pro 4) is the headline, which is pushed nearly to the bottom of my display by all the headers/nav/whitespace. This strikes me as remarkably bad design.

I don't know if it's designed to do that or not, but that's been a huge trend for a few years now. I still have people request that I design a site like that, where the only thing visible by default on the homepage is a 68pt logo and on specific pages is a 68pt title. A navbar at the top and a navbar under the logo are also requested, both navbars displaying the same options.

I don't get it either.

I went to a Microsoft store in the mall and they are showing off Oculus Rift. They've been showing off HoloLens for a couple years now haven't they?... still can't buy it?

You can buy it. You just have to bring your own software: https://www.microsoft.com/en-us/hololens/buy

I hope it also contains a decent CPU. The first version has a ridiculously underpowered Atom with no vector capability of any kind. Not the kind of thing you would expect in a $3000 device.

I agree that you would expect more from a $3000 computer, but that is not what hololens is. Considering all of novel sensors and capabilities with the hololens, it is pretty amazing to me that the hololens is only $3000.

Have you actually used it? I have it on my desk. It's garbage. Low res, narrow FoV display, graphics capabilities from 10 years ago. I wouldn't pay fifty bucks for it if I was spending my own money.

I have used one and everything you said is valid. It is fairly low res and the graphics are mobile quality, but that isn't what you are paying for, and calling it garbage is pretty unfair. Show me another device that is doing AR right now at the fidelity levels of HL. Your criticism is the equivalent of comparing an iPhone with a full on desktop workstation. They are not trying to solve the same problem and therefore do not need to have the same visual fidelity.

That said, IMO, it really is prototype hardware with the intention to iterate on it until it finds a market segment. I doubt MSFT is making any money on the device.

Disclaimer: Work at MSFT, have nothing to do with HL.

> They are not trying to solve the same problem and therefore do not need to have the same visual fidelity.

I think they are trying to solve the same problem. I'd love a desktop AR headset. They do need the same visual fidelity, we're just not there yet. I'm patient :)

Come on man... it's an x86 computer you can wear on your head that projects stable holographics in real time. There's literally nothing like it on the market.

Maybe the market doesn't need it?

For people who understand neural networks, this is HUGE! I have been dreaming for a custom chip for neural networks for 5 years! If they pull it off and deliver a product launch in a year or two, then they would smoke the competition again.

Google has had their TPU out for a while now. And the next generation of NVidia GPU is supposed to have neural network accelerating hardware.

But they don't have HoloLens ;)

Are you kidding me. This is basically a marketing name for an 8bit gpu

But will it be anymore practical to use than the first version?

You mean a matrix multiplication processor?

You mean headless GPU? (Or, in some cases, DSP?)

I remember HoloLens being announced to be available in "windows 10 timeframe". Still awaiting (EDIT:) Cheaper consumer version.

It has been available for more than a year.

If Windows 10 is the "forever" version of Windows with continuous upgrades then they've got plenty of time!

HoloLen hardware can be purchased here https://www.microsoft.com/en-us/hololens/buy

Hardly the device for my kids to play Minecraft on like MS hyped. Starting to feel like another Kinect - the preview movie was amazing, the actually shipped one, meh.

Wasn't it introduced at build, the developer conference? I imagine that should have given a good indicator as to their initial target market while use cases are built up for production versions. The Mixed Reality headsets seem to be in the same vain, an opportunity to start introducing that tech to developers at a lower price point.

They're not really focused on making a consumer version, they're trying to turn it into an enterprise product first

Don't they follow that model for everything? For stuff like RAM, HDD's and anything related to computers? They actually build the product for enterprise and then also sell it to individual consumers.

The "cheaper consumer version" focus for now seems to be on the Windows 10 Mixed Reality goggles coming out in the Fall. Yes, it's not HoloLens displays (it's classic LCD screens strapped to your face) or standalone computer (it's tethered to a PC), but it is cheaper and it will be consumer focused in the Fall (and it will use most of the same Windows platform support and niceties like inside-out tracking).

Well windows 10's time frame is now "as long as Microsoft is in business". It's going to be a series of beta quality builds until eternity.

Is it time to recognize that "AI" is the meaningless buzzword of 2017?

"Nothing is cooler because it is cyber." -- J.D. Falk, RIP.

"AI" might be, "AI ASIC" is pretty cool though!

Next they'll add AI to the Juicebro.

How is AI meaningless? I mean, I guess the term can be seen as silly, but I mentally map it to "statistical computing."

[edit] Since I'm getting down votes, can someone clarify? I wasn't being contrarian with the above comment. Is my mental map of what AI means as an industry term incorrect?

Any term/phrase can be rendered meaningless by overuse and misuse. The person you are responding to isn't claiming AI, as it actually is and what it actually means, is meaningless. They are claiming that "AI" the phrase is becoming meaningless.

Understood and my point was that anytime I see AI used in articles these days, they seem to be referring to statistical computing solutions. I know AI can also refer to rule based algorithms, but I rarely see it applied in those contexts these days.

Put another way, does anyone read the above statement and think "MS is developing a chip to accelerate A*?"

AI doesn't have to be statistical. It can be rule-based, for example. The reason it's useless is that knowing your headset has "AI" in it doesn't tell you anything about its functionality.

'AI' is often a delicate term (in the sense the real meaning is often hotly debated). It represents what many people worked decades of their lives for, but it is still really as hard and as abstract since the idea was first conceived.

Using the term might get points for marketing, but you'll only get contempt from the people who realize what it really stands for.

I think you've hit it on the head. 'AI' computations are a variety of mathematical and statistical techniques, which on their own are varied and interesting but it doesnt suggest a fundamental difference in hardware over general purpose computing. GPUs provide distinct advatages for some kinds of operations, but its not well defined that "AI" has its own set of computations for which hardware can be custom built for, and be distinguishable from existing kinds of CPU hardware.

"Statistical computing" is kinda outdated and stems from misunderstanding what ML is and the past reliance on some well-known statistical methods. Look at it as a parallelizable non-linear optimization able to reduce dimensionality by employing cool tricks and surprisingly efficient in some real-world tasks. If you just take a general, fully-connected DNN, it wouldn't work at all; those tricks like CNN/RNN/LSTM/GAN make it all work within a narrow setup of hyperparameters.

An "AI" coprocessor could be an FPGA, a neuromorphic chip, a DSP, a TPU, or a GPU -- it's incredibly vague.

Your definition feels pretty reasonable, but I think the issue is that the term "AI" is being slapped on tons of services that do extremely basic predictions. If it has a formula in it, it's now "AI".

AI uses more than statistics. Some algorithms doesn't use statistics at all, for example you can build decision trees using information theory instead of statistics. Statistics is a big part of AI, but in most algorithms you will find more than just statistics like optimization, algebra, ...

AI has been a word since McCarthy coined it in 1955. It is always "just around the corner", but it is hardly a meaningless buzzword.

The problem is people keep moving the goalposts. As soon as some AI technique is shown, suddenly it's just an algorithm and not actually AI. AI is something else. And then we invent that, and suddenly it's just an algorithm, not AI.

AI is magic. As soon as you know how the trick is done, it's just a trick. It's not magic anymore. There can never be such thing as AI, because as soon as it's put into a computer, it's just an algorithm.

Computers are constantly doing things that only humans could have done just a few years prior, but it's never AI. It's just algorithms.

Reply on news.ycombinator.com