Intel RealSense Stereo Depth Cameras | Hacker News Reader

Intel RealSense Stereo Depth Cameras

intelrealsense.com

96 points1sembiyan65 comments

Back in 2021 I needed some stereo cameras for an AR experience I was building for the 2020 World’s Fair Expo (yes, it took place in 2021 but was called 2020). We looked into RealSense, and right around when we did there was some news that Intel had either laid off the RealSense team or stopped manufacturing new cameras. Whatever it was, it made us look elsewhere, as it does not seem like Intel is serious about developing/maintaining this product line.

We ended up going with the ZED 2i from Stereolabs (https://www.stereolabs.com/products/zed-2). They work pretty well, my only issue was that their skeletal tracking doesn’t work well if you mount them vertically (which is probably a big ask, the other features do work in vertical orientation though, which was good).

Stereolabs was pretty active on the software side of things, pushing updates pretty regularly. They’d usually fix a bug within a sprint or two. The hardware is pretty simple, and I find it unsurprising that they’re still selling the ZED 2i at the same price as 4 years ago. It gets the job done, and most of the advancement in the past few years has probably come from throwing more sophisticated AI at the existing stereo video feed.

> it made us look elsewhere, as it does not seem like Intel is serious about developing/maintaining this product line

Everyone has to find out eventually, that unless „product line“ is high-margin CPUs, Intel will cancel it sooner or later. I hope they aren’t stupid enough to do that with their GPUs (again).

> I hope they aren’t stupid enough to do that with their GPUs (again).

They have to keep the GPU part alive, if only to be able to compete with AMD. It's no surprise that both the PS5 and Xbox run on a CPU+GPU combination from AMD - if Intel wants to ever get a share of the console market again (which is admittedly low margin, but extremely high volume to make up for it) they have to be able to match the kind of degree of integration that a modern console requires, and seriously I doubt that Intel will hand over enough knowledge to NVidia to get a competitive offer.

In the non-server general compute market, the situation is similar. The ARM threat all comes with established GPUs as part of the SoC, and so does AMD.

I think they were referring to the discrete/ non-integrated GPUs.

Nvidia has that level of integration too— they've been shipping integrated ARM CPU/GPU packages since 2014 on the Jetson series, not to mention the Tegra tablet that became the Switch: https://en.wikipedia.org/wiki/Tegra

It really is only Intel that isn't with the programme on this, so no wonder they're willing to bleed money to try to catch up.

Discrete GPUs on add-in cards are the high margin market in computer hardware currently and with nvidia universally hated there's a lot of opportunity in the market. The problem is, as usual, execution.

>Discrete GPUs on add-in cards are the high margin market in computer hardware currently

For AMD and Nvidia yeah, but Intel is currently bleeding money on their GPUs since they're made on the expensive TSMC nodes instead in their own fabs, and they price them very low to gain market share as sales aren't stellar due to mid performance.

The only reason they haven't cancelled it already is they're playing the long game.

The issue is if the AI bubble bursts by the time they release anything competitive causing margins to tank again.

Yep, Intel isn't serious about any product.

People criticise Google for killing products, but Intel has reached a point where their continued existence is threatened by a lack of faith in their ability to continue products without killing them. What lunatic would build on their fab when TSMC exists? And if no one is going to use Intel fabs, then what hope does Intel have to be competitive long term?

At least most of Google abandoned projects are software. With Intel, you may be left with a piece of unsupported junk.

Maybe if the fabs get spun off, they won’t have the luxury of cancelling foundry deals.

at this point, they should just discontinue the CPUs. They obviously can't compete in that space. Not being sarcastic here.

They should just sell one cpu.

At that price why not just get a full blown LIDAR from Unitree? I guess that wasn't available in 2020, but nowadays, why not?

252x252 points per second is really low for stuff that doesn't need the range and FOV and any other benefits. Azure Kinect was 1MP depth per 66.6ms in wide fov mode.

Same story back in 2016, they either laid off or pivoted heavily on the production plan. Can’t trust those real sense folks

Long, long ago the the Kinect came out and it was able to provide 3D imaging for robotics companies at an incredibly good pricepoint compared to everything that came before it. Then the company that made it created a version with just the 3D sensor and that was even better.

But then Apple bought them for what would later become face unlock. Apple wasn't interested in being a component supplier so they discontinued selling sensors to robotics companies and it was a bad time for robotics companies that had incorporated these into their products.

Into this Intel came with RealSense, with better performance in an even smaller form factor. This was really nice. Then Intel released a garbled statement about discontinuing RealSense in 2024 and everybody freaked out. A lot of companies, like the one I was working at, decided this was the time develop and in-house 3D sensor system.

Apparently Intel is spinning them out as an independent company, though, so maybe we don't actually have to worry about Intel losing interest and shutting them down.

https://www.therobotreport.com/intel-spins-out-realsense-as-...

I had both the original Kinect (xbox 360) and then Xbox One Kinect. I found that I was able to find a lot more libraries and projects around the original Kinect. Sadly, I misplaced the original 360 Kinect when I moved to a new house and have never been able to find it. I still have the xbox one Kinect. But it seems that there isn't lot of of development done around it. Even though it had more sensors and had better imaging.

I had the same frustration, I converted my Xbox One model into a webcam during Covid when high-end cameras were at a premium, but other than a head tracker for video games, never got anything else interesting going with it.

We built number of projects with the Xbox 360 one back in the day, fun toy.

I've bought some from Goodwill and yardsales for a couple of dollars

You can still buy primesense devices afaik

http://xtionprolive.com/asus-xtion-pro-live

Apple wasn't super interested in enforcing their intellectual property claims against Chinese factories producing a product they weren't competing with any more, but relying on those isn't a viable longer term plan for a commercial product.

I thought Intel shut down Realsense years ago. It turns out they didn't exactly, and the D421 module is actually a new product. Apparently they're going to spin it out.

https://www.therobotreport.com/intel-spins-out-realsense-as-...

$80 is a bargain for that module. I hope they sell them in fewer than 10s though...

It’s $800?

Edit: corrected it’s 800 for a 10 pack

https://store.intelrealsense.com/buy-intel-realsense-depth-m...

A 10-pack

Thank you, completely missed it

You're not really misremembering. There was a bunch of drama around a press release or something, with various robotics companies having to reach out to their Intel contacts to get a better picture of what was going on. The press release or whatever derived headline was indeed not quite accurate...

I've tried to use Intel RealSense in the past but found the output to be unreliable. I had much better results with Chronoptics KEA:

https://www.chronoptics.com/products/kea

In what way was it unreliable? And how does Chronoptics ToF sensor perform better?

Unreliable as in noisy in a way that I could not fix with temporal averaging. I tried to guide a robot arm to pick up the top object in a bag of evenly colored objects, but RealSense gave me randomly fluctuating values and rather large undefined areas (areas that were blocked from the view of one of the two sensors). The ToF sensor gave solid values even outside when it was snowing. Of course the ToF sensor had other problems, high requirements for power and cooling, and perhaps worse performance in very strong sunlight.

If you visit the "Buy" page, in the Breadcrumbs near the top of the page. they seem to be listed under EOL Products. End-of-life I presume, so I think we're late to the party here.

Am I crazy thinking that seems like a lot of money for what amounts to two webcams in one box? Does the hardware itself do any of the decoding of the stereoscopic image or is that all down to software?

I really have to hand it to these companies for making these sensors. They're thin margin, not a sectorial revenue driver, but still fundamental to the industry in a "who else is going to do it?" sort of way.

Every once in a while an exec will see the first part of the sentence above as a cost optimization opportunity and will set this industry back by years. This is what happened to most Kinects and previous Realsense cameras. And yet they keep coming back, precisely because they are terrible standalone businesses.

Is this going to finally replace all the Xbox Kinects everyone uses?

Underlying tech since acquired by Apple.

The final version of the Kinect, called "Azure Kinect" was based around the ADSD3100 time-of-flight sensor from Analog Devices. The Kinect has since been abandoned by Microsoft. However, Analog offers a match-box sized module ADTF3175 integrating the ADSD3100-sensor, optics and VCSEL (940nm laser illuminator) with MIPI 4-lanes output. A devkit [1] also exist and is available from mouser, digikey, et al.

[1] https://www.analog.com/en/resources/evaluation-hardware-and-...

I got the dev kit and looking for someone to team up for commercialization of the potential of one megapixel time of flight camera. There will be probably nothing better for a long time due to complex pixel analog circuitry design. It’s very interesting sensor at the moment.

You can also buy a rebranded Azure Kinect called the Orbbec Femto Bolt. Like the original Azure Kinect, the quality is amazing and blows the Realsenses out of the water.

We had serious reliability issues with USB versions that were only solved by switching to the D457 model with GSML connectors. Those needed other hardware vendors to build in support for it to work but once that was all done we never had an issue again. Happy times going from alerts daily of the camera dropping out to alerts never.

We managed to get pretty solid reliability with the USB C versions, to the point where I can't remember the last time we had a flaky robot due to this. I can't remember specifics of what we fixed (I think it was a combination of screw-in connectors at he camera, good cables, and kernel stuff). But yes... pain until then.

Recommend Zivid cameras instead of Realsense! They are industrial grade, structured light, and more pricy but worth the cost. Super impressive point clouds and 2D. https://www.zivid.com/comparison-point-cloud-quality

Anytime you see something cool with a global shutter (like this does), the next gen of the same thing will use a rolling shutter and be half or less the price.

Global shutter makes the math of everything easier, and is therefore good for getting a product out the door quicker, but rolling shutter cameras are cheaper and perform better.

Arducam actually makes one with stereo depth sensing that has global shutter. (I've never tried it though)

https://www.arducam.com/product/opencv-ai-kit-oak-d-lite-ov7...

Unless you’re moving fast, then a rolling shutter introduces that “slant” effect for captured imagery.

These have been around since at least 2016 and haven't really made any real waves... it's another technology Intel held out to dry. Stereo is a reliable tech that is relatively simple to implement but for the most part it seems to be have been left behind by other methods that are better suited for compact form factors in consumer like time of flight, or even LIDAR. Intel itself tried its hand at LIDAR, but shuttered the LIDAR real sense cameras in the early 2020s after no success.

I think the only consumer application where I know of stereo 3D being used is in hobbyist 3D scanners. I'm sure there's some machine vision applications in industrial QC, but besides that not really much else. Maybe in AR/VR but even there it seems ToF is a better match.

There certainly hasn't been a consumer use case for these.

But the RealSense line are quite commonly seen on a wide variety of robots (maybe specifically those from US-based start-ups) over the past 8 years or so, at least from what I've seen.

There are some use-cases that Apple pioneered. After they bought Primesense, who made the first Kinect using quite interesting structured IR laser light points and a single camera to get depth, Apple shrank that technology. It is now the core of FaceID where it does a 3D scan of your face. On the backside of iPhones, Apple is using LIDAR to estimate depth, allowing improved computational photography.

Unfortunately it never caught on in other devices. I'd love a more secure face-id kind of thing in my Laptop (Apple, ThinkPad etc.) that authenticates me.

Ahh yeah! That's true. I was solely thinking the realsense cameras and their general form factor not having consumer use.

Relatedly: A decade ago, where I worked, and around the time Primesense was getting bought (I think, without double checking), we bought about 250 Primesense cameras so we could build a bunch of robots with them before having to change what camera was used.

RealSense is a golden standard in Robotics research. ALOHA, which was one of the first swallows of the new AI+Robotics wave, was enabled by RealSense: https://aloha-2.github.io/

I'm confused, didn't Intel wind down the RealSense business 2021?[0]

[0]: https://www.crn.com/news/components-peripherals/intel-says-i...

Wow, I could have sworn that was a fake domain banking on r and n being homoglyphs when part of a repetition, but no, CRN is a real publication.

only the non-stereo realsense was killed. The stereo device was kept along for the ride

> Stereo is a reliable tech that is relatively simple to implement

why are the cameras so expensive then? I guess their optics aren't particularly high-grade either.

Going to assume you're genuinely interested here:

Intel's RealSense cameras typically also have an active IR illumination component that projects texture onto otherwise featureless surfaces where stereo would not give any measurements.

There is also an onboard vision processor that computes the depth information and sends that to your host system. Compare this with Stereolabs zed cameras that require you to have a separate GPU, supporting CUDA, to compute the depth image stream. Oh, and there is also an inertial measurement unit thrown in, for high frequency inter-frame motion estimation. Useful for things like visual odometey, 3D mapping, etc.

> Going to assume you're genuinely interested here

Sure I am! Thanks for the information.

One part that surely drives up the price is that they use global shutter sensors. These are typically reserved for DSLRs. Smartphones use much cheaper rolling shutter sensors. But that's also the reason why smartphone pictures get distorted when you move the camera too much. These distortions would make 3D triangulation impossible.

We use their stereo cams for pointclouds, but are not exactly happy about it. Very fragile USB-C transport, the headers are a horrible mess (only fixed in our local fork), their streaming server changed completely (we are still using the old one).

Lots of my code deals with just realsense resetting, firmware uploading just to unblock comms, restarting or even rebooting the device. And they blocked firmware uploads in their new versions, so you cannot unblock comms anymore without rebooting.

Not recommended.

Range? On the front page is says 60cm to 6m and on the other page is says 0.2m to 2m. That's a 3x difference in the same spec, which is it?

Can you clarify which other page?

Near the top of the "Learn More" page:

https://www.intelrealsense.com/depth-module-d421/

Right under the top picture of the module it says "Recommended range 0.2 to 2m" meanwhile on the main page linked from HN is says "Ideal Range 60cm to 6m"

So 60cm on the front, but one click away on "Learn More" link, it drops by a factor of 3.

That is the product page for the D421 module but "60cm to 6m" is under the D456's section on the main page. RealSense is a family of modules/cameras and each product section has its own "Learn more" link, here is the one from the D456's section: https://www.intelrealsense.com/depth-camera-d456/

Attempting to use Intel Real sense cameras on anything but Microsoft Windows has been a fools errand for me. The driver situation is awful and the SDK is annoying too.

Is it possible to take 180° 3d videos with it?

Their FOV is 75° × 50°, so no.

Intel® RealSense™ Depth Camera D456 $469.00

For that price you can get a used iPhone with LIDAR if you don't own one yet?

But how do they compare?

You can, but how well does that iPhone integrate as a ROS node?

Reply on news.ycombinator.com