Hacker News Reader

Comment by frizensami | Hacker News Reader

Right, a few things would be very useful:

- Increasing the resolution of ToF cameras (right now images are around 320 x 240) --> reflections from hidden cameras can then be more detailed, whereas now it's only 1 or 2 pixels each.

- Increasing the bit-depth of ToF images - right now every pixel is only 3 bits (8 colors). It's very hard to differentiate bright hidden camera reflections from everything else, so we had to do a lot of work for that.

- API improvements in conjunction with augmented reality libraries, e.g., a) allowing Android devs to enable the flashlight when AR apps are running b) more raw access to the ToF sensor if possible

View on HN

320 x 240, is this the resolution of the depth data provided by the LiDAR sensor?

Yep, exactly.

Samsung started marketing their ISOCELL Vizion 33D camera in 2020 with 640 x 480 resolution. So it's likely we'll see better ToF resolution announced in some phones in the next year or two.

Great project btw!

Thank you! Actually, my understanding when I started this project was that I would get a 640 x 480 image (IMX516 sensor). However, I could only get a 320 x 240 image from the sensor through the Android API, so that was a bit of an oddity.

The results are even more impressive with that considered!

Thank you! Hoping for higher resolutions soon.

Hm, if "ToF sensors" are LiDARs.. why does only Apple market them as LiDAR? Why are all the "3D scanning" apps only for Apple's "Pro" devices? How come no one knows that some Android devices can do the same things? I didn't know until this thread!

My understanding is that not all ToF sensors are LiDARs. There are also different types of LiDARs, e.g., some of them scan the environment and others just emit a single beam in the same one direction. I think iPhone Pro has the scanning one, but I haven't read much about it.

I assume one is active while the other is not. With two cameras you have the option of stereoscopic analysis where you have to match pixels of two cameras looking on the scene from a different angle. That uses significant cpu-time though, difficult for real time applications. Results vary because the matching isn't trivial and the difference in angle in smartphone cameras is very low.

If you have a projector, you can do more. I believe Apple uses a flash, which has a low resolution, but is perhaps less cpu-intensive and less error prone, although it has a lower resolution. That would be a real Lidar, which is an active measurement. Of course combining that with sensible stereoscopy nets better results.

All ToF systems are active.

ToF stands for "time of flight". It works exactly by measuring how long the signal takes to go from camera to object and back [1]

Stereoscopic cameras are another type of 3R camera, they are not ToF and they are not active.

Each has pros and cons.

[1] https://en.m.wikipedia.org/wiki/Time-of-flight_camera

There's a lot of marketing involved in this naming, with Samsung calling it DepthVision and Apple calling it LIDAR. There may be a difference here, however. My understanding is that Apple LIDAR is doing what we call "direct" ToF, where the round trip travel time of laser pulses is actually measured (this can be in nanoseconds). This lines up with what self driving car (and other expensive) LIDARs do.

Most other ToF sensors use "indirect" ToF, where they measure the phase difference between incoming and outgoing signals to derive distance.

However, it gets murky as cheap 2D LIDARs on say, robot vacuum cleaners, use geometric techniques to find distance (basically return angle of a reflection). I explored this in a previous work.

TLDR: I would recommend not taking any naming at face value and reading the actual datasheet or more commonly, technical marketing materials, since few ToF manufacturers that I see have a public datasheet.

Reply on news.ycombinator.com