Hacker News Reader

Comment by MahiShafiullah | Hacker News Reader

Thank you for the suggestions!

1. I'm ordering a microscope mount right now! We went with 3D printing because we could get a very good coupling between the stick and the robot mount. See this figure for a comparison: https://i.imgur.com/vWopcFB.jpeg

2. Our dataset (and dataset export code) gives the depth as numpy files and trajectory information in a JSON file with calibration and trajectory information! The r3d export format from the Record3D app we used is simply a .zip file with renamed extension so we were ok with using it. Do you have suggestion as to which cinematic format you are thinking?

View on HN

Also look into the file formats that the robotics community use, stuff like rosbag and MCAP. These are formally more like point clouds than cinematic video captures and they won't integrate well with the video-processing side of things, but if you're dumping depths and trajectories from numpy and shuttling them to other robotics stuff then they might be a better fit than a 3d video format. In particular, if you end up adding anything other than video-like data streams - temperature, internal estimates from SLAM, etc - you'll have a much better time adding it to an mcap or rosbag than you would trying to shoehorn it into a video stream format.

Rosbag is just a capture of published Ros topics which can include point clouds, image/video data, imu, pose etc

Yes, Record3D is an option. The problem with it, you’ll never get large scale data in the format recorded by a custom app. If you use some standard app and format that is used by general public, there is a bit higher chance. For the iPhone with the depth sensor cinematic format from the standard camera app seem to contain depth/RGB/traj. Maybe a candidate format. Also it’s much easier to operate standard camera app.

In a longer term the stick is likely not needed too. Ideally you’d want a human demonstrating with regular human tool, like pliers to generalize.

This is a particular mount that I’ve used. https://www.amazon.com/Starboosa-Smartphone-Microscope-Unive...

It’s pretty solid, but it doesn’t open completely, so you need to disassemble and reassemble the stick to put it on (which is not that difficult).

> For the iPhone with the depth sensor cinematic format from the standard camera app seem to contain depth/RGB/traj. Maybe a candidate format. Also it’s much easier to operate standard camera app.

I agree with you there. My impression was that the cinematic format is closed, unlike .r3d which is a bunch of open format files in a .zip. Do you know whether Apple has published the specs for the cinematic video format anywhere, or if there are there good libraries for handling it?

I think you will find openexr surprisingly well supported across many platforms.

OpenEXR looks very interesting! Especially if OpenEXR can compress depth data nearly as efficiently as we can compress image data by making everything a video, it would be a game changer. For example, our RGB dataset is 814 MB right now while the RGB-D dataset is 77 GB, even though depth "just" adds one new channel to our data.

Yes, that’s another thing. When Apple is recording RGBD in a cinematic format, it is tiny.

Reply on news.ycombinator.com