I recently got a spherical camera and I am trying to use it for photogrammetry. I also have an array of four 4k cameras with hardware synchronized shutters hooked up to an NVIDIA Jetson Xavier[1]. That system can record four 4k streams at once to the SSD. I wonder how many 4k streams a Jetson Nano could record, because then you could use 16 of these [2] and four to eight Jetson Nanos to make a camera system with all hardware synchronized shutters that could easily record the data and export it via the network. It would cost around $2500 though. These projects get expensive and I keep thinking I want a sponsor, but the slow pace using what hardware I can buy is probably fine for now.
I'm trying to do a complete photorealistic photogrammetry capture of hiking trails, so I can run my robot in simulation on virtual hiking trails and train real computer vision networks. Lately I've been wondering if there is a GAN in my future...
The frustrating thing about my project is the sheer amount of computation required. I really don't need a direct photogrammetry capture of a trail, an approximation would be fine to some degree. But I take like ten gigabytes of video data and then process each frame to find keypoints, run correlation on all these points, and all this (using COLMAP). This stuff can take days to process on my desktop.
Meanwhile there are neural networks that can compute depth from video in real time, and I wonder what it would take to stitch sequential depth estimations in to one 3D model with RGB textures in one continuous calculation. There's so much research to do!
By the way I found the work in this paper pretty fascinating. [3] Facebook is working on 6dof video recording and playback, which is quite the challenge on many levels!
[1] https://reboot.love/t/new-cameras-on-rover/ [2] https://www.e-consystems.com/4k-usb-camera.asp [3] https://research.fb.com/wp-content/uploads/2019/09/An-Integr...