Reprojecting the Perseverance landing footage onto satellite imagery
matthewearl.github.io
matthewearl.github.io
The graph in the article can be seen as a factor graph. VSLAM systems usually have a (kind of bipartite) factor graph with vertices/variables that are either keyframes or ‘good’ features, with edges/factors between features and the frames that see them and between adjacent frames; each of these edges are the factors in the graph. This structure results in very large but sparse graphs, and there are factor graph optimization libraries that take advantage of this sparsity (e.g. g2o or GTSAM.) These libraries also use specialized optimization techniques for some of the nonlinear manifolds (e.g. SO(3)) that arise in SLAM problems.
It only reinforces that I really need to learn my matrix math.
Reminded me of the video tracking work these folks do: https://forensic-architecture.org/
30+ years ago, I had friends reconstructing crime scenes for court proceedings. Architectural drawings and 3D scenes. They used AutoCAD and AutoSolid (?). Showing stuff like blood and ballistics.
Super effective. They turned my stomach.
I don't have words for these Forensic Architecture recreations. I almost feel like that I'm there (present).
I can only imagine their future VR recreations will be overpowering.
For example for MSL (Curiosity, the previous rover) the EDL CK and SPK provides orientation and position data if I'm interpreting this description right: https://naif.jpl.nasa.gov/pub/naif/pds/data/msl-m-spice-6-v1...
The downside being that it'll take probably 6-12 months until the data is put to public
(EDL = entry, descent, landing; IMU = inertial measurement unit; PDS = planetary data system)
On earth were used to being able to use GPS for route planning. If you could use this process in reverse to constantly determine ones position in 3D space above the surface using stored satellite imagery with a downward facing camera cross referenced with whatever gyro / accelerometer based positioning they’re using I wonder if there’d be any benefit. Maybe what they’ve got already is sufficient for anything you’d want to do in the near future.
That is pretty much exactly how TRN worked for the EDL. I don't think Ingenuity has much in terms of navigation ability, probably just basic INS. But its also not intended to fly any extended distances, so it doesn't really need any navigation abilities. I'd imagine future copters would use TRN style navigation.
(TRN = terrain relative navigation)
As a benefit, the transformation could use images after being projected onto a deformable mesh to model the hills etc.
A next step could be to leave the already projected images where they are, and only draw over them, while marking the latest frame with a border. Eventually use frame sections which cover multiple frames to perform multi-frame superresolution.
Extra kudos to the author for not calling the work done in Torch "learning".
https://docs.opencv.org/3.4/dc/d6b/group__video__track.html#...
Thanks
https://arstechnica.com/science/2019/10/heres-an-example-of-...
The landing ellipse for Perseverance was 7.7km by 6.6km. The goal is to land at a safe spot within the ellipse rather than land at a specific location.
The new Terrain Relative Navigation capability determines the rovers position relative to the surface during descent by comparing camera images to onboard satellite imagery. On Earth you'd use GPS. No GPS on Mars.
Once the rover knows it's position, it can now determine the safest spot to land using an onboard hazard map. The spot it chose to land at vs the spot it actually landed at was 5 meters apart.
To add a bit more info, poorly remembered from this excellent We Martians episode[0] interviewing Swati Mohan, who is the Mars 2020 Guidance, Navigation and Controls Operations Lead and was the voice of the landing. Go listen to it!
On the way down an image is taken. Using data about how the atmospheric entry is going, and with a lot of constraints that include the hazard map and what kinds of manoeuvres are possible with the descent system (in particular it does a divert and there are minimum and maximum distances the divert must lie between), a single pixel is chosen from that image to aim for. That pixel represents a 10m x 10m square, and the rover landed with 5m of that square.
The hazard map is created from images with a 1m x 1m resolution, from one of the orbiters (Mars Reconnaissance Orbiter I think). Those images are scaled down for the hazard map, as the on-board image processing had very tight bounds on how long it could search for a valid landing site. The podcast goes into some cool detail about that whole system and its technical design.
0: https://wemartians.com/podcasts/94-guiding-perseverance-to-t...
Pershing-2 missiles had radar correlation guidance back in the 80's.
An obvious consequence of Google maps imagery and open source is that a capable college student can make an optical terminal guidance unit out of a mobile phone.
The yellow oval is the target landing zone, though it looks like it's a bit too tall on this map compared to other sources.
You can see it's landing targets within the oval here: https://www.jpl.nasa.gov/images/jezeros-hazard-map
So it looks like it landed a little over 1km from the center of the oval, if that's your question.
When precisely talking about space travel, things tend to be discussed as "nominal" instead of being on target or correct. This is because some variance is expected, and systems are designed to work successfully within that variance. In that sense, Perseverance landed within the landing oval and on a safe landing spot, so it was 0 meters away from target.
An analogy would be it hit the bullseye and got the points, even it if it wasn't exactly in the middle of the dart board.