Supercomputers will start building a 3D map of the world

It seems like they are using IMAGES of the area and then using visual processing to determine the 3D height map.

That would be intense visual processing, stitching together various angles of photos to work out a terrain elevation.

They are not dealing with a 3D point cloud of elevation data.

> calculation that can't be subdivided into parts -- e.g. weather

I'm not sure how weather simulation work but I always wondered why they aren't performed cellular automaton style. Bottom up rather than top-down. At each time step a given cell state is computed based on the state of its neighbors at t-1. This should be parallelizable. I feel it's also how it works in the real world anyway.

Supercomputers are computers with 10.000s of cores of consumer-like CPUs. So it is the opposite of your impression. They can only work with tremendously parallelizable tasks. I don't know the exact details of weather simulation or nuclear explosion but they would have to be parallelized to work on HPCs. Even if the computation is not parallelizable, the scientists can leverage supercomputers by running a simulation with randomized parameters at every node and reach a consensus result.

If you look at georeference systems, they may fit a parametric equation with coefficients to a large set of earth data. The geoid in the link below is a refinement over an ellisoid (think lumpy potato vs smooth pebble), that gives you a 3D fit that has some level of accuracy with a very compact equation compared to carrying around the raw data. I'm thinking a supercomputer might be pretty useful to do the fit of data as it's updated, as well as giving options of using equations with different costs-to-fit on large data sets, or providing better fits on finer granularity.

https://en.wikipedia.org/wiki/Earth_Gravitational_Model

Creating a 3d model out of 2d images requires computer vision to extract objects in the images and estimate their dimensions (including elevation). This will most likely require implementing an end-to-end deep learning model that's gonna need training, validation and test. Given the amount of data it'll have to deal with (100ks to millions of images) it'll need to load (high dimensional?) images in batch for them to get processed. This can still be done arguably on aws or Azure (or or...) with TensorFlow and HPC, but two things here, HPC bring a bit more overhead to the table, and a supercomputer could do better since none of the current cloud service providers have supercomputers that can compete especially in terms of cpu performance.

Good question! This does seem like an embarrassingly parallel problem. Whenever I've used the big HPC centers, the secret sauce has been a fast low latency network interconnect. The fast interconnect is useful for PDE solvers which need a lot of processor-to-processor communication (i.e. for sending data back/forth periodically along the boundaries of the grid cells you're solving for).

The distinction between "supercomputer" and "enterprise cloud" is nearly meaningless now that a cloud compute instance is considered in the top 500 fastest super computers: https://medium.com/descarteslabs-team/thunder-from-the-cloud...

I used what was then a top-10 system on the Top 500 when I worked at a national laboratory in the early 2000s. An embarrassing number of jobs in its job queue would have run well on much smaller clusters with less expensive hardware. Only once in a while would we run a single job that used more than half of the entire system and achieved decent scaling.

I suspect that the mismatch is worse nowadays. Although software and interconnects have improved, core counts and node counts have gone up even faster.

IMO simulation-guided research would probably have gone faster at the lab if the money for the top-10 system had been spent on a bunch of smaller clusters with less exotic hardware, divvied up according to the actual lines of research scientists were pursuing. But there's prestige and budgetary room for a new Grand Challenge system that may not be there for a bunch of more affordable, less exotic systems. And once in a while somebody does have a job that only runs well on the big machine.

This is also why I don't much worry about China building systems that rank higher on the Top 500 than American systems. Until Chinese research groups start churning out Gordon Bell Prize-winning software to go with the giant systems, they're probably just misallocating even more money than American labs.

EDIT: well that was arrogant and foolish of me to dismiss Chinese HPC. I looked up recent Gordon Bell Prize winners and Chinese researchers won in 2016 and 2017. It looks like they're making good progress in using those really big systems.

https://en.wikipedia.org/wiki/Gordon_Bell_Prize

Well, your impression is just wrong. Simple as that. No shame in it.

Today what is called a super computer is usually just a cluster (i.e. multiple connected normal spec computers). It is normally connected with a high speed interconnect though (100 Gbit/sec and more) what is its most defining capability.

Why they are using this cluster? My speculation is that probably because it is available and does not have much use for the real scientific computing (because it is old https://bluewaters.ncsa.illinois.edu/hardware-summary) and intelligence agency prefers to rather support the academia then feed some commercial entity.

If you don't want discontinuities at the seams between tiles, you need communication between nodes processing different tiles.

>As an example here in Australia I can get free 1m resolution DEMs of cities

source please?

MERIT is probably the best (free) LIDAR-derived global DEM right now: http://hydro.iis.u-tokyo.ac.jp/~yamadai/MERIT_DEM/

Basically fixes the problems with SRTM, but you're still left with 30m resolution.

You can read more about ArcticDEM here: https://www.pgc.umn.edu/data/arcticdem/

Seems like EarthDEM is the same. So all the advantages and disadvantages that come with stereo photogrammetry. Very high resolution, but potential for inaccuracy and quite difficult to check. Probably they used SRTM or something similar to validate their models.

>IceSAT altimetry data points are used to improve the vertical accuracy of both the DEM strips and mosaic files. IceSAT data points are filtered to exclude points in areas of high relief and over hydrographic features. Additional filtering is applied to remove altimetry points collected outside the temporal window of the source imagery acquisition date.

An xyz translation is calculated for each strip and the offset is added to the metadata file. The individual DEM strips are not translated before distribution. Users can apply their own corrections to the strip if they do not agree with the one originally provided.

Where available, additional control information such as LiDAR or surveyed GPS points have been applied.

Interestingly, that Arctic DEM covers where I live with much better resolution than the national one. On the Arctic DEM I can see my house!

However, it clearly doesn't do well with water, lots of "fake" terrain going on where the ocean meets the land.

For those interested 1m DEM and other GIS data of the United States, check out the USGS' National Map: https://viewer.nationalmap.gov/advanced-viewer/

They've changed quite a bit it since I last used it for my CS senior project a few years ago, but figured out you can see DEM segments for download by clicking the icon with the popup name "Layer List" then click "3DEP Elevation - Index", then "DEM Product Index". You will need a GIS program like QGIS or ArcGIS to view and raw DEM data.

> And I guess for the whole world you do need a decent computer.

Nope, resolution is the problem, not scale. You need low orbit stereo photos, like from an airplane. Without clouds. With satellite you only get 30m, with airplanes you get under 1m. Cities usually rent a plane a year to adjust their maps. Countries don't, they usually just rely on the cheap satellite photos.

And imagine when some internet forum decides to put a phallic object on top of some significant building, and all go vote it as legitimate. I'm not claiming it's impossible, but there will certainly be obstacles (see wikipedia). This has to be one data-set we get right; with self-driving cars coming, it could cause injury or death if we don't.

I hope so too. There should be a service that allows game developers to just import the real world. On top of that they could then customize specific locations according to the story or gameplay. Especially valuable would be an algorithm that automatically creates interiors (ideally infered from the shape of the building).

If you want to drive real circuits or fly across actual countries then real world data would be awesome, but for anything else it probably wouldn't be. Compressed game world environments (eg mountains next to open grass lands next to coastline next to forests within a few miles of each other) are designed that way because that variety contributes to the game being more fun. You don't get that in the real world.

Flightgear - https://www.flightgear.org/

edit - Can build scenery off of OpenStreetMap data http://wiki.flightgear.org/Osm2city.py

Combined with an LOD filter, this could be useful. Otherwise, not so much.

I'm confused by this comment, especially since you seem to work with super computers. One of the biggest challenges in ECP (exascale computing project) is disk IO. So much so that they were inventing complicated heterogeneous architectures. In fact many teams are just skipping disk all together and using in situ methods, only saving results. I would think that would be needed here.

But the question is what heavy processing computer doesn't have IO issues? Also, Blue Waters isn't really a GPU super computer like Summit and Sierra are (or the up coming Aurora and Frontier). It has 4228 nodes (out of ~27000 nodes) that have GPUs on them, and they only have one, and they are Keplers. Those aren't great GPUs and aren't going to do very well in parallel either. There's a big bottleneck in GPU IO. I think this program will not be utilizing GPUs very heavily. Worse, they don't have many CPUs per node. It's 8-16 cores and 32-64 GB memory per node. There's going to be a lot of time spent in communication.

I'll admit that BW doesn't seem like the best computer to the job, but you use what you got. I'll buy the argument that this is the wrong computer for the specific job, but what would you use besides a super computer? (I think Summit would be a good computer for this job)

There's not really a huge difference between supercomputers and a large number of ordinary computers these days. The computational hardware is basically the same. Only the communication interconnects are different.

I strongly suspect that they don't need a supercomputer, but they had one available and it sounds cool. I sit one building over from the ~13th largest supercomputer in the world and I've definitely used it when I needed a bit more computing power than my laptop could provide. I used the supercomputer because I could, not because I needed to.

Seems like they are using photogrammetry, most likely with a large amount of high resolution images. Photogrammetry under these conditions can be fairly computationally expensive, specially if they use aim for high quality reconstructions (which they probably should, given this is a one-time or few-time cost). Of course, it depends on what algorithms they're using, of which I have no idea, but there's plenty of fairly fancy methods, many involving deep learning in the pipeline.

Here is one option: https://www.usgs.gov/faqs/where-can-i-get-elevation-data?qt-...

USGS Elevation point query service with an API:

https://nationalmap.gov/epqs/

You say that like it's written in the article but I don't see it. And even if (!) the quality of the map were slightly below google maps, having it freely available would be a huge plus nevertheless.

Most of Google Earth is just based on publically available data from Nasa. They have suplemented it in places with commercial data.

Google Earth was not strictly a private effort. Back in the early 2000s, Keyhole (then the company that owned the software that would become Google Earth after the acquisition) was bankrolled by a venture capital firm run by the CIA called In-Q-Tel. I'm sure there are other relations Google Earth (in particular) has had with government as well.

Google earth uses older publicly available NASA data as a base and then improves it with better, often commercial, data in some interesting points. One problem with this approach is that the quality and accuracy differs wildly from place to place.

For any place in at least the western world there almost certainly exists better (and sometimes open) data than this project will provide. The big win for this project isn't that the data will always be the best available, but that it will be a single source of open data of consistent quality and format for the entire earth.