Scientists begin building highly accurate digital twin of our planet | Hacker News Reader

Scientists begin building highly accurate digital twin of our planet

ethz.ch

I am highly skeptical of this approach. Not only has this been attempted before, it appears to be on course to repeat the same technical design mistakes that caused prior attempts to fail. You can't decompose weather/climate style supercomputing models below a certain resolution because the fundamental characteristics of the computation are not meaningfully representable that way. Scalable models that correctly and efficiently capture the macro effects of many types of sparse, high-resolution dynamics require much more sophisticated computer science, systems engineering, and ways of representing data. You can't brute force it.

In particular, this is a notoriously poor approach to modeling complex large-scale interactions between humans and their environments. There was a study I was involved in last year to determine why one of the epidemiological models for COVID was so badly off target. The root cause was modeling human behavior in the same way you would model weather, which is quite inappropriate but the implementors of the epidemiological model did not have the expertise to know better.

The selection of GPUs is also not appropriate for the nominal objectives of the program. When modeled correctly, i.e. not as weather, these kinds of things aren't the kind of workload GPUs are good at. They tend to look more like very high dimensionality sparse graphs -- latency-hiding is more important than computational throughput. CPUs are actually quite good at this.

This looks more like a program more designed to produce press releases than useful results.

I know it's off topic but I'd love to learn more about how the modeling of COVID went wrong. You can pm me if you want to take the discussion elsewhere.

Seconded!

I would like to read about that too.

The models tended to overshoot the number of deaths by huge amounts. For example, the Imperial College of London estimated 40m deaths in 2020 instead of the 2m that occurred.

https://www.wsws.org/en/articles/2020/03/18/covi-m18.html

https://www.nature.com/articles/d41586-020-01003-6

https://www.imperial.ac.uk/news/196496/coronavirus-pandemic-...

The model authors have since argued that the data was correct, but people responded to the pandemic by changing the way we live. That's OP point: that feedback cycles and corrections exist and they make modeling dynamic systems very difficult.

But they never learn either, the ICL react studies are still getting it spectacularly wrong, 4 weeks ago they claimed cases were rising in the UK, that R was above 1.

Even a cursory glance at the actual data, even the data available at the time, shows they were completely and utterly wrong.

This seems unsurprising, and the correct way to model a situation like this to me.

"Lots of people could die if you keep behaving as you currently are"

"Okay, lets behave differently"

And then less people die.

Trying to frame it as "they modelled it wrong" is nonsense. What even is the point of predictions like this if not to change behaviour - predicting outcomes based on everyone taking precautions and not telling people what might happen if they don't would be dangerous and irresponsible.

You're missing that they modeled all of those scenarios out, "do nothing" was just one of their models.

And even their best case scenarios overshot the mark -- and by a lot.

This isn't to criticize modeling -- it's only to point out how hard it is to get right.

You claimed:

> The models tended to overshoot the number of deaths by huge amounts. For example, the Imperial College of London estimated 40m deaths in 2020 instead of the 2m that occurred.

The very article you cited pointed out that the 40m figure was based on a "left unchecked" scenario. It was not an attempt to predict the actual number of deaths that would occur. Claiming that this is indicative of overshooting because the actual number of deaths is 2m is completely wrong.

> Trying to frame it as "they modelled it wrong" is nonsense.

It's not that. It's that when the system you model responds to the existence of your model, it becomes anti-inductive. It's no longer like weather, but is now like the stock market[0]. Your model suddenly can't predict the system anymore[2], it can at best determine its operating envelope by estimating the degree to which the system can react to the existence of the model.

[0] - I use the term anti-inductive per LessWrong nomenclature[1], but I've also been reading "Sapiens" by Yuval Noah Harari, and there he uses terms "first order chaotic" for systems like weather, and "second order chaotic" for systems like the stock market.

[1] - Introduced in https://www.lesswrong.com/posts/h24JGbmweNpWZfBkM/markets-ar....

[2] - I think it becomes uncomputable in Turing sense, but I'm not smart enough to reduce this to the Halting Problem.

The issue is that the example nostromo gave (40m) was not intended to be predictive of what would actually happen. It was based on a worst case / left unchecked scenario (useful for establishing an upper bound), and therefore irrelevant w/r/t the system responding to the model.

Also many early models were based on SARS and MERS because we had no comparable illnesses, and these were worst case respiratory diseases.

This is also known as the Lucas critique, dating from 1976

https://en.wikipedia.org/wiki/Lucas_critique

and also https://en.wikipedia.org/wiki/Campbell%27s_law

https://en.wikipedia.org/wiki/Goodhart%27s_law

Edit: I think this is not the first time the good people from lesswrong dug up some well known idea and gave it a new name. Good thing, too, giving this important concept more attention. Too often we forget how many people have dealt with the problem of modeling complex systems in the past. And while we can not read everything, it's often a good idea to have at least a glance at where they failed!

Too often I read/review some new "revolutionary" paper based on the idea that hey, we can model this process (involving people) like XYZ from physics, where this stuff works great! Surely, this is better than the plebian approaches in the literature! And then, to the shock of all involved, it doesn't work great....

Also: https://xkcd.com/793/

fun fact comment, Asimov thought about this, I'm quoting the short part that is relevant from this article on wikipedia :https://en.wikipedia.org/wiki/Foundation_series "One key feature of Seldon's theory, which has proved influential in real-world social science,[3] is the uncertainty principle: if a population gains knowledge of its predicted behavior, its self-aware collective actions become unpredictable."

> The model authors have since argued that the data was correct, but people responded to the pandemic by changing the way we live. That's OP point: that feedback cycles and corrections exist and they make modeling dynamic systems very difficult.

Were they actually so naïve that their model did not allow for the possibility that human beings change their behavior in fear of death by pandemic?

Assuming present trends continue, even when things are clearly heading for terrible outcomes, is the basis for most doomsday predictions.

Quanta had a decent write-up of challenges faced by some researchers about a month ago: https://www.quantamagazine.org/the-hard-lessons-of-modeling-...

Of course there are many ways that things went wrong, and not everyone made the same mistakes.

Of course people changed their behavior because of Covid models, that's the fucking point. The modelling wasn't wrong it was just predicting what would happen if people didn't follow any precautions, in order to work what precautions were necessary.

The models are incomplete.

This is the same reason as to why Econophysics, despite all its promise and grandeur, has yet to yield any useful results.

Interdependent human beings are a lot more complex entities than particles or flows.

Our emergent behaviour comes from emergent behaviour of particles, because we are them. It’s just harder to predict a few levels of emergence up.

> This looks more like a program more designed to produce press releases than useful results.

Isn't it interesting how often one reads about science in the press that seems more geared towards generating press releases than useful results?

Snarky, but true point. Also consider that scientific efforts geared towards attracting funding tend to be the ones that get funded.

This hasn’t always been the case so it’s fair to consider what circumstances might foster a better situation, where research can be directed to areas most promising to add progress and value to society at large.

I honestly find it rather interesting how much it is possible for scientific research to gain funding though it be completely bereft of any practical benefit for mankind, or profit margin.

Consider palæontology as an entire field; there is no financial benefit nor practical application to be had for it, yet it seems to find ways to attract funding all the same, most likely because it does have a habit of generating spectacular news articles which sponsors would probably enjoy the publicity of.

But there is truly no practical benefit for mankind to be had in trying to answer to what extent various dinosaur species were endothermic and feathered.

> But there is truly no practical benefit for mankind to be had in trying to answer to what extent various dinosaur species were endothermic and feathered.

Except we’re an insatiably curious species and it sates our curiosity.

It's a good example of how curiosity is sated not by truth, but by anything purporting as such.

The image of dinosaurs that became canonically entrenched in popular culture is almost certainly completely wrong, but the truth is of little consequence, exactly because it is not used for anything that might depend on it's veracity.

It really does not matter whether it be accurate or completely false, for this purpose.

Is 'digital twin' anything other than a buzzword? Is there anything that distinguishes one from a model or a simulation? I had the misfortune of being introduced to the term by an executive who repeated it until is lost all meaning so I still bristle whenever I hear it. I've read the Wikipedia article, but it only increased my skepticism with blustery paragraphs like:

> Healthcare is recognized as an industry being disrupted by the digital twin technology.[45][34] The concept of digital twin in the healthcare industry was originally proposed and first used in product or equipment prognostics.[34] With a digital twin, lives can be improved in terms of medical health, sports and education by taking a more data-driven approach to healthcare.

It is a shame they are using these buzzwords, It's basically an earth system model, one that we had for many many years. They probably will improve the code base, I don't know if they want to build everything from the ground up or use some physics aware hybrid machine learning approach for sub-grid parameterization but really nothing seem novel except trying to improve the resolution of the current models we have.

And building science[0]. But building science has always been full of buzzwords.

[0] https://www.iesve.com/digital-twins

The one distinguishing feature (in theory) of digital twins is it is supposed to be such a hyper accurate model that it can be used to predict absolutely anything about the system in question. No changing of model setup, it's a "perfect" representation.

The down side is everything explodes exponentially - setup time, mesh count, solve time; and we usually get worse results than more focused simulations because we can't squeeze enough detail in across the board.

It generally starts because some manager hears that we've created 8 different specialized models of something due to different areas of interest, and has the bright idea of "lets just create a single super-accurate model we can use for everything". I've been fighting against them my entire career, although 10 years ago it was "virtual mockups"

The next buzzword in the pipeline seems to be "virtual lab" which I can't figure out either. I've been simulating laboratory tests for over a decade and no one can explain to me why that isn't exactly what we're already doing.

None of this is to say that this team isn't doing great work, but somewhere along the way it got wrapped up in some marketing nonsense.

Edit: Restructured my reply to better address OPs question.

Yes it's fine if people want to create a new buzzword for some special case. But people act like it's a revolutionary idea that will allow them to finally address unsolved problems (and ergo deserve funding for).

"Light fields" is one that always annoyed me. People who are apparently unaware of centuries of knowledge and methods in electromagnetism, developing "new" ways to solve problems crudely. That's great if they can make some cool new imaging system, but is it research deserving of long-term high-risk funding? It's just something that anyone skilled in optics can work out if they thought to build it.

Usually Digital Twin = Model = Simulation. Different industries have different words for it. Some call a difference between a Digital Twin / Model and a Simulation where the simulation is the result of some external input being applied to a Digital Twin.

Either way I wouldn't think too much about it. Tech is full of these things. I have been working with AI and neural networks for years before it was called Machine Learning. Now I'm forced to use the term ML to sound relevant even though it is the same thing.

Though it's a buzzword now, the idea behind 'digital twins' was that you not only have a detailed and faithful model (of an item, or process, or system, or network, etc.) whose granularity is congruent with the level of granularity that interests you about the real thing, but you also have bi-directional movement of data between the 'real' thing and its model.

So you can have sensor and measurement data from the real thing be streamed to the model in (ideally) real-time, you can make decisions off of the state of the model, and have those decisions be sent back out into the real world to make a change happen.

The specific wording of digital twins originated from a report discussing innovations in manufacturing, but I find that railway systems and operations make for some of the best examples to explain the concept, because they manage a diverse set of physical assets over which they have partial direct control, and apply conceptual processes on top of them.

Here's three assorted writings [1][2][3] that explain how railways would benefit from this.

[1] https://www.anylogic.com/digital-twin-of-rail-network-for-tr...

[2] https://www.railwayage.com/analytics/how-digital-twins-suppo...

[3] https://www.railwayage.com/analytics/realizing-the-potential...

In my experience complex sites like railways, airports etc. tend to have a lot of nominally digital data already. Things like topographic surveys, engineering drawings, surveys etc. But usually they are more "drawing" than useful data products. Massive directory structures full of random cad files with obtuse layering for small limited contracts, often using a local coordinate system. For a long time there have been efforts to improve data quality under the bim banner, and now perhaps digital twins.

As soon as I saw Gartner mention digital twins as one of the next big things in innovation in one of their reports, I concluded it was a buzzword.

It's not all a buzzword fest.

I work as a Computational Researcher at Stanford Med. My work is quite literally translating 3D scans of the eyes (read MRI) into "digital twins" (read FEA Models).

I think that there is a subtlety in differentiating a digital twin from a model/simulation in intent. Our intent is to quite literally figure out how to use the digital twin specifically, NOT the scan that it is based on, as a way to replace more invasive diagnostics.

Of course, in the process, we figure out more about diagnosing medical problems as a function of just the scans themselves too.

I sympathize. I was buzz-worded half way to brain death by this at a prior job.

Maybe we were in the business of building better business builders, I am still not sure.

It is getting buzzwordy, probably by virtue of being in proximity of the Big Buzzword: "Industry 4.0". But it's a real thing.

I work with digital twins in chemical manufacturing, and there the term is directly coupled with Model Predictive Control. The basic idea is that you build a model of the system (e.g. a chemical plant) you want to control, use that model to optimize controller behavior, apply the results to real controllers in the real system, and then sample the system to reground the model. Rinse, repeat. Such a model is called the "digital twin" of the real system - the idea is that it exists next to the system and is continuously updated to match the real world.

I am familiar with it in the aerospace industry. Digital Twin implies a higher degree of fidelity in terms of importing data from sensors and modeling of physics than just model or simulation might apply, even though it is a model and simulation.

For GE's digital twins in the jet engines, they will build a high fidelity representation of the each individual engine based on as built parts, and then they will simulate every flight based on accelerometers, force sensors, humidity sensors, temperature and pressure sensors which they have placed in the engine. This is different from a general model or simulation which will build a model from CAD and then have a series of expected flight simulations and use that to predict life of the engine.

This is an extremely ambitious project, and hopefully turns out to be worthwhile. For complex/interconnected systems, building ~accurate simulations can vastly decrease the cost of decision-making, and I'm very bullish on digital twin technology, generally. Trying things out in a simulation can be a cheap way to guide more detailed engineering analysis and 'rough in' an approach.

As an example, my startup, Bractlet (bractlet.com), uses detailed, physics-based energy simulation (aka "digital energy twin") technology as a tool to optimize HVAC design and controls in large commercial buildings. I'm sure efficacy varies widely by domain, but it's worked extremely well for us so far; we typically help our customers save about 20-30% on their energy expenditures annually, and the digital twin is the bedrock of our approach.

I also like the digital twin concept as it applies to sci-fi. Sometimes I daydream our universe is a digital twin built by an advanced civilization meant to simulate the universe's history looking for clues as to what lead to a calamity that they're trying to undo. Random thought :)

Can/does your system simulate the quality of the air as well? In particular particulate matter and CO2 levels.

We've given this a lot of attention in the age of Covid, as you might imagine. Many modern HVAC control systems include CO₂ sensors in the return air streams to make sure they're bringing in enough outdoor air to meet indoor air quality standards, and that's fairly easy to simulate. Particulate counts are trickier, and may be better estimated using out-of-simulation engineering calculations based on the type of filtration in place.

Interesting, mind if I shoot you an email? My family has a business in this space (residential side) and I'd love to chat more.

Sure! brian at bractlet dot com

Fascinating. Would love to see a "digital energy twin" product for residential properties.

Would love to hear what you're thinking! We've prioritized commercial office as it's great bang-for-buck for our business, but we'd love to expand to other verticals in the future.

Essentially, provide a way to build a digital "twin" version of my house, and get advice about improving heating/cooling efficiency.

Interesting, I have a couple of questions:

- How do model random variables with high variance like occupancy or equipment use? Are you using sensor/monitoring equipment or just trying to model it as best as you can?

- Did you guys use the initial energy model (built by by the design/consultant team) or do you guys build your model from scratch?

- What energy simulation engine do you guys use? I'm just curious if there's any big advantage between the engines for digital twin applications.

- We use a ton of empirical time series data from the building to calibrate the model.

- There's almost never a model. We build it from scratch in every case.

- We have a highly customized fork of EnergyPlus. They are all difficult to use in our experience, but EnergyPlus allows us to get the detail required for our needs.

>"Observational data will be continuously incorporated into the digital twin in order to make the digital Earth model more accurate for monitoring the evolution and predict possible future trajectories. But in addition to the observation data conventionally used for weather and climate simulations, the researchers also want to integrate new data on relevant human activities into the model. The new "Earth system model" will represent virtually all processes on the Earth's surface as realistically as possible, including the influence of humans on water, food and energy management, and the processes in the physical Earth system."

I think I've seen something like it on the latest season of Westworld. Jokes aside this reminds me a little bit of the brain project. I'm really not sure if attempting to build full fidelity models of hugely complex systems is the way forward to understand this systems. It seems increasingly that scientists are trying to replace theoretical models of the say, the mind or the city with purely data driven approaches that don't necessarily produce any insight, or even accurate forecasts. Sometimes it feels like with increasing computing power we've gone back to the problems of empiricism of the mid 20th century.

You couldn't call it full-fidelity.

Last I saw anything of it, the EU brain Flagship project had lots of actual brain scientist unhappy about having almost no scientific or professional benefit from the huge public spending, and one happy Henry Markram for having the final say that huge chunks of the money should go into his guys building a really, really big computer.

I foresee a Big Science project with little to no benefit for meteorology, climatology, geology, etc and a lot of public money going into building a really, REALLY big computer.

Which isn't necessarily a bad thing to buy per se, but if the intent really was to understand Earth, they'd be better off putting the money into fundamental research in the Earth sciences.

> ...It seems increasingly that scientists are trying to replace theoretical models of the say, the mind or the city with purely data driven approaches that don't necessarily produce any insight, or even accurate forecasts.

It may seem like a logical desire to integrate the existing massive data feeds about various aspects of the planet into a progression of coherent states of it. With the hope to gain insights for the dynamics and trends.

The first challenge is to figure out if such feeds are indeed integratable and at all could lead to any sort of coherency.

The next challenge is to understand what should be considered a "state". It is a complex system, yet one needs to choose the parameters.

It is nice to have such a global view of the observations. But I'm sure the goal stretches further and means to produce projections and forecasts. This may have some political impact, but practically may just be as good as a speculation, given the vast scale.

You know the fjords in Norway? I got a prize for creating those, you know.

here's a towel

My student has started training GANs based on satellite imagery to model changing landscapes.

I think they are mesmerizing to look at. https://youtube.com/shorts/sdlx8yxdLPo

From the press release I had the same reaction as many commenters here: this seems like a terrible idea because any model that tries to be better at everything inevitably winds up being better at nothing.

But I want to be charitable and so I wonder if there really is a game-changing idea in here that the press release does a poor job communicating.

Could this wind up physically measuring a ton of stuff that hasn't been measured at a decent resolution before, and so produce genuine meaningful improvements?

Or is there really some kind of new viable "supercomputer" architecture unique to climate modeling that will pay off massive dividends?

The AI part worries me the most, since it can be a notorious black box where critical biases and errors get amplified without even being detectable. But are there actually techniques here to drastically speed up the production of expensive calculations, that are cheap to verify as correct?

Or is there genuinely a divide between earth scientists and computer scientists where neither side is benefiting from advances in the other, and there's a huge genuine opportunity here for a massively productive paradigm shift?

I'd really like to hope there's something of value here, and perhaps someone here who has worked with climate modeling and knows actual details about this project has some insight.

The carbon generated by running this simulation will be offset by planting trees in the simulation.

they better plant some trees in real life too or we'll diverge from canon

To think, there was a time before we had a permanent, high-res historical record of the entire planet that we could wind forwards and backwards and observe history through.

“Digital twin” is such a misleading term, as it implies the ability to perfectly model a non-digital system, which is inescapably impossible.

The best we will ever be able to create is approximate models that have varying tradeoffs. The term "digital twin" obscures the fact that there are tradeoffs involved in the first place. It also causes harm to decision-makers, who ought to be able to choose which tradeoffs to make.

I would hope (and where I work, it seems to be the case) that decision makers understand that "digital twins" are only twins with respect to a select set of parameters that are being optimized by use of such model. But then, buzzwords do have a life of their own.

I am reminded of this short story by Neil Gaiman https://kartoweb.itc.nl/kobben/files/The%20Mapmaker-Neil%20G...

An interesting story. But the nerd in me screams about an unexplored obvious plot point: such a map of the land would have to contain in it the map of the map, creating infinite recursion, which is especially unresolvable in the last map the Emperor desired.

You are thinking of the story as it would have been written by Neal Stephenson ;-)

Eh... I'm skeptical this will work out very usefully. The proof is in the pudding though. But sure, I can see why someone would fund the research project. But this is mostly just a press release with buzzwords, right?

Reminds me of the biosphere projects whose failures have been the subject of multiple debates over reasons why.

https://www.euronews.com/living/2020/07/09/living-in-a-bubbl...

Highly sceptical the digital version of this will be playing with a full deck of data resulting in skewed results. still think it's a good idea but definitely not the accursed 'settled science', more an experiment.

I'd imagine that like weather predicting, the best way to simulate a complex system is to slightly vary the starting conditions and sample the results. Obviously it's not going to be perfect, but it gives a clear idea about what things to worry about.

Reading the article, it seems that the purpose of the twin isn't simulation, it's mapping. That data can then be used as the input to simulation of your choice.

Wonder if there is a way to wrap it in Dwarf Fortress or something and crowd source the intractable parts.

Does anyone know any evidence for or against the feasibility of this concept? Policy makers in the uk are talking about investing in digital twins for the purpose of AI research and it just seems to be buzzword salad. Can someone point me to a reference?

There is sound mathematical evidence that certain systems are impossible to model this way: https://en.wikipedia.org/wiki/Chaos_theory.

"Small differences in initial conditions, such as those due to errors in measurements or due to rounding errors in numerical computation, can yield widely diverging outcomes for such dynamical systems, rendering long-term prediction of their behavior impossible in general. This can happen even though these systems are deterministic, meaning that their future behavior follows a unique evolution and is fully determined by their initial conditions, with no random elements involved. In other words, the deterministic nature of these systems does not make them predictable. This behavior is known as deterministic chaos, or simply chaos. The theory was summarized by Edward Lorenz as: Chaos: When the present determines the future, but the approximate present does not approximately determine the future."

Chaos theory absolutely, unequivocally prevents such models from making detailed long-term predictions. But that doesn't mean the models are worthless; they can still make detailed short-term predictions and gross long-term predictions.

Ed Lorenz was a weather modeler; he invented chaos theory to explain the long-term failure of his models. But weather models and climate models are nevertheless useful today if we're aware of their limitations.

Just expand the map in GTA V to cover all countries, should be accurate enough.

If you need clarification, isn't this similar to Deep Thought?

This will be a failure of epic proportions.

Due to the complexity of the real world we can't even predict weather for 2 days in advance and you they think they can predict the world...

The sad part is that they will then use this model and its result to create policies not knowing that you can never rely on such a "copy" because even the smallest difference will create an insane amount of divergence even 1 day down the line of future simulation let alone years...

dibs on biyomon

Reply on news.ycombinator.com