Hacker News Reader

Comment by Kalium | Hacker News Reader

Perhaps there might be a higher level question at hand here. What level of caution is appropriate when the deliberate choice of inaction has a price tag of six digits of lives attached? What level of risk and potentially unforeseen consequences are we willing to accept?

This is not an idle question. It's a critical question for informing a decision-making framework that rests on calculation. Which we need here - I think we can all agree that personal ick-factor is a bad approach to large-scale public policy.

Which is to say you're right. We cannot be certain that won't happen with this approach. Indeed, we cannot be certain of all consequences with any approach.

With that in mind, what level of certainty should we require?

View on HN

It sounds like you want to take a calculated risk. I'm not convinced that this is a calculated risk -- it's an unbounded risk. What if it, for example, kills of pretty much all of the insects over a large part of the earth?

I'm saying that taking a calculated risk requires a framework for calculation. Which must of course include the ability to handle for unbounded risks.

These types of examples aren’t useful though because you can make up any number of them, and make them as extreme as you want.

Risk management, on the other hand, is about determining risks that are reasonably likely, and figuring out contingencies and mitigation strategies.

In your opinion, is it reasonably likely that a fungi that is engineered to kill a specific species of insect (and only those carrying a specific disease) can somehow go out of control and end up killing all insects on the planet? In my opinion, it is not.

I think there are a number of people waking up to the level of concern that you need to put on unbounded risks. See the popular book, The Black Swan. To throw out these large but improbable risks is taking a large part out of these distributions. Maybe it's not a good idea.

On the other hand, we cannot afford to be paralyzed by concerns about black swans. If we did, nothing would get done.

I don't think the book and others with similar thoughts suggest anything like being paralyzed with fear, so we should be OK here.

I disagree. The often unstated sentiment under such thoughts is always "hmm, this sounds way too crazy, we probably shouldn't do it because it feels too much like 'Playing God'!".

It's a form of concern-trolling: deep down inside you are against something, but don't want to say that openly, so instead you pretend to side with the proponents, but choose to play the role of "concerned citizen" and invent scary scenarios at the extreme ends of the probability spectrum, with the suggestion that "we should probably research this thing more before we try it!"

That's not the underlying reasoning and I have heard very clear arguments. The argument goes something like 'the extremely impactful event has never occurred in the past so our risk assessment is very vulnerable to large magnitude errors'.

What is encouraged is taking actions that have well understood OR bounded risk.

By bringing up "concern-trolling", I am concerned you have given up the fight. By avoiding the direct argument and retreating to this supposed hidden argument, which is much easier to counter, there is not anything left to discuss. Tell me if I am misunderstanding.

Like the number of nuclear meltdowns in a thousand years?

This is a situation where the potential risk that comes with the new invention of man is catastrophic. It's on the same level of Hamming having to, while onthe Manhatten Project, calculate the odds of igniting the atmosphere. There is existential risk to our species and possibly all animal life. So, yeah, we can spend a few years trying to learn about the mutability of this bug.

I fully agree.

I'm suggesting that some sort of risk evaluation framework more sophisticated than "Gee, I can imagine a scenario where this kills us all!" might be in order.

Another criticism of this haste is that you set yourself up to be taken advantage of by profiteers or genuine malicious actors. I understand the urgency to save lives as soon as possible; but the risk is high and uncertain.

You're right: the risks are both high and uncertain.

With that in mind, it's perhaps worth considering the potential benefits of having a uniform framework for evaluating risks that are attendant upon any given possible course of action. Risk evaluation neither starts nor stops with personal ick-factor or the worst scenarios imaginable. Risk evaluations also need to include the probability of success, expected costs, the estimated probability of any negative outcomes or side-effects, and so on. It's a non-trivial endeavor.

Again, you're right. The risks of any possible course of action, particularly poorly understood ones, are very real. They demand meticulous and systematic evaluation.

For something like this? An extreme amount. There are other options (a vaccine is already being tested) that can't mutate into a pathogen that kills off more people than the thing it was supposed to cure. Let's finish vetting those out completely before we move to what can only be described as the unstable nuclear option.

To be clearer, the question of what level of caution is appropriate is one that cannot depend on the approach being considered. It needs to to be one that can be applied evenly across the whole of the solution-space.

Approaches like insisting on fully and completely explore all the ideas that are readily palateable before we try to understand the potentials and risks of others seems like making policy primarily based around personal ick-factor. Which is great when choosing what to have for dinner, but maybe somewhat less than ideal when every delay means thousands more dead people.

Also, vaccines can run horribly off the rails. One cannot mutate into a pathogen, as you say, but tainted vaccines are a known risk. There may be unknown long-term risks or other things that haven't been fully studied. No approach is risk-free in this arena, and we do a disservice to humanity when we pretend otherwise.

>but maybe somewhat less than ideal when every delay means thousands more dead people.

You literally just defeated your own logic. The reason we do medical trials and "delay" is to ensure the vaccines aren't "tainted" as you put it, and to ensure that there are no delayed side effects.

But... you want to rush an alternative option that could kill literally millions, because waiting could mean thousands die??

I can see I have been unclear. Please accept my apologies for my error.

I do not think it reasonable to rush through any thing, option, or approach in this arena. I do not suggest or advise any actions of the sort in any way, shape, or form. I do think it's worth taking the time to ensure that risks are handled properly. I think it's worth having a uniform framework for identifying what those risks are and reasoning about them to enable planning. I also think we should consider carefully the potential benefits of exploring multiple possibilties, against the chance that the most pleasing and lowest risk ones do not pan out.

Again, please accept my apoligies for being unclear. I hope I've made myself clearer! Please don't hesitate to ask if there's anything else I can communicate better.

> Also, vaccines can run horribly off the rails. One cannot mutate into a pathogen, as you say

The live polio vaccine absolutely can. Before polio was largely eliminated, it didn't really matter, but once the risk of getting the wild-type virus dropped below the risk of getting it from the vaccine, it stopped making sense to use the live-attenuated vaccine.

http://www.virology.ws/2015/09/10/why-do-we-still-use-sabin-...

This makes me think that it's good persuasion to acknowledge that all anti-vaxxers aren't necessarily fundamentally irrational (some are I'm sure). Most people are anti-vaccine with respect to smallpox for example, as well as the polio example you give. We continue to use the vaccines we do because they improve the overall odds, not because they are a panacea with no downsides.

The modern inactivated ("killed", if you consider a virus alive) polio vaccine is nearly perfectly safe though (other than one-in-a-million allergic reactions), and can't give you polio. Nobody's used the live vaccine in the US since 2000.

People don't generally get vaccinated against smallpox anymore because the pathogen has been eradicated.

Reply on news.ycombinator.com