New Google SRE book: Building Secure and Reliable Systems

I was trying to avoid being over-pedantic. By safely, I mean mitigating the risk to an appropriate level. So, to reword the statement. I mean, "If the risk can not be mitigated to an acceptable level, then we should not do that thing." The appropriate level of risk depends on the action in question. If it involves human lives, then society should evaluate the acceptable amount of risk. If it is bank accounts, then it is up to the bank, customers, society, on the appropriate amount of risk to take on.

My response is a IF X, THEN Y, so should be read: "IF you do not believe anybody can reduce the risk of software in systems that can kill people to a societally acceptable level, THEN we should not use software for those systems." The statement takes a belief as an input and outputs what I believe is a logical output. I frankly find it hard to disagree with the inference since it is nearly tautological. Like, "I do not believe anybody can reduce the risk of software to a societally acceptable level, but we should build such systems anyways." seems like something almost anybody would disagree with. So the primary concern is if you believe the antecedent. Personally, I believe software that reduces the risk appropriately can be made, so I believe we can make software to manage these systems which is contrary to what you think I believe.

My bug bounty point is actually that a long-standing large bounty provides reasonably strong evidence that the difficulty of compromise is on the order of the bounty. Consider, if somebody offered a $500M bug bounty that is easy to collect and nobody collects it for 10 years, I think this provides strong evidence that it is really hard to compromise, possibly on the order of $500M hard to compromise (the main problems at these scales is that $500M is actually a lot of capital, so there is significant risk involved in actually getting into the general vicinity of investment, so you might actually be limited by capital instead of upside). This is consistent with my original point which was providing an example of a concrete indicator of expertise that I, and likely others, should agree is convincing.

What I meant by $500M is too small is that $500M is a small number relative to the expected difficulty of compromise someone might expect from these institutions. For instance, a large bank can have 100M customers. So, if a total breach could get all of their information for a cost of $500M (which I actually believe if way, way too high), it would be "economical" for a nefarious actor if the per-customer data was only worth $5 (obviously they would want profit, so more, etc.). I don't think a big bank would advertise that in their commercials, "Your data is safe with us, unless they have a $5.", let alone the massively lower number that I actually believe it is. Obviously I do not mean that any specific person can be targeted for $5, just the bulk value rate of a large hack for it to be economical.

Putting a $ value on things is reductive, but also quantitative which I find very useful. Using your joking example, if the weakest link is paying every member of the security team $100M and they probably have thousands on their security them, then that is ~$100B which I will accept as being excellent security in their problem domain. However, by putting a number on it, it makes it easier to see if that number sounds like what we want. Since you said it in jest, I will assume you think that is a ludicrous number. However, if you apply that level of security to a different problem domain, such switching all US cars to self-driving cars (not that I am suggesting Google is suggesting such a thing at this time), I think that is far too little. If the entire US used self-driving cars and you could get a total system compromise, you could drive millions of cars into oncoming traffic within a minute killing millions. With such a risk, the number that you think is impossibly high is not enough to mitigate the risk acceptably in my mind. So logically, the actual state of security, which you believe to be less than your impossibly high number, is not enough and indicates that we are very far from achieving the necessary level in my mind. Obviously, you can disagree with this statement by determining that the problem is smaller than I am saying, the chance is low, the socially acceptable risk profile is different (we accept small chance of catastrophic failure vs. the status quo of frequent small failures), etc. this is just an example.

I have no concrete information with respect to statements 1 and 2 you mention. As a result, I do not conclude what you conclude. Why do you believe statements 1 and 2? In fact, both seem inherently hard to demonstrate convincingly since both are related to knowledge of an unknown variable.

For statement 1, how do you evaluate the transparency of a system when you are not privy to the underlying rate? A possible strategy is a sting-like operation where some entity, unbeknownst to them but authorized, compromise their systems and evaluate if they are transparent about it. You might need to run such an operation multiple times to get a statistical amount of information. Has such an operation been done? Another alternative is a trusted third-party watchdog which is privy to the underlying information. Are there such entities with respect to Google internal security? Another one I would find highly credible is a publicly binding statement indicating that they disclose all compromises in a timely manner and being subject to material damages, lets say 4% of global revenue like GDPR, if they fail to comply. I am sure there are other strategies that would be convincing, but these are some I could come up with off the top of my head.

For statement 2, how can you tell they have a better-than-average record when even they are not privy to the underlying rate? It is pretty much like trying to prove a negative which is notoriously hard. My primary idea would be to subject them to various techniques that have worked on other entities in the past and see if they defeat or at least detect all of them. Given that nation-state actors such as the CIA can allocate billions of dollars to such attacks, does Google have a billion dollar red team to simulate such attacks? Do they routinely fail? Another strategy is proving that their systems do get compromised. Vulnerability brokers routinely have zero day vulnerabilities for Google Chrome and Android. This demonstrates that systems developed by Google have compromises that Google is not aware of. Those are high profile projects, so it seems reasonable that they would be representative of Google security as a whole, so if we extrapolate that to other systems then those systems likely also have vulnerabilities known to someone, but not known to Google which can be used to compromise their systems. State-level actors are some of the main clients of vulnerability brokers, so them being able to compromise systems, but not be detected is a highly likely outcome in my mind. Is there some reason that Google Chrome or Android are not representative of the security used by Google on their other systems? If so, why, since Chrome and Android seem kind of important in my mind. Do you have other ideas for how to know if they are detecting all state-level actors?

I largely agree with your first point that I will probably not get answers that are materially different than PR speak. I'm not really sure how that is related to the questions themselves being interesting or not, anybody can give a boring answer to an interesting question. My goal is questions that, if answered honestly, would elicit useful/filtering responses. I can see an interpretation where the questions are boring because nobody will give them an honest response, but that is half-orthogonal to the question itself.

So if Google doesn't care what the OEMs do with Android, it definitely shows that Google doesn't care about security on Android as a whole, as long as it can write blogs about how perfect the security in Pixel devices looks like, which by the way are on sale just in a couple of selected tier 1 countries.

That isn't caring about security, what Apple does, it is caring about it.