Doesn't this make it kinda pointless? If the target has a say in how they should perform their audit/attack, how does that not produce results biased to the targets favor? Wouldn't the most unbiased way to do such a thing would be for the target to have zero idea what the auditor would be doing?
> which opens up room for misinterpretation of the findings
If Mullvad dictated how to do things or imposed limits on the reach of the testing, the results are worthless anyway
That's the whole point of a threat model: Mullvad has a threat model, and they build a product resistant to that. When someone audits the product, they should audit it against the threat model.
To do an audit you have to audit against some sort of pre-established criteria. That is how audits work. In security, that will typically be a standard (or set of standards) alongside a threat model. In finances, you audit against what is legal in the areas you operate.
>[...] zero idea what the auditor would be doing?
That's a practical impossibility. From the client side you want to be able to evaluate quotes, stay within a budget, etc. You don't want to pay good money (audits are really expensive!) for areas that you are works-in-progress, or non-applicable threat models (e.g. lots of security software explicitly does not protect against nation-state actors, so they don't do audits from the perspective of a nation-state actor).
From the auditor side, you want to know what staff to assign (according to their expertise), how to schedule your staff, etc.
>If Mullvad dictated how to do things or imposed limits on the reach of the testing, the results are worthless anyway
Not at all. The company says "This is the set of standards we are auditing against and our threat model. This is how we performed". The results are useful for everything covered by those standards and threat model. By explicitly stating the threat model, you as a consumer can compare your threat model to the one that was audited and make an informed decision.
For example, this X41's threat model only supposes that an attacker could execute code on the system as a different, unprivileged user. They don't consider the situation where an attacker might have an administrative account on the system.
For my personal devices today, this matches my threat model. If an attacker has an administrative account on my machine, I assume that my VPN isn't going to be able to protect my traffic from them. There's no need to worry about laying out all the ways this could impact Mullvad's client.
It's mostly just a matter of having a defined scope. They could of course say "You can only attack this one exact thing" that makes them look good, but this is true of many things.
Defining the threat model is standard in the infosec auditing/pentest world, FWIW.
> If Mullvad dictated how to do things or imposed limits on the reach of the testing, the results are worthless anyway
That's only true if your threat model is "literally every possible thing that could ever happen", which is so broad to be meaningless and impossible to test anyway.
Computer programmers also do not typically design their programs under the assumption that someone stuffed newspaper between their CPU and heatsink and it caught on fire. They work on the assumption the computer is not on fire.
If I ask a person to do a audit I will tell them what the scope of their audit is, e.g. check the physical security measures of our server rooms. Otherwise they would have to take literally everything into consideration (what if the accountant is a malicous actor, what if the server rooms are attacked by a military, what if our hardware is swapped out during delivery, what if..) and they would never be able to stop.
If you take security seriously you try to defend against likely attack scenarios first. Your way to control that is by choosing the scope of audit.
So, first you have to determine what the target says.
Then you look around to see if that seems accurate.
Then you look around to see what are the systems and controls that are in place to keep things in a controlled state in the future.
> We believe the reason these vulnerabilities exist is because gocryptfs doesn’t have a clearly spelled-out threat model. Some of the attacks seem hard to avoid given gocryptfs’s performance goals and may have been introduced “by design” to meet these goals. We suggest writing down an explicit threat model and updating the website to better communicate the security guarantees that gocryptfs provides. This way, users are less likely to rely on it in ways which would make them vulnerable.
Later established: https://nuetzlich.net/gocryptfs/threat_model/