Bitwarden second security audit report
bitwarden.com
bitwarden.com
Last page of the PDF indicates that they just did an external VA and pentest, but looking at their product set , I'd have expected (at least) a review of the web, desktop and mobile apps and the browser plugins for it to be a "thorough security assessment and penetration test" (as quoted in the blog).
Not to say external reviews have no value, but they're only part of what's needed.
So I wonder if they just forgot to mention that this second audit report doesn't cover that, or if there are more reports coming.
When I was a pentester, I once ran the numbers and concluded that each pen test must have cost some absurd amount of money for us to be profitable. And they do, because it’s effective. But I wanted to point out a likely possibility: they wanted to do what you were saying, and concluded a million dollars spent on a pentest was beyond reach.
a review of the web, desktop and mobile apps and the browser plugins
If a million dollars sounds like an overestimate, you’re right to be skeptical. But $270k seems entirely reasonable; that’s $30k per app, for 9 apps.
So it might be tempting to feel like “it’s just a browser plugin though. How can the cost be anywhere comparable?”
Because pentests are billed in days, and a day on a plugin is a day on an app.
My point was, that there was a gap between how the blog appeared to be billing the test "thorough security assessment and penetration test" and the report's statements around scope.
Obviously companies can't always afford all the testing that they need to get as much coverage as they could, but when your major selling product is a downloadable application, a comprehensive review would usually at least touch on it as part of the work performed, for it to be called thorough.
What you want or need will fit somewhere in between.
You can get it cheaper but a lot of it - for better or for worse - really comes down to the skill of the individual consultant. You can pretty much halve these prices, but then you'd end up getting stuff outsourced to India and it wouldn't be any good. Depends if you care about the security of your product or just want a box ticked for some arbitrary compliance and want it done as cheap as possible.
I haven't been a tester/consultant for a few years now, but the prices hold up. That being said one development which has happened since I've left the industry is the advent of crowd sourced pentesting. I know a lot of friends who moonlight with these things and are very good at their jobs, and the rates are lower. The name crowd strike comes to mind, but I'm not 100% sure if that was the company or not. I know a lot of good UK based companies (if it's a web app/remote then the physical testers location doesn't matter) if you needed.
I think it's a bit sneaky as for a product like this, people expect this to be a code and crypto audit. The "network" part should be emphasised and in the title of the page, instead of just the PDF.
* expensive commercial vulnerability scanning tools.
Accountability and consistency is a real concern in crowdsourcing. There is a reason we dont spend too much time designing an idea and then crowdsourcing all development. Why would security be different?
On average I'd even say that my day rate went down compared with 10 odd years ago - when you needed an interview at GCHQ to get CHECK - as there were just few people doing it whereas there are loads now.
The discussion of relative merit of bug bounty versus a pentest is well trod ground, so I won't rehash here except to say I would never consider a bug bounty replacement for a pentest, and if you're asked for a pentest report as part of third-party vetting etc. many organizations will be concerned to see a bug bounty program compiled report.
The latter example sounds like https://cobalt.io/. I've seen several reports and all I can say is if I were vetting a third-party or otherwise looking for assurance of security posture I would still want to see a "real" pentest from a reputable firm.
I dont want to name companies and start a war, but the industry is moving in a dangerous direction with some of the other options -- there are companies offering pen testing where those companies have no full time employees. They post the scope, and their registered users can sign in, take the work, and deliver it. Quality is all over the place. And things like confidentiality, data processing, etc, and any way to confirm a corporate entity adheres to their contractual obligations? Nonexistent.
Boutique Firm X billed at 285/hr with an average of 60 hours for a small application. That comes out to $2,280 USD a day.
Standard Small Consulting Firm Y billed at 250/hr. In the past 6 years I have yet to see anything below 235/hr, which is still $1880 USD/day (1479, GBP).
Hope that helps, GP.
More bespoke services like proper red teaming, DDoS simulation IOT/connected cars/hardware were about double that.
£800 a day is the very bottom of the price scale in the UK for general SME public sector companies.
£800/day is low, but not unheard of especially if you use freelancers/small boutiques, but £2k/days is more than I've seen for most things in the UK.
We didn’t really deal with retail banks the banking clients would be investment and asset management banks like RaboBank.
When I was working in financial firms, there were internal red teams running vulnerability scanners or manual pentest (manual requires much more planning and coordination) . No point in paying external firms £10k per app to run an automated test. I am gonna have to consider changing side if audit firms are really billing £2-4k a day for this.
You can get something like that for £5-10k if you go through one of the typical audit firms (KPMG, Deloitte and co).
In addition you can look into some ISO certifications or industry specific regulations. It's basically a checklist of a thousand questions: do you use TLS? are your applications protected by authentication? can custom folks access personal data of customers? are there audit logs of support accessing customer information?
$8k - $12k for a "platform" that hires 100% outsourced pentesters of highly variable skill and quality, and takes no liability themselves, and whose pentesters are located in developing countries and good luck getting damages out of them, ever, regardless of their platform "reputation" pretend points. also communication tends to be difficult as the norm is ESL.
$25k for a US-based boutique firm with in-house pentesters of vetted high quality, who accept liability and against whom you can actually expect to enforce an NDA and/or extract other damages. source code and design audit starts at about this price.
then, you can run your own program on top of the bounty platforms, however scope and focused work is not going to happen (a lot of the work is boring), and it will cost you a lot of your own time. the money you save DIY is not worth your own time investment.
if you actually want a good pentest, go with a boutique, quality firm. if you want something to give to an auditor or to meet a VSA, go with cheap.
I am astounded to see this missing from the report. Apparently the report was just their external API configuration or something?
1. External vulnerability assessment of the Bitwarden computing systems and web applications
2.External penetration testing of the Bitwarden computing systems and web applications
Was this discussed with the prior audit?
EDIT to add: Here's the 2018 "cryptographically right answer" on password hashing ( https://latacora.singles/2018/04/03/cryptographic-right-answ... ):
Password handling
Percival, 2009: scrypt or PBKDF2.
Ptacek, 2015: In order of preference, use scrypt, bcrypt, and then if nothing else is available PBKDF2.
Latacora, 2018: In order of preference, use scrypt, argon2, bcrypt, and then if nothing else is available PBKDF2.
You care about this if: you accept passwords from users or, anywhere in your system, have human-intelligible secret keys.
But, seriously: you can throw a dart at a wall to pick one of these. Technically, argon2 and scrypt are materially better than bcrypt, which is much better than PBKDF2. In practice, it mostly matters that you use a real secure password hash, and not as much which one you use.
Don’t build elaborate password-hash-agility schemes.
Avoid: SHA-3, naked SHA-2, SHA-1, MD5.
EDIT to UPDATE:
Bitwarden has commented (about an hour ago) that they'll fix this! Cool.
They found something worth fixing, despite the project being open source wit bug bounties and previous audits being made.
Being open source just increases the chance of a problem being spotted if there are sufficiently clue-up people looking. Being open does not at all guarantee that any given problem will be spotted during the normal course of work. Security issues can be dues to combinations of flaws in widely spaced code so even if working directly on one part you might not realise there is an issue in conjunction with another part. That is why it is necessary to have tests/audits like this, for oth open and closed source systems, where someone is task specifically to look for security problems.
It isn't right to criticise Bitwarden for being tested and issues being found (unless those issues are systemic and/or just plain stupid, or you believe the project's response to resolve them is too slow or incomplete). Instead concern should be aimed at security related products that are not regularly subject to external audit at all. Not having any issues because you have not checked for them is a much greater worry!
That, despite the software being open-source and therefore more likely to have bugs spotted, and despite having a bug bounty program, the auditing company found a moderate, therefore they must be thorough.
I wonder if that's the case here. I don't work in that space but the issues they found seem like they might be low hanging fruit. I've pasted them below for anyone that's curious.
> The Cross Origin Resource Sharing (CORS) configuration on Bitwarden server APIs allows for any clientorigin to access its endpoints.
> The Content Security Policy (CSP) configuration on the Bitwarden web vault application allows for'unsafe-inline' CSS styles to execute.
If they were appropriately thorough and all they found were low-hanging fruit, then that is a good thing.
Of course a detailed report is no absolute guarantee: we once had a test done that I think was more than shoddy: there was not nearly enough activity on the web server over the testing period for the amount of automated work they claimed to have done, and I spotted an issue a couple of weeks later that at least one of their documented processes really should have picked up on. That company is no longer in business thankfully.
Such companies sometimes offer a range of penetration testing options from relatively superficial to aggressive, in-depth, and detailed, so you'd need to read the report (I will when I have more time as we are considering Bitwarden for our credential management) to see if what it is saying is sufficiently reassuring.
That happens.
I can't comment on Insight Risk Consulting, as I don't know that company. They write they had a previous audit from Cure53. That's a well known and very skilled security company and I would expect that you can't buy an "please ignore as many vulns as possible" report from them.
The thing is half worthless, verifying that the CDN has TLS and raising warnings about obscure HTTP/CORS headers.
But occasionally it can find some really bad misconfiguration or library with a critical vulnerability in dire need of an upgrade. (Of course they would never publish a report finding issues like that).
I know that encryption primitives are almost never the breaking point in systems like this, but I wonder in situations like this where breaches would allow adversaries to attempt offline attacks whether they are particularly pertinent.
Specifically, while the number of iterations on the PBKDF2 SHA-256 function are high (100,001 on the client), PBKDF2 always felt to me like a footgun when compared to scrypt or argon2 which don't have as many (any?) insecure modes of operation.
The website states that AES is used, but is it in an authenticated mode (e.g. GCM ?)
Finally, their website states that they use "popular and reputable crypto libraries" and that they don't roll their own crypto, but the libraries they use are awfully low-level. Something like libsodium or FilSottile's age would be something I'd be more comfortable with when considering a hosted method.
In the meantime, I think I'll keep using KeePass2 (w/ Argon2 and ChaCha20) and synch'd with SyncThing to minimise my attack surface.
Actually it looks like Argon2 is being discussed as of just a couple hours ago https://community.bitwarden.com/t/switch-to-argon2/350/24
Quick googling on this topic says that as of 2018 they were using authenticated AES-CBC.
Stretching isn't magic. If your password is 'jszymborski' then no practical KDF will prevent bad guys just guessing "Um, maybe it's just jszymborski?" and getting in. And on the other hand if it's two dozen random alphanumerics you can use SHA256() as your KDF and be absolutely fine.
Because their users will (even if told emphatically not to) use bad passwords, Bitwarden needs a PBKDF with stretching here to buy those people more margin, but nit-picking the choice of PBKDF is missing the wood for the trees. As an end user the right thing to do regardless is use good passwords, which of course is how we got here...
However, they need a cross platform solution that integrates with .NET and will also work on a budget smartphone. I've sketched out such an architecture, but I lack the time and budget to do it.
For example, if you use a third-party KeePass app on your phone, besides having to figure out a secure way to sync it, you also now have to trust the developer of the phone app as well. Larger attack surface.
I'd definitely introduce and use Bitwarden for teams.
I also found this suited my devices and usage, Linux, Android, Mac, Windows... happy across the board.
Also... employers tend not to use Bitwarden, they pick 1Password or LastPass, so it means I can have both work and personal on my BYODs.
But isn't this what the backup codes are for?
I use the notes for each entry in Bitwarden to indicate what kind of 2FA I have enabled and whether I have a backup code already stored in the other vault.
I also convinced my employer company to use it.
I moved from 1Password to BitWarden... 2 years ago now? (2 years 2 months) Oh the experience was SO much nicer than 1Password. And the iOS app WORKED!
I've considered using Pass or other open-source self-hosted/synced alternatives but I don't really want to fiddle with something like this quite yet because Bitwarden meets my needs perfectly.
After a few months, I watched back to LastPass. Bitwarden never quite worked right and as far as I know doesn't provide a way to review access history (I was hacked and wanted to see if other IP addresses accessed Bitwarden).
One fantastic feature is that you can add the second factor 6 digit generator to a given password, just like an authentication app. When you log in by filling the username/password and hitting enter, your second factor is copied to the clipboard. That lets you just paste it in, which is very convenient for those annoying sites that make you log in with 2FA every 30 to 60 minutes.
I'm considering switching to 1Password or Bitwarden. But I'm not sure about BitWarden using the same password both for encrypting the vault and accessing Bitwarden server. Chrome for example has an encryption password which is different from your usual Google Account password.
It all depends on the risk you’re trying to mitigate. A MITTM or a server attack won’t be able to gain access to your passwords, even if they intercept the data. A user with knowledge of your password or a key logged on your client could. However in either of those cases, you’re not protected all that much by having two passwords as opposed to one long one.
The nice thing about Bitwarden_rs is that you get features which you would have to pay for with normal Bitwarden. For example 2FA with U2F. As a note Bitwarden_rs is written in Rust.
I also use it at my company, and personally with my wife. Also got my mum to use it!
At my company, we also use it for server secrets, using envwarden: a simple wrapper we created and open-sourced[0] for managing server secrets with Bitwarden.
I'd love to hear an official stance on it from Bitwarden to know their take and whether they're considering supporting this important use case in an official capacity (e.g., sponsoringor providing some kind of support for the project). Seems like it could be a big differentiator over other password managers.
I intentionally use other things for my 2FA and TOTP so that my most important accounts are still not accessible even if you somehow get into my password manager. I use YubiKeys where I can, Google Authenticator when it has to be TOTP.
I found LastPass painful to use and sync between local vault and server side to be broken. No thought has been given to layout, commonly used options are buried and basic things like selecting the right credentials by subdomain do not work. Their recent UI refresh has simply made things slower rather like Google's admin UIs. They have rather annoyingly decided, against NCSC advice, that I need to see a reminder to pointlessly cycle my master password every time I log in.
The final straw was when they applied a large renewal charge without authorisation to a card they were not given permission to keep and then mishandled the resulting complaint in every way you could possibly imagine.
Bitwarden is cheaper and far more usable, I can't find any single thing that LastPass does better for twice the price.
Work is switching to Bitwarden due to easier ability to integrate into our environment than 1Password.
Now I'm testing waters with bitwarden. I like the cross platform functionality so far and the self hosting option. I also like that I just need a master password and don't have to worry about keeping any extra keys safe. I'm not a security expert so I'm not sure whether encrypting before syncing with bitwarden servers is actually safe (this is what bitwarden does afaik). I'm yet to try out their cli option. I also wonder what would happen to my passwords if it shuts down abrubtly. Do I have a backup/copy of the passwords somewhere? This is something that concerns me, where I feel pass is superior. Maybe if there was an option for pass, to use passphrase for encryption rather than gpg, that'd be really cool (maybe not good security wise? I'm unsure on this aspect)
I also liked that when I add the URI of the website login, it gives the icon for it too. Bitwarden's user experience is top notch. I recommended my parents to try it out, except for a few basic questions they were up and running within a few minutes. That's something I really appreciate.
If anyone has self hosted bitwarden, how do you make sure that it is safe from attacks? I'm still exploring this option. Bitwarden uses azure and lets the MS team take care of managing the infra (I'm guessing this includes taking care of attacks).
For more important things I use KeePass and keep it all offline.
Problem with Bitwarden etc. is that if your computer or one of the devices gets compromised, then all of your passwords are lost at the same time. With hardware based vault you can mitigate this somewhat, with rate-limiting, physical prompt etc.
I have 1600 passwords, I use perhaps 10 different per week. If I get compromised I loose 1600 passwords, but with hardware based system I potentially loose just the 10 I used within the last week.
If you do need to have shared passwords (dev/stage/prod servers and services) why not Bitwarden for Business? https://bitwarden.com/#organizations