Worth mentioning their VPN recommendations: algo by trailofbits and freedome. There is another paid service they recommend but I can't recall the name.
First of all, there does not appear to be a whitepaper available that describes the security architecture in any detail. This is an immediate red flag.
Second, they do have a "Security Features" page which is rather light on the details; it mentions that ProtonVPN uses AES-256 (encryption), RSA 2048 (key exchange) and HMAC-SHA256 (auth).
I'll start with RSA: the fact that they use RSA at all for a new cryptosystem in 2017 is a red flag for me. I also can't see any details of how they use RSA, so I don't know if they have implemented padding. If they haven't implemented padding (and done so correctly!), the VPN is insecure and we can stop right here. Honestly, they should be using ECC. I'm assuming they're not using something like ECDSA because RSA is faster (but not so much so to justify the potential security tradeoff, even in a VPN client).
On to AES: they commit the common marketing-mandated-security-page sin of focusing on the key size instead of the block cipher mode. They don't explain which block cipher mode they're using for AES at all - another red flag. For all I know they're using ECB (in which case, the VPN is insecure and we can stop right here). This is putting aside the question of whether or not they correctly implemented AES in whatever mode they're using.
With regards to HMAC-SHA256: in theory this is fine, but again we have no details. I'm going to go ahead and dock another point here because they're choosing to use separate primitives for encryption and authentication, when the best practice would be to use authenticated encryption like AES-GCM or AES-CCM. I admit this is bikeshedding a bit: respectable cryptographers (like cperciva) have a preference for separate construction. However, this is a VPN we're talking about, and an authenticated encryption mode would be faster than separate encryption and authentication.
A few caveats to my points: I'm quarterbacking their cryptosystem design based on one paragraph of the security page, because that's all I can find that describes their crypto. It doesn't describe it in detail, so it might still be secure. I have no knowledge of their implementation, so I can't critique that. That said, if I had to weigh the red flags I've observed here against their "developed by scientists from MIT and CERN" marketing and nothing else, the red flags win out.