Urgent and Important – Rotate Your Amazon RDS, Aurora, and DocumentDB Certs
aws.amazon.com
aws.amazon.com
Or have they done it in a different way for security reasons?
And behold, Amazon are themselves a globally trusted root CA! (Have a peek in /etc/ssl/certs, on must current 'nixen I would expect to see the four Amazon roots). So why not have the Amazon root sign the RDS CA certificate as an intermediate CA, so that clients who could trust an RDS-issued cert that way, will magically trust it?
I've met some of the RDS team and they are smart cookies. If you will agree to my use of cookies, then I can think of at least three good enough reasons (and just to be clear, this is speculation and extrapolation, not insider chapter and verse):
1. Changes in behaviour during another potentially disruptive change are the province of the unwise, the unwary, and the otherwise soon to be fired for cause.
2. Not all clients can use a default trust store; even different clients for the same database may differ. For example, the PostgreSQL C client (as used by the command-line tools and many language bindings) cannot access the system certificate store; the bundle must be configured explicitly. For one client to have a magic behaviour that others do not, in the realm of cryptography and trust, is going to make security admins and auditors very unhappy. Inconsistency is the enemy of policy, and policy makes the DevSecOps world go round.
3. Not all clients support a certificate chain, so the RDS team already has to maintain a bundle of the regional intermediates. Making the RDS CA an intermediate just adds more complication to that activity, more special cases to document, and more potential for confusion in client configurations.
So these are security reasons, although they're more from a policy and human factors and software inadequacy perspective than anything to do with, say, large prime numbers. It's quite possible the service team have more reasons besides; I would certainly agree to their use of cookies or indeed any baked product. For me, those reasons alone would suffice, and I am not surprised to see that the new RDS root is another standalone self-signed CA.
I just put my root certs in ~/.postgresql/root.crt and it all works, right? (unless the root itself changed, but this should be stable)
> Not all clients support a certificate chain
Any specific examples? psql and PostgreSQL JDBC both support chains.
What would even cause that? Handrolling an SSL library?
Also ensure sslmode is set to verify-ca or verify-full, either as a set parameter or via the PGSSLMODE environment variable. See https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/ssl-c...
As ever, the #1 tip to maximise confidence in your configuration is to automate it, write a failing test case, then fix the test.
> Any specific examples?
Some of the new-wave SSL implementations (including yaSSL, with which mysql-client can/could be built) didn't always support them (not sure where yaSSL is at now), and the AWS documentation hints that it's cropped up in some circumstance in GovCloud, but I think the most likely circumstances will be enterprise folks using appliances and/or appliance-like software that's configured only via a web UI, one in which they just didn't think chains were necessary (or that only supports a small bundle etc etc; the sky's the limit in this class of software gore). I've seen this exact issue, albeit for an LDAP client not a RDBMS.
Re: anything w/ security (esp how it works), if you don't know ASK. ASK. ASK.
PLEASE ASK.
Security is incredibly complex and minor/subtle mistakes can destroy companies and other potentially catastrophic consequences (especially to your employment.) If you're not sure about something, ASK. End-users are bad enough when it comes to security, but engineers should always feel empowered to "make sure" about all matters of security, large or small.
Obviously it's best to to be familiar with best practices, but if you're not a security "expert" and you're unclear about something, ASK.
There no dumb questions in security, just dumb practices. The dumbest thing of all is not asking your question because you think it's "obvious."
tl;dr: ASK!
For better or for worse, that's not the case. These days, security mistakes just end up in 2 years of free credit monitoring. Ironic case in point, Equifax.
I'm not saying you shouldn't take security seriously. But it's not fair to say that insecurity could destroy your company. Mainly, most end users just really don't care (or have a choice).
Yeah, that would work if someone at AWS would make a bit of sense and would sign them using their pubic CA, but they decided not to and instead generate new RDS CA every 5 years.
Anyway at least with PostgreSQL clients that rely on libpq (the main postgres client library) by default while it prefers SSL when available it won't verify the certificate unless you explicit use verify-ca or verify-full sslmode.
My small SaaS company's PostgreSQL RDS instance and app servers are in a VPC with security groups configured to only allow connections from the app servers to the DB (no public access to the RDS instance). My client (ruby-pg) on the app servers is connecting via SSL, but not currently with certificate validation (though I believe the cert date still needs to be valid [?], hence the need to rotate the PostgreSQL server's certificate).
In this scenario, how important is certificate validation? I understand the theoretical risk of clients not being able to fully trust that they're connected to the database I intend, but from a practical standpoint, it seems that if an attacker is able to poison the VPC's DNS and trick the app servers into connecting to something else, I'm already hosed and cert validation wouldn't do much to help me. Am I missing something obvious and very dangerous?
However, in general certificate validation would protect you. The attacker wouldn't have a valid certificate, so your application would refuse to connect to the malicious endpoint.
[0] https://kevin.burke.dev/kevin/aws-alb-validation-tls-reply/
Of course the keyword here is “should”, and depending on your threat mode you may or may not want to have an additional defense layer of SSL certificate validation.
This is how I lost the last two days of work. Learned a lot at least.
SELECT s.pid, s.ssl, s.version, a.client_addr, a.usename, a.datname, a.query
FROM pg_stat_ssl as s
JOIN pg_stat_activity as a ON a.pid=s.pid;
I wonder why?
Are these CAs subject to... shall we say, additional goverment oversight?
Something tells me those master keys aren't quite as secure as you'd expect in other regions.