1. Take the site down and prevent any further leaks. But accept the absolutely massive flak that would come from that.
or
2. Continue to let the leaking happen and try to hotfix the bug. Be accused of failing to get a handle on it, but perhaps get away with it if fixed in the next 20 minutes.
I'd bet that it's been like this for ages and only just been noticed. That will make fixing it a bit harder I should think - not simply a rollback to the previous push.