He may already sense something isn't right so it could be a lot easier than you think. Someone leaving the company always has the added bonus of reducing overheads and increasing runway (this is little consolation if you're irreplaceable). I wouldn't worry about it too much, he'll survive and you'll feel infinitely better once you've said those words.