I see your point, but I think that attitude of preserving your own financial freedom is much like an adhesive bandage: If you pull it out slowly, the pain will be bigger. To keep his financial freedom he'd have to: 1) be very careful in which charity to invest (which likely would go like the joke: you are the optimal sort function, always comparing, never evaluating on an external scale), 2) invest just enough to keep it, which means some charity may get underfunded or not be as successful as it could otherwise.
You'd want him to "make things happen that would be simply impossible otherwise [if he didn't had 5 million]". My rebuttal is that he's done exactly that, but he's done so in one take, not allowing his fear of losing independence from later interfering with his ability to change things. That takes knowing oneself, and should be admired.