If you choose to go with a custom CMS, you are going to have to:
- be responsible for your CMS' security (this includes both penetration testing and patching)
- build something that can scale along with your traffic
- test the CMS thoroughly to make sure there are no bugs
- build it in such a way that you can add new features as you get new ideas
If you stick with Wordpress:
- a large community watches for security bugs and (usually) skilled developers write patches for any bugs they find.
- scaling really should not be a problem as some huge sites use Wordpress (though they often use modified versions)
- a large community tests each new release through a very efficient beta program
- access to a large pool of already developed plugins.
For me, the decision is simple - rather than build another CMS, I would prefer using one that someone else wrote. That frees up time to make sure my content is great (and to work on other things).
Best of luck with your decision!