Depends on what your reasons are for leaving. If you are leaving for more money then maybe the CEO will be willing to pay you to stay.
If you're not willing to approach your manager to ask for a raise then I would argue that's enough of a reason to leave even if they're willing to offer you more cash.