If they let you turn it off, then the person who got your password and started forwarding the mail would turn it off, and you'd never see it. If they let you turn it off, there's no reason to have the message in the first place.
Edit per the response: I don't think this is a threat model that you would care about in theory, but it seems like a scenario that can come up fairly often in the real world. Especially in family or school settings, someone else will often have short durations of access to your machine. These people aren't often going to be technically literate enough to install key loggers or really mess up your machine, but they may well be able to quickly set up forwarding of your email to their account.