Do you realise this reads as "don't bother validating your inputs"? Surely that's not what you meant?
> It begs the question: validate for what?
All inputs necessarily follow some protocol. Command line options and configuration files have syntax. Network packets have sizes & fields. Binary files have some format. So you just make sure your input correctly follows the protocol, and reject anything that doesn't.
Now the protocol itself should also be designed such that implementations are easily correct and secure. In practice, this generally means your protocol must be as simple as possible.
And it's probably a good idea to make sure your parser is cleanly separated from the rest of your program, either by the OS (qmail splits itself in several processes), or by the programming language itself (with type/memory safety).