(gross simplification below)
I've done a little research on caffeine, and as far as wakefulness goes (not touching the other effects here), it works by binding to the same protein receptors the body uses to decide if it is tired. Body unable to get signals that should trigger a feeling of tiredness == Body not conscious of being tired.
BUT the body responds by producing more of those receptors, making it more likely that it will start getting signals of fatigue even when the signaling proteins are at a low concentration. Thus, regular caffeine takers will find they need caffeine to feel a "normal" level of tired. It takes roughly 2 weeks without caffeine for a regular caffeine taker to achieve a no-caffeine level of such receptors.
Now, how much of the above is pseudo-science? Great question. FWIW, when I did the research the sources seemed credible enough (naming actual proteins, referring to actual studies, not just giving me animated drawings and hand-waving details) and it at least correlates with my anecdotal data - I don't use caffeinated drinks or supplements as a normal rule but do use them for special events (unusually late night/early morning) and have found the wakefulness effects to be effective unless I slip up and get too lax in my definition of "unusual". I also never thought caffeine was a big deal ( I never knew withdrawal symptoms, for example ) until I stopped using it as a general rule and my sleep improved notably. That said, my sleep quality is generally terrible so it doesn't take much to improve it.