It's adenosine blocker not inhibitor, so you're producing more adenosine while under caffeine than without, it just doesn't work. That what kicks in after withdrawal. The good news are, it's not going to kill you, so going cold turkey is ok. Of course, you can also reduce your intake gradually. Try switching to decaf for after dinner coffees, this would be the first step.