"At least 20 years" is almost certainly an overstatement.
Slippage of what people call it is only to be expected for something that doesn't really formally give itself a title (ADVENT being more of a filename), and what it does most directly identify itself as became so synonymous with the genre. The game starts up with "WELCOME TO ADVENTURE!!" and then the instructions (if you ask for them) begin "SOMEWHERE NEARBY IS COLOSSAL CAVE..." so the naming is not exactly out of left field.
You can pretty readily find evidence of people calling it "Colossal Cave" and "Colossal Cave Adventure" already in the 80s.
I think the first time I saw it (a Mac port from the early 90s or earlier among a pile of other free/shareware games on a set of floppies), it was called "Adventure," obviously also a popular choice for what to call the game, and my preference.