Honestly, the very first one was a lot more innovative:
- completely original licence, with terrific characters, world design and music composition
- incredible graphisms pushing the limit of what the console could do
- new gameplay mecanisms everywhere: dual play, animals, barrels...
- humour a the center of platformer for the first time
The 64 version just reused that. I have to assume this was amazing for the people that started the serie with this one, because while it was nice, it's probably the least remarkable of the whole serie.
It's like when everybody told me how good Mass Effect was, the pinnacle of RPG. Then I played it and was stunned people enjoyed it so much. It was a far cry from even classic titles of 10 years before. Linear, bad UI, lacking combat system and a scenario ripping off Babylon 5. But then I realized: if this is your first big RPG, then yes, it makes sense.
So if it was, indeed, your first DK and you haven't played D&K, then playing the rest may feel backward. And you enjoyed all the goodness in one big shot.