This is because Kelvin, like meters/miles, are already a measurement of some physical feature, in this case of kinetic energy of the particles.
The word "degrees", on the other hand, does not represent a meaningful measurement of anything. It just means a given part of some arbitrary whole, defined by convention.
e.g. the "whole" of an angular degree is just a single complete rotation. The "whole" of a Celsius degree, I believe is the range that goes between water freezing (0°C) and boiling (100°C) points, and that's why it was also known as the centigrade scale.
Extra curiosity: still as of nowadays, if you say the complete temperature unit in Spanish, you don't say "35 degrees Celsius", you say "35 centigrade degrees" (35 grados centígrados). At least in a colloquial context; not sure how they'd say it in a science-related professional environment.