In my opinion it isn't arcane, at least, no more arcane than other "soft" design fields. But the simple rules are complicated to follow.
Your first three principles are about efficiency. That's a big part of usability. When you reduce clutter, you make it easier to use by clarifying the next action the user should take.
But "No surprises" is rather complex. It means that you not only have to explain what you are doing, but also work with the user's mental model, which might be totally wrong.
Here's an example from my current project. We are designing a web form to upload images and video to Wikipedia. In 2010, the users' model of media uploads comes from Facebook, ImageBucket, and maybe Flickr. Even after you make them click on a box that says they give up their own rights before proceeding, they don't understand what they did. Apparently people's mental model just continues on blithely until it smacks against a major contradiction.
So we've tried various metaphors. "License your work" was replaced by "Release rights", and the next iteration is going to be "Donate your work". That wording was the suggestion of a usability testing and analysis firm we hired, by the way. So it does take a certain amount of inspiration to come up with alternatives, but recognizing where the problem areas are is just a matter of measurement.
Inspiration is aided by just having a bigger bag of tricks, too -- knowing common problems and solutions. For instance, if there is so much to do on one page that users are finding it overwhelming, it is helpful to break it into multiple steps (a "wizard"). That's another approach we have taken.
Anyway, I'm not sure if my expertise with usability is so deep that I could pronounce just a few principles. Maybe "reduce cognitive load to the minimum" is one, which has been more memorably distilled by Steve Krug as "Don't Make Me Think". Since we want to avoid onerous information transfer and cognition, there are some models of communication that might be helpful here; a classic text on human communication is The Responsive Chord, by Tony Schwartz. He argued that most communication is about activating stuff you already know (at the right moment), which you can see at play in the example I gave.
But ultimately it's about motivating human behavior, so it's only about as complex as the human heart. ;)