command-tab will let you toggle back and forth between last two apps - sort of a classic alt-tab behavior from Windows and various UNIX WMs.
command-tilde goes through your windows in a stack, in one direction, and you have to modify it with shift to go a different way. Further, if you destroy a window after command-tilde to get to it, suddenly the direction gets reversed, and you will then command-tilde back the other direction.
So ... one of these is "wrong". I'll leave it to you to decide which. From my perspective, the ability to rapidly switch back and forth between two windows of the same application is valuable, and I hate having to modify it with shift as I go back and forth...
Command-Shift-` will reverse the order Command-Tab will flip back and forth unless you exceed the delay while still holding command, at which point you can cycle through the stack. Holding shift reverses the direction.
That being said, I agree that swapping back and forth in between windows of the same app is very useful and it's a pity that the two features aren't made to work in the same way. I'd like to do a quick Command-` to toggle two windows, or hold it down to see the list of windows for the current app.