It's more what don't they do that Finder does poorly?
Personally, I find that explorer pretty much does what I'm expecting (most of the time). And it's not really just a Windows thing, as far back as I can remember, across several OSs, you don't really have to "figure out" how the file manager works. You click on something and it pretty much does what it says. You drag something and it moves there. Various *nix window managers, Amiga, geos, BeOS, and on and on and on are pretty much just walk-up usable.
With Finder I feel like I'm gambling. Where in the filesystem hierarchy will the new folder I just created get made? Why is enter rename? Why do you show me a useless file path that I can't use for any navigation? Network shares disappear. It's never wide enough to actually see the names of my files. All my files is probably one of the dumbest notions I've ever seen in a file management tool -- yay, I have a uselessly long list of all 2.7 million files I can go through, and we'll default to it also! and on and on and on, hour after mind-numbing hour. How about this folder? How much space do I have left? How big is the selection of files I have currently selected? Why don't the unintuitive iconoglyphs have some kind of description? Why are these ones down in the window, but other ones I need to use a million times a day up in the tool bar?
Almost every interaction with Finder feels like I'm about to lose data, or the default is so completely brain dead that I have to go into preferences to fix almost every interaction to make finder even remotely useful. It's no wonder there's a dozen or so finder replacements that end up as default installs for most of the old-timer Mac guys I know.
It's unbelievably unintuitive bitrot -- endless preference fiddling, and the feeling of walking a tightrope anchored with eggshells with every interaction. There's really few pieces of software I loathe to use, and it's a shame that it's pretty much the most basic piece of software in the OS.
In the context of how abysmal finder is, it actually makes sense that Apple has gone out of its way to abstract the user away from the file system. From the perspective of a Windows user it seems like madness, but after enough time in OS X finder hell it makes perfect rational sense. You simply don't want to subject users to this level of frustration just to organize their mp3s or pictures. It may also explain why similar efforts in Windows land don't do as well, managing and organizing files is no less mysterious or confusing than any other organizational app, usually less so.