It's the logical end product of the odyssey from Apple's original philosophy of a resource and data fork model for files to the UNIX stream-of-bytes model for files. The UNIX model traditionally kept metadata about files separate (anyone else remember naughtily hand-editing directory files with sed(1), back in the day?) but gradually picked up a bunch of new stuff that has to be stored somewhere. Meanwhile, the Mac platform adopted UNIX binaries with the move to NeXTStep underpinnings (we're going back to the late 90s here, and the adoption of OSX over traditional MacOS), obviating the need for the original resource bundle, which got rid of those annoying errors about missing bundle bits but left us with a legacy of .DS_Store turdlets in every directory to hold file metadata that was formerly stored in the resource fork.
As a destination it's laudable — design consistency is almost always laudable — but it's the end goal of a very messy process and it looks like APFS isn't quite there yet; NSFileManager and NSURL need some way of distinguishing files with different unicode representations, and the Finder in particular needs to be robust. I'm guessing this is going to be fixed when High Sierra finally ships, but isn't supported in Sierra at present, hence the OP's alarm at the situation.