On the one hand, the goal of Unicode is to be able to encode all scripts, historic and contemporary. The swastika made it into a number of scripts before the Nazis ruined it for everybody, and so if Unicode is to encode those scripts, then it needs to include that character. It's easy to argue that it doesn't belong in the emoji block, but instead with the glyphs for the scripts that include it, but this is a technical detail that does not present any obstacle whatsoever to its use.
On the other hand, removing an emoji does not, in any way, prevent expression of the concept it describes. It leaves a very bad taste in my mouth, particularly when justified by what amounts to magic-bullet media theory, but to call it censorship is frankly absurd.