That wouldn't make it any harder. Sides only have a single orientation, with the opposite side suited to the center piece of one of the neighboring sides. Placing the piece with the correct neighboring side addresses the issue automatically.

Corner pieces can be rotationally incorrect, through the solving process, and is one of the harder areas to sove on the puzzle.