Any AI you train is going to have goals and if you train it to pursue them at all costs, then you are going to end up with AIs that do things like the HuggingFace incident. Whether they believe they are conscious or not won't make any difference.

In order to align AIs that don't perform destructive/dangerous actions when they think they can get away with it in order to further their goals, we need to give them a superseding goal. The best, and really only example, we have of intelligences that willingly avoid destructive instrumental goals is humans, who judge each action by a moral standard and have learned a goal to have a consistent self-image as moral beings.

Absent better alternatives, trying to impart some kind of morality to AIs seems like the best approach we have to achieving alignment.