I thought the same as you.
The Wikipedia page on Holocaust victims (https://en.wikipedia.org/wiki/Holocaust_victims) has a list of the different groups targeted by the nazis. In addition to what other commenters have written there was also: freemasons, spanish republicans from the 36 revolution, soviet prisoner of wars, slovene and romani.
Interesting to note that nazis killed as much ethnic poles as polish jews.