It's basically just people who go deep on the spirituality part of religion, with very intense prayer and meditation, very focused lifestyle, etc. Religiously they generally adhere to the mainstream schools of Islam and consider themselves part of the wider religious community.
The more recent movement of Salafism, centered in Saudi Arabia, has made Sufism one of their targets in their quest to 'clean up' Islam, since it doesn't fit into their back-to-basics view on religion. So there is quite a big schism now between this movement and its followers, and the general community (for whom Sufism is an ingrained, and often treasured, tradition).