I like the design concept of SFML a lot, but it's been neglected for so long that it's way behind modern C++. I see they plan to move to C++17 in SFML 3, and start using std::unique_ptr and such.

Ultimately I just moved to Godot to get stuff done, which is not pretty C++ at all, but it's about as lightweight as you can get for a fully featured game engine, as opposed to a framework like SFML.