header-only usually means there's one or more .h/.hpp files that you can include in your source. There's no source file or library to compile and include
e.g. if I had Hello.h and Hello.cpp, you'd need to either add Hello.cpp to your make/build or build a library and link to it.
Header-only version means all you do is
#include "Hello.h"
and you're all set to go.