On iPhone, it is. Can set “limited access” for contacts just as you can for photos.