I can't speak to his work on finance as a whole. Regarding deep time, his claims about pre-literate society from archeology are not widely supported, they use thin evidence to argue badly.
His anarcho-socialism isn't the concern. It's his lack of historicity, and inability to bring his peers with him on radical ideas which concerns me.
He's dead, he can't defend himself. So there's that.
As for Graeber being controversial: yes, though I vaguely recall "The Dawn of Everything" being (moreso) the trove of interesting historical anthropological hypotheses, rather than "Debt"?
Anyway, it's been a while, but my main point is that I wouldn't let Graeber's controversial-ness stop anyone from reading Debt. If anything, going in with that information makes you think harder about the topics he covers.
I saw this in the flesh at a book festival. Dale Spender, a notable feminist author who moved sideways into IT tech (she was involved with online learning systems) did a book talk and the majority of questions from the audience were "Gotcha" attempts about here philosophy and feminism, with nothing to do with the subject at hand.