Your colleague is full of shit. Generally, neurodivergence is for everyone who regularly experiences that the way their brain works causes them trouble.
Self diagnosis is surprisingly accurate but people also tend to under estimate the severity of their symptoms.
Symptoms of mental disorders can be normal human traits taken to the point it affects person so severely it's an impairment.
This means anything normal can be pointed to as evidence of a disorder.
Even the "no empathy" sociopaths can spend decades thinking that they're perfectly normal, everyone is like them, and people just pretend to be sad and grieving at the funerals because that's some kind of established convention and breaking it would be very rude.
What I'm saying is: maybe you just think you don't show any signs of autism - because you think your experience is "normal", and you think that everyone has the same struggles as you do, even when it isn't true.
Or maybe you genuinely aren't autistic at all! It's just very, very hard to say at a glance.