I think we are already in the stage where AI can and does improve itself. But why should this stage be called "Singularity"? Like a Black Hole? That sounds like hype to me.
When AI can improve itself, wouldn't it still be able to explain to us how it has improved itself? If it can not it still has a lot of improvement to do.
Or are we saying that some things are "unexplainable" and AI will discover such things without being able to explain to us what they are? That sounds like mysticism, or hype to me. Or religion. We can not explain God, right?