Right, this has been the problem with FSD is that it is incapable of dealing with situations that it did not see in its training dataset. It seems incapable of abstract general reasoning that AGI would have, and that humans do.
This is important for all the generalized cases of mayhem you see driving on local roads. Wrong way drivers, cars driving in reverse, emergency vehicles driving in opposite lanes, bikes not in bike lane going wrong way, etc.
There was a nice YouTube video last summer of a Tesla FSD car perfectly identifying that there was a tram moving in its direction & rendering it in 3D on the center screen. However, FSD tried to make an unprotected left turn in front of the tram anyway, the excuse being the usual "well it hadn't seen an unprotected left in front of a tram in its training data".
AGI/more abstract and generalized reasoning is required for "things that never happened before happen all the time" state of the real world.
None of the self driving systems seem to be able to infer intent from other vehicle behavior/pedestrian body language. Something that any B- or better driver should be able to do. Signals, coasting, braking in certain manners that you can infer they are about to turn, change lanes or try and park. Or that they see an obstacle in the road, etc. This is free information that these systems do not attempt to use.