If it's V2V you have the potential for malicious actors to send signals not actually describing what the car is doing. Visual/radar/lidar and AI interpretation thereof are needed as a fallback to prevent crashes. If it's not V2V you need mediation, authentication, and extremely high availability, which has antitrust xor lack of non-telco infrastructure. Cars need visual/radar/lidar and AI interpretation thereof as a fallback to prevent crashes just because a cell tower goes down, and will be completely inoperative until they go up in the first place. Human drivers change their minds all the time, and represent the vast majority of drivers right now. Humans don't even do a good job of following their GPS directions, let alone using turn signals. So visual/radar... you know the rest of this sentence.
Basically, the ground truth is always more important than whatever signals are being sent. If a malicious actor can't fool a human, it's mustn't fool a non-human driver. If missing or unavailable infrastructure can't make a human unable to drive, it must not make the non-human driver unable to drive. Decreased efficiency, okay. Slower trips, fine. But total denial of service (either by failure to operate or crashing) is unacceptable. All of it eventually relies on the AI fallback. So we should be (and are) building the AI fallback first. Every car will have the AI fallback, including V2X-equipped cars, and all V2X signals will be treated as potentially unreliable forever.
I don't doubt we can have some efficiency gains from V2X, but we'll likely never have bumper-to-bumper at 60 mph, interleaving though intersections, etc. Cars just aren't mechanically reliable enough. Say someone drops a nail or caltrop or cylinder of compressed gas out of car #0's window. How many milliseconds does car #2 have to react to the pending tire blowout of car #1 immediately behind? Even with an AI fallback, its sensors might miss the nail. Is a sudden pressure drop (presumably signaled) going to be transmitted and processed fast enough by the entire convoy of cars to avoid a large multicar pileup? Are ALL the cars' brakes in perfect working order? And that's not just a linear problem - all the cars in all the other lanes will have to brake too to allow car #1 to swerve unpredictably as the tire explodes as the car learns the new driving model it has to contend with (and it will need the fallback AI to figure out how its inputs are translating into real outputs). Not that this is an easy situation for human drivers to deal with now! But now we have 1-4 seconds (de facto-de jure) of following distance. V2X comms could make 1 second following distances safer. But it'll need an intelligent (artificial or human) fallback anyway, and having a human one means that the efficiency gains will disappear. So we need the AI fallback.