Even in swift it isn't "turtles all the way down".

The AD stuff is hardcored into the C++ guts of the compiler, whereas Julia's source to source autodiff accesses a compiler pass from a fully Julia user package.

Aside from making it easier to hack and improve the AD system as just a Julia user, this capability enables other package program transforms like that in https://github.com/MikeInnes/Poirot.jl for prob programming.

So Julia is already further ahead in that regard and it's more hackable.