Julia is flexible enough that you can essentially define domain specific languages within Julia for certain applications. In this case, we are using Julia as an abstract front end and then deferring the concrete interface to vendor specific GPU compilation drivers. Part of what permits this is that Julia is a LLVM front end and many of the vendor drivers include LLVM-based backends. With some transformation of the Julia abstract syntax tree and the LLVM IR we can connect the two.
That said we are mostly dependent on vendors providing the backend compiler technology. When they do, we can bridge Julia to use that interface. We can wrap Vulkan and technologies like oneAPI.
https://github.com/JuliaGPU/Vulkan.jl https://github.com/JuliaGPU/oneAPI.jl
As for syntax, Julia syntax scales from a scripting language to a fully typed language. You can write valid and performant code without specifying any types, but you can also specialize methods for specific types. The type notation uses `::`. The types also have parameters in the curly brackets. The other aspect that makes this specific example complicated is the use of Lisp-like macros which starts with `@`. These allow for code transformation as I described earlier. The last aspect is that the author is making extensive use of Unicode. This is purely optional as you can write Julia with just ASCII. Some authors like to use `ε` instead of `in`.