But if your primary concern is performance, please don't roll your own vector or matrix "native" interface. You will certainly never come close in speed to what has come before (BLAS implementations galore, et al). Also it's just a lot of work that is basically keeping you from working on the higher order problems out there that we desperately need to tackle.
If your goal is more "Clojurey" syntax then just spend a day or two wrapping the functions you want over a tried and tested numerics implementation. Additionally, there is likely a pre-existing Java wrapper which does just that for whatever you need considering that Java is still beloved by university professors, a key demographic for fast math libraries.
On the other hand, I think Vertigo ( github: https://github.com/ztellman/vertigo ) is taking a very interesting approach to the Clojure->Native problem, which I believe might be of use to any library wanting to bring performant numerics to Clojure. Unfortunately, ztellman has deprecated his OpenGL and OpenCL libraries, but I think that Vertigo in combination with OpenCL and the kernels courtesy of clMAGMA would be fantastic.