It's true that with the isometric view, what is "right" and "up" can be confusing. I could teak the buttons on mobile, but not sure how to solve this for desktop.
On desktop, you could use keys other than the arrow keys (e.g. q/s for one direction and w/a for the other, since they're more diagonal, or even 1, 3, 7, 9 on the numpad). Edit: I had no trouble on desktop, for what it's worth.
Another option would be to change the view angle so that the directions more closely match the orientation of the arrow keys. It would throw off the nice isometric view a bit, but I assume it's real 3D, so this would be easy to change, right? :)