This is a nice feature of APL, and the derivatives such as J which use plain ASCII are much less nice to work with.
For looking pretty, I assume: that's probably a matter of UI (editor displays arrow character instead of equals+greaterThan), but should not propagate into actual source code.
To use your example, typing "=>" could display the "follows" sign, but the string representation should not depend on exotic characters like "ř", "⸙", or "" (is that a box symbol, or "symbol not found in current font"? Oh wait, the HTML input field ate it! See?!).
Most development IDEs are configurable and extendable in such a way. Or well, at least the one I am using is (Emacs). Just like opinionated languages have not had those choices stop them from becoming widespread (eg Python re indentation), so shouldn't the character set used either.
You could also redefine your keyboard layout (eg. a happy hacking keyboard has no marks on the keyboard) or come up with a programming input method (IM) to use — not everyone would have to do it, someone would make it and others would use it. But making wider use of the characters available has to start somewhere, and it can't start with input systems (before there's a widespread need for them).
Mathematics has developed a very large alphabet for the very same reasons, and if it was constrained to ASCII, we'd be learning integrals today in the "Newtonian way".
If you don't see those properties as offering any benefits to a craft that is based on precision and that has new languages popping up regularly to cater to new expression forms, that's fine. I still believe it's an unexplored area, and we'll only see benefits once we start to make heavy use of the advances.
As to mathematics: do you imply that using a wider charset is akin to completely new mathematical methods? Or that a symbol needs to be one character? Both sounds implausible, I still must be misunderstanding...
As for maths, I was referring to the fact that notation (signs we express ourselves in in writing) matters, and that further advances in calculus were enabled by using a nicer and more concise character set vs doing everything with "fluxions" and "fluents". You seem to insist on keeping us restricted to ASCII, whereas I am open to exploring new approaches without understanding if there are any benefits first (I am not focused in RoI :)).