In symbolic programming languages (like Lisp), very few characters are "part of language", as you say. In fact, some lisps (at least the ones I use: Common Lisp, and Racket) even support reprogramming the reader.