I'm a CS PhD, and I agree. I always tell people I'm studying math because it gives them much more of a flavor for what I'm actually doing everyday. When I say computer science, people think I'm just programming something.
Also, when people ask me, "So What do you actually do studying math?" I reply, "Sit and stare at the wall for several hours a day. Occasionally I write something down."