We did exactly what you are talking about (clockwise to zoom in, counter to zoom out). People got used to it, but it was mentally the same as using a slider bar to zoom: they felt like they were using a tool to zoom the view, not interacting directly with the view. Plus, some people had trouble understanding that it was the sweep angle that we were measuring and not the linear distance that your finger was traveling.

I like the idea about the circle turning into a line though - that at least solves one of our problems.