Not rinsing also leaves all the food residue, germs (dead or alive) and debris that has collected in your mouth throughout the day. Not rinsing my mouth after brushing feels like showering and leaving the soap on.
Think of the paste more like something you need to apply and let sit, and less as some kind of fancy soap to wash the teeth with.