Agree and disagree. The touch screen is the most important aspect of a touchscreen device. Observable lag affects the user experience when they're moving faster than the device will allow.
It is a big deal but I don't think the majority of users are bothered because they see it as an inherent trait of a computer to lag.