You are right. The reason Google supports net neutrality (for example) is to prevent Comcast from making a deal with Vimeo to offer QOS on Vimeo videos, allowing them to play at noticeably higher quality.
Comcast could even make a deal with a yet unheard of startup and suddenly Google would lose lots of market share to an upstart who was willing to accept Comcast's terms.