It should be enough to look at the IP address.
While that's true, it is not that many IPs, and they do not change that much.
Maintaining a list of IPs for YouTube, Netflix, and other mainstream services is probably a couple hours of work per week at most, probably distributed between multiple people in whatever department handles it.
At this moment, they will have allowed IP ranges, that would cover for example; YouTube.
Edit: Also, an ISP can always see what your endpoint is, as they would have to in order to give you the connection. The only exception to this being VPN's and TOR.