How does this work with both? Is there a defined band plan standard that cable operators know to avoid? Do you need to install a notch filter on the line coming into the house?
To answer your question, cable TV uses frequencies between about 55MHz to 1GHz, but mostly starting at about 500MHz for digital cable. So if I wanted to transmit Ethernet over cable, I'd use a 2GHz band or something to avoid interference.
Well, to be pedantic, that is still how it works. And it's how cable Internet works. And it's also how a lot of techniques over twisted pair work, too.
The conductor is used as a waveguide and a very-much-analogue signal encoding a digital signal is what is sent over the line. DOCSIS 3 uses up to QAM-4096, which can encode 12 bits in a single symbol on the line, by using multiple steps of amplitude shift and phase shift, to encode bits. Quite similar to how FM radio works, just digital steps, rather than an analogue continuum between 0 and 100 amplitude and 0 and 100 phase, at the decoder.
This has even started showing up for links within a single computer, now. The latest revision of PCIe uses modulated RF (4-level pulse amplitude modulation) rather than simple binary voltage levels.
You also have to use appropriate splitters that are rated for the top ends of the spectrum.