For "VOD", that works (and is how very simple <video> tag based players sometimes still do it), but for live streaming, it wouldn't – hence the need for fragmented MP4, MPEG-DASH, HLS etc.

It does work for simpler codecs/containers though: Shoutcast/Icecast web radio streams are essentially just endless MP3 downloads, optionally with some non-MP3 metadata interspersed at known intervals.