Ahh so that's what it is. I was confused because my coworker has the MBP with an M1 and dual monitor works fine for him. But that ran contrary to all research on my MBP specifically. Seems I'm due for an upgrade at some point in that case...
The 14” and 16” MacBook Pros have M1 Pro and M1 Max processors and they support more video streams. The M1 Pro supports 2 external displays and the M1 Max supports 4.