That’s because regular M1’s can only support two displays. In the case of MacBooks, that’s the internal display and one external.
M1 Pro and Max do not share this limitation.
Apparently DisplayLink adapters get around this limitation.
Can confirm DisplayLink works fine
Sounds like you didn't do enough research. I am using a CalDigit TS3+ with 2x4K and 1x1080p external displays on a M1 Macbook Pro to write this comment. It is not difficult at all to do and took less than five minutes of searching for me to find the guides and instruction videos when I decided to upgrade my Macbook to Apple Silicon.
What cable and connector types are you using please?
All monitors have DisplayPort and HDMI inputs, using DP<->DP, usb-c<->DP, and usb-c<->HDMI (latter for the 1080p display). Also using DisplayLink Manager to drive the 2nd and 3rd extra displays.