That $35 SBC is running a CPU that costs a comparatively lower amount to produce than an M1. Putting 20 billion transistors on a single die reliably at 5nm is extremely more costly than a billion or so at 28nm (the BCM2711).

If you want Broadcom to release a CPU that's on par with the M1, half of your Pi's circuit board will be just the CPU die's BGA pins.

Intel's (and others') price gouging and market segmentation does not in any way imply that it's possible to manufacture an i7/i9/R7/R9/M1 at $20 a piece.