The cause in this case was AWDL (Apple Wireless Direct Link.) Holding the Option key while clicking the Wi-Fi icon and clicking "Enable Wi-Fi Logging" and then checking /var/log/wifi.log will show AWDL scans starting and ending randomly, and when the scan is active it causes latency spikes every 1s like clockwork. Unrelated to AWDL, but if a process is requesting a Wi-Fi network scan (different from an AWDL scan), /var/log/wifi.log will also tell you the name of the process, such as "locationd" when the Location Service needs your location. (Tangential, but the locationd process rarely causes these latency spikes for me - on a default macOS install it very rarely requests scans in my experience, backed by my analysis of the log.)
AWDL has to be used for things like AirDrop, so it's expected to have this latency increase while you have the AirDrop window open scanning for nearby devices / sending files to other devices. There are other uses of AWDL (AirPlay, Auto Unlock, Universal Clipboard at the very least)[0], but I don't know what was triggering it so actively in my case... and why it wasn't happening on my M1 Air. It also wasn't always happening in the background like this, it just started that day.
The "fix" was to disable the awdl0 interface, but that may also cause AirDrop/AirPlay and related services to not function (I did not test.) It's easy to re-enable it though.
To disable: ifconfig awdl0 down
To enable: ifconfig awdl0 up
Upon disabling, the latency spikes go away permanently.