Also in case it’s useful: much ink has been spilled about TRRS and its alternatives, but I haven’t found any references to folks using an I2C connector (4-pin JST) instead, which eliminates hotplugging risk without introducing the risks of USB/etc.
I purchased a KB2040 (which has a female I2C connector), a 400mm male-to-male I2C cable, and an I2C GPIO expander, which is slightly cheaper than a second RP2040 on the second hand (though a second KB2040 is cheap anyway). I’m optimistic about using I2C and avoiding TRRS, but wanted to highlight it as an option since I hadn’t seen it mentioned anywhere else.