Unremovable through remote access, but reliably removed by reflashing the firmware through JTAG, or whatever interface used during manufacturing to initially load the firmware.
It's not impossible to do in the field, but you can't really count on vendors surfacing that interface usually.
Pogo pins are only really needed for mass production, especially for reducing repetitive stress injuries. For one-off updates, if a header isn't populated, it's easy to hold an unsoldered header in place, for long enough to flash an update.