The EDSAC's "boot ROM" consisted of a bank of large electro-meachanical "uniselector" rotary switches, hardwired to contain the 31 instructions ("initial orders") of the tape loader.

Subsequent computers could have followed this "electro-mechanical ROM" approach, but I suppose it was rather bulky and inconvenient to change.

You can see what the uniselectors looked like here at 8:50

https://www.youtube.com/watch?v=Yc945sNB0uA

Of course programmable ROM chips would have to wait, so in the meantime the alternatives were either the toggle-it-in approach (still also used by the 1974 MITS Altair 8800), or other more compact types of handmade/hardcoded physical ROM such as magnetic core or diode matrices.