This is why you should always toggle the setting that disables 2g/3g fallback.
With 4G, for example, your device will refuse to connect fully unless the network can pass the cryptographic challenge that proves it shares the key material included in your SIM card (I know, I know, symmetric keys are not ideal). The best an attacker can hope to do in 4G+ is harvest your subscriber ID (IMSI) or deny you service while you are in range.