It's quite possible to do this on-prem and even on-device. TabNine, a very similar system with a smaller model (based on GPT-2 rather than 3), has existed for years and works on-device.