What boom would you suggest for a physics based approach to QC? Other than a solid understanding of quantum theory of course
On the other hand, quantum control (physically manipulating your quantum computer) has a lot of physics in it, but also crosses into EE. Likewise, looking into physical systems that can behave as a quantum computer is physics and EE. However I'm very much unfamiliar with that part, so I can't say anything there.
[1] https://www.nature.com/articles/ncomms5213
[2] https://iopscience.iop.org/article/10.1088/1367-2630/18/2/02...