I was an aerospace major who worked mainly in software. It’s two very different worlds. A degree is a must. I wouldn’t say physical engineering is harder than software, but like say medicine the corpus of knowledge required as a baseline is much larger. A lot of the job is analysis, and you just have to know the math, the heuristics, etc.
You’ll also be a much smaller cog in a much larger machine. You can’t single handedly design a bridge the way Torvolds wrote the Linux kernel. That’s ultimately what convinced me to pivot to avionics and then ultimately software as I got through my degree.