Definitely semantics. For example, I have a BS/MS in applied math from a good engineering school university. I am a software engineer mainly working with ECEs, physics, and other math guys, who are all "software engineers".
I see where you're coming from about being a SW eng without knowing a lot of math - but there are other majors - math/physics/stats etc that will be very math heavy and not "engineering degrees".