Engineering has a well-defined constant you're always building against: gravity.
In software, you choose your gravity, and it changes all the time. Sometimes there are twelve gravities. Sometimes there are different types, and they interact in complex unpredictable ways.
In short, it is a much more difficult problem. But personally I believe we can reach a meta-engineering capable of achieving quality in a predictable way.
But if you're saying that engineering has to follow the laws of physics, then yes, I'm totally on board. Computer Science departments in most universities started of as branches within the mathematics department. While most of software engineering is more than simple math; it does operate on its own plane, with no regard for the laws of physics (except maybe of time :) ).
In that regards, Software Engineering seems more of an art, doesn't it? But it does have its own guidelines; which govern how good software is written.
I disagree with your hand wave of "software = art". Software is also bound by many constraints. Time (as you mentioned) is not trivial because it's the difference between useless and useful software. Memory also puts limitations in that can't be ignored.
Useful software runs on real machines bound by physics. Theoretical computer science can disregard the constraints to see what's possible, but that's CS and not SE.
For example, sorting cannot be done faster than O(n log n) -- that is as hard and objective as anything physical. (In fact, one would think it is even harder in some sense, since it is so purely logical.). Software is built within algorithmic constraints.