Software Engineering is in a way meta engineering discipline. Problem domains are vastly different. A mobile game development is different than building a system software for surgical robot. The first step is to define the vocabulary of the solution space and then build the solution based on that vocabulary. Expectations of reliability, maintenance, requirement changes adds another dimension to its solution space.
Engineering has a well-defined constant you're always building against: gravity.
In software, you choose your gravity, and it changes all the time. Sometimes there are twelve gravities. Sometimes there are different types, and they interact in complex unpredictable ways.
In short, it is a much more difficult problem. But personally I believe we can reach a meta-engineering capable of achieving quality in a predictable way.
Engineering is not as simple as building against gravity. There are a lot many things that "engineers" (non-software) have to take account of, even in something as mundane as laying out a road. In this example, you not only have to build the road within the area; you have to build it so that it has the right curvature so that vehicles can make the turn etc. You have to make sure that the road you're laying out has a solid base; you need to make sure that the road can withstand the natural forces of that region for whatever time it is designed to last against. So I don't understand how engineers build against gravity.
But if you're saying that engineering has to follow the laws of physics, then yes, I'm totally on board. Computer Science departments in most universities started of as branches within the mathematics department. While most of software engineering is more than simple math; it does operate on its own plane, with no regard for the laws of physics (except maybe of time :) ).
In that regards, Software Engineering seems more of an art, doesn't it? But it does have its own guidelines; which govern how good software is written.
A road is a bad example. The only reason a road is required is because of gravity.
I disagree with your hand wave of "software = art". Software is also bound by many constraints. Time (as you mentioned) is not trivial because it's the difference between useless and useful software. Memory also puts limitations in that can't be ignored.
Useful software runs on real machines bound by physics. Theoretical computer science can disregard the constraints to see what's possible, but that's CS and not SE.
Software engineering is an art in that it's more like theory building than theory using. Each project, each context is its own set of new physical laws.
Software is not all made up out of thin air -- you do not 'choose your own gravity'. The equivalent of phyical laws in software are computational/algorithmical, logical laws.
For example, sorting cannot be done faster than O(n log n) -- that is as hard and objective as anything physical. (In fact, one would think it is even harder in some sense, since it is so purely logical.). Software is built within algorithmic constraints.