Given that there is a physical difference between events separated by positive and negative distances in spacetime, I think it's pretty clear that the prefactor is a property of spacetime.
Yes, it is mostly semantics but the question really is what you think the space-time metric is. In order to attribute the prefactor to time you would need to argue that the space-time metric really is Euclidean, but because time is different from space there is this negative prefactor showing up in the metric. But you can also claim that time and space are the same and the space time metric is hyperbolic. Since there is there is no good argument why metrics necessarily need to be Euclidean, I find the second option more convincing.
