These timing attacks are all about local time deltas, not absolute times, because it’s measuring the CPU caches, not anything to do with the network. Things like “this operation took 5µs, which suggests that the data it was accessing was/was not cached”.