You're forgetting about Congress. Laws don't go away when a new president is elected. If civil servants can point to a law then that says they have to do something then the president can't directly override it. (Though there are workarounds.)

Executive orders are limited by what's allowed by the law. There has been some boundary-pushing but some changes require an act of Congress.