Hmm, no. The supreme court is not the only arbiter of that.
The president and congress are just as sworn to upheld it, and with respect to several portions of it (e.g. impeachment) are the only arbiters of what it means. With the rare exception of cases where the Supreme Court has original jurisdiction [1] - congress is even allowed to replace the Supreme Court with a new final court of appeals. The Supreme Court has surprisingly little constitutional power if Congress decides to sideline them.
[1] > In all Cases affecting Ambassadors, other public Ministers and Consuls, and those in which a State shall be Party, the supreme Court shall have original Jurisdiction
[2] (Emphasis added) > In all the other Cases before mentioned, the supreme Court shall have appellate Jurisdiction, both as to Law and Fact, with such Exceptions, and under such Regulations as the Congress shall make.
You must make the difference between the institution and the members. Just because Congress can change the makeup of the Court, doesn't mean that the institution has. It's a Catch 22 - in the end the final interpreter will always have absolute liberty over meaning.
And again, in theory Congress can make exceptions to it, but in the end its still up to the Supreme Court to decide that they make sense. A piece of paper has no meaning in and of itself.
Marbury vs Madison is a result of an interpretation of the constitution, and the constitution de facto grants that power to the highest court of appeal which happens to be the Supreme Court. As a result, the highest court of appeal essentially is the constitution.
