This distinction is a technicality, given the The Hague Invasion Act signed into law by G.W. Bush. Or to use your example, would a hypothetical attack on the UN building in New York not also be a violation of American sovereignty?
Call me when we’ve invaded The Hague or when someone else invades New York. An international institution that claims members from Asia, Africa and across the Americas doesn’t become a European institution just because America left the room. Japan is actually the largest national contributor to the ICC’s budget.