Revoking a CSCA especially towards the end of the lifetime (passports are usually valid for 5 to 10 years) is probably not economically possible since it would invalidate millions of passports all at once so you have to fall back to optical features anyway
It's actually done this way.
Back in 2014 there were basically two ways to get the public certs.
Donwload them from each government site individually or use the only PKI infrastructure available for this task which was pretty expensive and did not have all certs either.
And as far as I recall signatures were only checked for being signed by a valid key not if the key matches the country of origin. So someone in country A could sign a forges passport from country B (but not 100% sure on that anymore)
If the biographic details were cryptographically signed, you'd need only a public key for each passport-issuing country.
I'm not sure how you'd deal with key revocation in situations like this.