Right, if your scripts can screw up, after the CA issued a certificate, their costs are locked in, (and thus Let's Encrypt rate limits apply here) and you should make effort to be able to recover the key and certificate if you fail rather than start over.

The private key only you have, so that's the thing you most need to avoid throwing away over and over due to a bug. If you lose the certificate, that's a public document, you can just get another copy manually if necessary.