If your recursive DNS servers are Unbound DNS, they will keep note of which infra (upstream forwarder) nodes are responding and will blacklist the ones that do not. You can see this with

    unbound-control dump_infra

After the infra ttl expires, they will probe the down nodes. If they respond, then traffic will be distributed among them again.