AWS says "We are investigating elevated errors resolving the DNS hostnames used to access some AWS services in the US-EAST-1 Region." Is that coincidental, or are they being DDoSed also?
Apparently us-east-1 is backed by Dyn (and only Dyn) as well?
$ host -t NS us-east-1.amazonaws.com
us-east-1.amazonaws.com name server ns3.p31.dynect.net.
us-east-1.amazonaws.com name server ns1.p31.dynect.net.
us-east-1.amazonaws.com name server ns2.p31.dynect.net.
us-east-1.amazonaws.com name server ns4.p31.dynect.net.
That's… utterly bizarre to me. us-east-2 has a more diverse selection:
$ host -t NS us-east-2.amazonaws.com
us-east-2.amazonaws.com name server u4.amazonaws.com.
us-east-2.amazonaws.com name server u6.amazonaws.com.
us-east-2.amazonaws.com name server u3.amazonaws.com.
us-east-2.amazonaws.com name server u2.amazonaws.com.
us-east-2.amazonaws.com name server u1.amazonaws.com.
us-east-2.amazonaws.com name server u5.amazonaws.com.
us-east-2.amazonaws.com name server ns2.p31.dynect.net.
us-east-2.amazonaws.com name server ns1.p31.dynect.net.
us-east-2.amazonaws.com name server pdns1.ultradns.net.
us-east-2.amazonaws.com name server pdns5.ultradns.info.
us-east-2.amazonaws.com name server ns3.p31.dynect.net.
us-east-2.amazonaws.com name server ns4.p31.dynect.net.
us-east-2.amazonaws.com name server pdns3.ultradns.org.
Not that anyone should be running a service whose availability they care about solely in us-east-1 anyway…
$ host -t NS us-east-1.amazonaws.com
us-east-1.amazonaws.com name server pdns5.ultradns.info.
us-east-1.amazonaws.com name server ns3.p31.dynect.net.
us-east-1.amazonaws.com name server pdns1.ultradns.net.
us-east-1.amazonaws.com name server pdns3.ultradns.org.
us-east-1.amazonaws.com name server ns4.p31.dynect.net.
us-east-1.amazonaws.com name server ns1.p31.dynect.net.
us-east-1.amazonaws.com name server ns2.p31.dynect.net.
us-east-1.amazonaws.com name server u1.amazonaws.com.
us-east-1.amazonaws.com name server u2.amazonaws.com.
us-east-1.amazonaws.com name server u3.amazonaws.com.
us-east-1.amazonaws.com name server u4.amazonaws.com.
us-east-1.amazonaws.com name server u5.amazonaws.com.
us-east-1.amazonaws.com name server u6.amazonaws.com.
us-east-1 is the oldest region and predates Route 53. Not adding extra DNS providers to the older regions is probably an oversight.
(The EC2 API team requests load balancers from a separate load balancer team. The load balancer team probably didn't exist as a separate team when some of these regions were created.)
If that were the reason I wouldn't expect this update:
6:36 AM PDT [RESOLVED] Between 4:31 AM and 6:10 AM PDT, we experienced errors resolving the DNS hostnames used to access some AWS services in the US-EAST-1 Region. During the issue, customers may have experienced failures indicating "hostname unknown" or "unknown host exception" when attempting to resolve the hostnames for AWS services and EC2 instances. This issue has been resolved and the service is operating normally.
That might explain why we are down - most of our EC2 instances are in us-east-1. Looks like Amazon SQS is impacted too. We are getting a stream of undeliverable messages, and our 'dead letter' queue is filling up!
