This is very cool! I love the idea of having a structured way to incorporate tags and roles into DNS aliases.
However, since DNS is just about the most critical infrastructure service you can have, I want to point out that most of this can be accomplished nowadays with Route 53 internal DNS by syncing tags and roles into CNAMEs on Route 53.
The difference to me is that even if your syncing daemon goes down, your DNS is still up (it's hard to imagine a situation where Route 53 is down without something even more fundamental being down on EC2).
We push DNS into Route 53 at Runscope, and I trust our approach for the reasons you mentioned. Route 53 has a great track record for DNS uptime. If our sync system breaks, Route 53 will keep serving DNS records while we fix the sync system.
On a side note, we actually have two DNS sync systems at Runscope. One handles DNS for individual hosts (like web01.runscope.com) and the other handles public DNS (like www.runscope.com).
Our public sync daemon is tied into ZooKeeper, which keeps a list of active hosts for each of our public services. The daemon updates entries in Route 53 whenever the hosts change in ZooKeeper. We have 3 different types of public domains, and the tool handles all 3. We have standard domains with load balancing, geo-routed domains for services in multiple regions around the world, and regional domains for routing to a specific region. The sync daemon has been very helpful in keeping our infrastructure nimble.
If this interests you, I'll be giving a talk about our cloud infrastructure in a few weeks in San Francisco, along with our founder/CEO John Sheehan. Full announcement coming soon...
How do you authorize the right to tag instances? It seems risky to automatically map an instance into DNS based on a tag, unless you have strict access control over who can tag what.
To expand, suppose the "app" tag adds an app server into a DNS load-balanced cluster. Then anyone who can tag a server with "app" can start receiving requests from end users; including passwords, PII, etc.
IAM permissions can be filtered, for example: allow these actions, but only on a given resource. You can use ARNs (Amazon's globally unique identifier for a resource in any one of their services, but structured to allow wildcards) to limit the scope.
It's an interesting attack angle, but since it's based on tagged instances in their account, you'd have to launch a box in their account. If those credentials you have to launch boxes don't include credentials to make changes to security groups, VPC settings, etc... maybe you can't even get those logs and passwords off the host except for copy/pasting. There's a bunch of other hinderances I could think of (internal SSL usage where client certificates are verified, but said certificates aren't part of any of their base AMIs so your host never gets actual traffic handed to it, etc)
So getting somebody's AWS credential is a lot easier than it sounds.
The obstacles you mentioned are generally granted to any IAM User which has ec2 RunInstances privilege:
* Can associate an instance with a security group
* Can put an instance in a VPC
* Can reach the internet from an instance
The security group and VPC settings don't need to be modified, the box just needs to be placed appropriately. EC2 Security Groups in a VPC can restrict outbound traffic, but EC2 "classic" Security Groups only restrict inbound traffic.
To me this seems like a lot of holes to have to close in order to gain the convenience of using tags for DNS.
With a forward zone in bind with a long TTL, an unreliable DNS server shouldn't be an issue, as long as it's sometimes available. And you can have multiple forwarders, as well.
However, since DNS is just about the most critical infrastructure service you can have, I want to point out that most of this can be accomplished nowadays with Route 53 internal DNS by syncing tags and roles into CNAMEs on Route 53.
The difference to me is that even if your syncing daemon goes down, your DNS is still up (it's hard to imagine a situation where Route 53 is down without something even more fundamental being down on EC2).