"DNS propagation" is a really bad term that could conceivably cover two distinct things:
1. The "propagation" of fresh records from a domain's authoritative name servers to users' caching recursive name servers.
2. The "propagation" of updated zone information from a domain's master authoritative server to the domain's slave authoritative servers.
This tool attempts to check for #1, but no tool can actually check for this. For one thing, it's not really propagation; it's caching. If you query a caching name server and the name is not yet cached, you'll get fresh information immediately. If it is already cached, you'll get stale information until the cache entry expires. There are an untold number of caching name servers out there (you'll find them at every ISP, every company, and even in some home routers). If you look at the list of name servers this tool queries, you'll find only a small slice of the Internet's caching name servers[1]. If none of these servers have your name cached when you run the tool, they will all return fresh information, and the tool will report that your DNS update has "propagated" when in reality there are many caching name servers out there at other ISPs which still have stale information cached.
Fortunately, there's a better way to know when your change has "propagated" - look up the record's TTL, which is the maximum number of seconds it can remain cached. Until that amount of time elapses from your update, there are potentially name servers out there serving up stale information.
Now, a tool to check for #2 would actually be quite useful. Transfers from master to slave are supposed to happen immediately, and you shouldn't have to worry about this if your domain uses a good DNS service, but I've seen many misconfigured or over-complicated DNS setups where this doesn't happen reliably. A tool that queried all the authoritative servers for a domain to see which ones haven't updated would be quite useful for debugging these sorts of misconfigurations.
[1] Specifically, those which are misconfigured to allow queries from anyone. Wide open name servers like these can be used as part of DOS attacks. In a perfect world, the only caching name servers you'd be able to query would be the ones operated by your ISP, and public ones which have anti-DOS features, like Google Public DNS.
While you're correct that a 'worst-case-scenario' would be to judge total possible propagation time on the TTL of the record prior to it being modified, this doesn't hold true in a few cases:
1. Every DNS resolver caching it will have accessed and cached it at different times, not just when you modified it.
2. Your original TTL could've been 'too short' for some resolvers, and they substituted a larger value, which you don't know.
3. Your TTL could've be too long and it expired, or cache was reset/filled, or cache too small, and it was dropped from cache.
4. The operator of the resolver could've chosen arbitrary TTL to assign to all records.
So instead of guessing across these scenarios, it gives a 'sample' of how resolvers are handling your particular situation at a particular time, and gives some perspective into how other, non-reachable resolvers may also be handling things.
To one of your other points, keep in mind that many resolvers are stub resolvers, using an ISP's name resolver, and so these DNS servers queried here may be in use by much larger audiences than you imagine.
Its certainly not a guarantee of much information, but Imagine scenarios:
A. You see that 90% of the returned resolvers are giving you the old record.
B. You see that 90% of the returned resolvers are giving you the new record.
I'd consider that information of value in determining overall 'propagation' of the new record to resolvers.
Awesome tool Sam! You may want to consider having the code pull a gzip'd version of resolver-list.yml to speed things up. It's 240K uncompressed, but only ~49K compressed. And even perhaps having setup.py grab the resolver list at install instead of first run.
EDIT: You're right regarding Github serving it gzipped. You can disregard my comment on that; I was using wget for testing without specifying --header='Accept-Encoding: gzip'.
The HTTP library (requests) by default will pull a gzipped version of the resolvers list from Github (Content-Length returned for current version is 59029 compared to 239359 with gzip disabled). Compressing with `gzip -9` gives me a file size of 49648 so I don't think the added complexity of having to consciously deal with the compression in the application outweighs the small gain over the standard HTTP compression Github and Requests provide by default.
I had looked at pre-downloading the resolvers file in the setup script, unfortunately there doesn't seem to be a decent, reliable way to do it. If people download the source and run `setup.py install` it's easy but I'd imagine most people will just install with `pip` or `easy_install` which makes things a bit more complicated since nether of them seem to run post install actions.
Both good suggestions though, I'll keep them on my todo list.
I'm happy to submit a pull request if you'll accept them. Already have forked the repo.
EDIT: You're right regarding Github serving it gzipped. You can disregard my comment on that; I was using wget for testing without specifying --header='Accept-Encoding: gzip'.
This is a similar service, but will check your actual domain resolution as seen by hosts that never heard of the domain before: http://dns.squish.net/
It will not tell you anything about the caching servers of course. As agwa says - you're only guaranteed (+/- client issues) that the new entry will be visible after the full TTL period.
This looks pretty cool, waiting for DNS to propagate is a PITA and should help mitigate those "works for me" scenarios relating to out-of-date nameservers.
1. The "propagation" of fresh records from a domain's authoritative name servers to users' caching recursive name servers.
2. The "propagation" of updated zone information from a domain's master authoritative server to the domain's slave authoritative servers.
This tool attempts to check for #1, but no tool can actually check for this. For one thing, it's not really propagation; it's caching. If you query a caching name server and the name is not yet cached, you'll get fresh information immediately. If it is already cached, you'll get stale information until the cache entry expires. There are an untold number of caching name servers out there (you'll find them at every ISP, every company, and even in some home routers). If you look at the list of name servers this tool queries, you'll find only a small slice of the Internet's caching name servers[1]. If none of these servers have your name cached when you run the tool, they will all return fresh information, and the tool will report that your DNS update has "propagated" when in reality there are many caching name servers out there at other ISPs which still have stale information cached.
Fortunately, there's a better way to know when your change has "propagated" - look up the record's TTL, which is the maximum number of seconds it can remain cached. Until that amount of time elapses from your update, there are potentially name servers out there serving up stale information.
Now, a tool to check for #2 would actually be quite useful. Transfers from master to slave are supposed to happen immediately, and you shouldn't have to worry about this if your domain uses a good DNS service, but I've seen many misconfigured or over-complicated DNS setups where this doesn't happen reliably. A tool that queried all the authoritative servers for a domain to see which ones haven't updated would be quite useful for debugging these sorts of misconfigurations.
[1] Specifically, those which are misconfigured to allow queries from anyone. Wide open name servers like these can be used as part of DOS attacks. In a perfect world, the only caching name servers you'd be able to query would be the ones operated by your ISP, and public ones which have anti-DOS features, like Google Public DNS.