On 1/5/10 12:31 AM, Alexander Gall wrote:
[ I've already sent this to our colleagues at Swisscom, but this should really go to the list ]
On Mon, 04 Jan 2010 20:16:14 +0100, Benjamin Schlageterb.schlageter@ebm.ch said:
Hi Anybody knows something about big DNS troubles with Bluewin ADSL/VDSL? As I saw, my router can¹t resolve any domains... Lucky I got some other dns servers =)
The perils of DNSSEC<sigh>. It was entirely our fault. Due to an error, lots of DNSSEC-related resource records (NSEC3 to be precise) were missing in the ch zone file generated shortly after 7pm last evening. Unfortunately, the error went undetected and the truncated zone was published.
The DNS caches of Bluewin have DNSSEC validation enabled for the ch TLD and probably started to produce SERVFAIL for most subdomains at this point (depending on whether they were already in the cache or not). The zone was fixed some time after 9pm.
Sincere appologies for any pain this has caused.
Painful to say the least but thanks a lot for the honest and transparent report on what caused the issue.
Thomas