You know, sometimes I really feel like giving myself a good beating up. Since yesterday morning most of my sites have been basically flaking out due to one of the servers being down. I had dns hosted on that server so it affected not just sites hosted on that server, but also dns for all my other servers.

Firstly the main server was hosted at Layered technologies. The powersupply died. Unfortunately they took about 12 hours to fix it and boot up the server after they received the support ticket. That is just not on. My main ISP EasySpeedy provides fantastic support. I can not imagine this ever happening there.

Ok, at this point you might ask your self if you know anything about dns, why didn’t his secondaries kick in? Well I asked myself that as well and quickly ignored the question. Bad move. What I did do was to setup dns servers on all my other servers and change the nameservers for my domains to include these. This gave me flaky dns rather than fully working dns. Meaning a domain would work and then an hour later it wouldn’t.

This morning I checked and the failed server was up but I still had problems accessing my other servers via dns. After a long time playing around with my 2 new dns servers I couldn’t find anything wrong there. In the end it struck me that maybe it was my original secondary server (which I should have realized yesterday). Basically the firewall on that server was blocking incoming dns requests and it looks like it has done so ever since I first installed it. Duh!

This particular server runs Suse linux and I had configured the firewall via Yast. Unfortunately this doesn’t give you an option for UDP ports (like dns port 53). So I had to dig a bit. It turns out you edit your /etc/sysconfig/SuSEfirewall2 file and set the following settings like this:

FW_SERVICES_EXT_UDP="53"
FW_ALLOW_INCOMING_HIGHPORTS_TCP="DNS"
FW_SERVICE_DNS="yes"

After this run as root:

/etc/init.d/SuSEfirewall2_setup restart

Well all should be well now I hope.