Disaster Recovery
Thursday 21st March, 2013 09:46 Comments: 0
For the last few years my Disaster Recovery (DR) server involved me periodically copying data across from the production server. It was a very manual process, which I did very irregularly (I'm lazy, and my production server has been pretty reliable, get over it). In the last few weeks I've made this an almost entirely automated process (using my existing SSL certificates), with everything properly configured, secured and penetration tested (go me!). Okay, so I haven't setup email yet, but that's not a difficult job.
Some people might call me crazy for not having done this from the very start, but I finally changed one of my original DNS servers into a secondary, to match the third DNS server (I originally only had two DNS servers and decided it was easier to tweak the entries on both servers than worry about enabling zone transfers, not that there's anything sensitive on them).
Last night the production server was taken offline so it could be put into a shiny new rack by my friend Chris, which meant I got to try out my DR "process" for real. I'd already changed the DNS entries (on the primary DNS server, the secondary servers did their job picking them up okay) to point at the DR server earlier in the day, and I changed them back first thing this morning (at some point today I'll firewall off the ports to the DR web server again). Everything went smoothly, with three (known and quite acceptable) exceptions:
The medium-to-long-term plan is to move to Windows Server 2012 and rely on Hyper-V 3.0 virtualisation (and possibly modify my server/site to use something like active-passive replication) and migrate the VMs between a couple of physical servers (I was thinking of hosting both in LDEX, but perhaps it'd made sense to have them in different racks/locations). When it's done, it'll be quite an overkill setup just to host this blog and my personal email (which I download to my home PCs, so nothing's on here for longer than 24 hours). I probably should do something more useful with it, but it looks like someone already beat me to scanning the Internet.
Some people might call me crazy for not having done this from the very start, but I finally changed one of my original DNS servers into a secondary, to match the third DNS server (I originally only had two DNS servers and decided it was easier to tweak the entries on both servers than worry about enabling zone transfers, not that there's anything sensitive on them).
Last night the production server was taken offline so it could be put into a shiny new rack by my friend Chris, which meant I got to try out my DR "process" for real. I'd already changed the DNS entries (on the primary DNS server, the secondary servers did their job picking them up okay) to point at the DR server earlier in the day, and I changed them back first thing this morning (at some point today I'll firewall off the ports to the DR web server again). Everything went smoothly, with three (known and quite acceptable) exceptions:
- My email went down overnight for a couple hours because I don't have a second mailserver (yet).
- My webmail showed an error message because I hadn't setup a DR mail server (see above).
- Logins were temporarily disabled because I couldn't be bothered to manually move the database back to the production server (I'm not using clusters/replication yet).
The medium-to-long-term plan is to move to Windows Server 2012 and rely on Hyper-V 3.0 virtualisation (and possibly modify my server/site to use something like active-passive replication) and migrate the VMs between a couple of physical servers (I was thinking of hosting both in LDEX, but perhaps it'd made sense to have them in different racks/locations). When it's done, it'll be quite an overkill setup just to host this blog and my personal email (which I download to my home PCs, so nothing's on here for longer than 24 hours). I probably should do something more useful with it, but it looks like someone already beat me to scanning the Internet.