In the last post, I explained how to setup an anycast address on a loopback interface, and add a static route to the loopback.
Now, it’s entirely possible that you’d like to stop working after the last entry. If this is a publicly facing service, I’d actually recommend you stop there, as a mitigation method against DOS attacks: if one of your anycast servers is taken out (lets say it’s the closest server to Comcast), your router will continue to send traffic to the now-dead service, which would end up preventing the nearby attackers from shifting their attack to the other anycast servers. In other words, a DOS attack could shut down one class of users, while leaving everyone else unaffected—or end up splitting the attack amongst a lot of different servers.
If you did fail over, then you’d setup a cascade failure similar to the one that took out the electrical grid: one server dies, the and the traffic is shifted to the next-closest sever, which dies, and shifts the traffic over to the third server, and so on.
However, if this is an internal service, you probably do want to fail over: when one server is taken down for maintenance or whatever, users are automatically redirected via the network itself to the next-closest server, and if that server is offline for whatever reason, they should connect to the next-closest server, and so on.
In order to do that, you have to figure out a way to have your routing table influenced in some way by your application state.
There are two methods to accomplish this. The first one is to run a routing daemon that can communicate using one of the IGP protocols on your server. So you have something like Quagga running on your server, and it talks to your router, and exchanges routes. You then write a little monitoring script which watches your daemon, and tells quagga to stop announcing a route to your anycast address, or simply shut down quagga, when the daemon goes away.
Meanwhile, your network engineer is off to the side, breathing into a paper bag, because when quagga blows up, he will be the one that has to put your network back together. So, lets look at a slightly less crazy way of controlling routes—one that doesn’t depend on your server behaving in order to handle the case when your server isn’t behaving 🙂
The less crazy way to do this is to use a pair of features that are present in Cisco routers: IP SLA and route tracking. For this I’ll assume you have a pair of servers attached to separate WS-C3750 router/switches (a fairly common 48 port switch which also has IPv4 routing hardware build in) in two different sites, attached by a 10Mbps Ethernet line:
So, let’s assume you’ve already got your static routes to the anycast address, and OSPF setup between your two routers:
ip route 10.10.10.10 255.255.255.255 10.30.30.30 name dns.example.org ! router ospf 10 log-adjacency-changes router-id 10.30.30.0 passive-interface default no passive-interface GigabitEthernet1/0/52 network 10.30.30.0 0.0.0.255 area 0
ip route 10.10.10.10 255.255.255.255 10.20.20.20 name dns.example.org ! router ospf 10 log-adjacency-changes router-id 10.20.20.0 passive-interface default no passive-interface GigabitEthernet1/0/52 network 10.20.20.0 0.0.0.255 area 0
The first thing to do after that is setup redistribution, so each site knows about the other site’s route to the DNS server:
ip access-list standard static-to-ospf-list permit 10.10.10.10 ! route-map static-to-ospf-rmap 10 match ip address static-to-ospf-list ! router ospf 10 log-adjacency-changes router-id 10.30.30.0 redistribute static subnets metric-type 1 route-map static-to-ospf-rmap passive-interface default no passive-interface GigabitEthernet1/0/52 network 10.30.30.0 0.0.0.255 area 0
ip access-list standard static-to-ospf-list permit 10.10.10.10 ! route-map static-to-ospf-rmap 10 match ip address static-to-ospf-list ! router ospf 10 log-adjacency-changes router-id 10.20.20.0 redistribute static subnets metric-type 1 route-map static-to-ospf-rmap passive-interface default no passive-interface GigabitEthernet1/0/52 network 10.20.20.0 0.0.0.255 area 0
The next step is to setup IP SLA on each router so it tests DNS:
ip sla 10 dns example.org name-server 10.30.30.30 frequency 3 ip sla schedule 10 life forever start-time now
ip sla 10 dns example.org name-server 10.20.20.20 frequency 3 ip sla schedule 10 life forever start-time now
So now you’ve got the router trying to perform a DNS lookup for
example.org every 3 seconds at the regular IP address of the server. The last step is to configure the route tables to hook into the SLA monitors:
track 10 ip sla 10 ! ip route 10.10.10.10 255.255.255.255 10.20.20.20 name dns.example.org track 10
track 10 ip sla 10 ! ip route 10.10.10.10 255.255.255.255 10.30.30.30 name dns.example.org track 10
And that’s it. You’ve now configured your routers to watch the DNS server running on each physical interface, and remove the static route to your anycast address from your route tables if it can’t properly query the address record for
example.org. If it does that, the network will automatically shift the traffic to the anycast IP on the other server.
In my next post, I’ll cover some of the finer points of when and where you may want to use Anycast services:
- Anycast: Networking Introduction
- Anycast: The Loophole
- Anycast: The Interface
- Anycast: Handling Routes
- Anycast: DGRAM vs. STREAM
- Anycast: IP-SLA HOWTO