Distributing Static Routes with DHCP

I’m setting up an isolated network for people to test internal applications on, since the developers all have Sun workstations with a dual-port Gigabit NIC on the motherboard, and we’ve got a bunch of older network equipment that we haven’t gotten around to eBaying yet. What I’m doing is linking the second NICs together with some virtual machines and the older network equipment to create a separate development network.

The development network is a full Layer-3 network running an IGP between multiple nodes with attached client boxes. This allows me to play around with a decent lab network, and provides developers with a way to discover that Linux sets the TTL of multicast packets to “1” well before they are called to explain why their application didn’t work even after loads of testing, spend 8 hours playing head-desk, and finally start questioning me about firewalls on our internal network, forcing me to claw it out of them that they are driving multicast without a license and explain how to use tcpdump.

Not that I’ve had to do that a dozen times now, or anything…

This means I have to configure static routes on the developer workstations so they can access things in the lab outside their local subnet. You start off by configuring static routes in your distro’s chosen format (this is RHEL5 at work, so it’s /etc/sysconfig/network-scripts/route-ethX), and then you step it up a notch by writing scripts to distribute these files, then start using rgang or func, and start thinking about using your systems programming tool to distribute the routes. And then you smack your forehead and figure out that this is all stupid: there is already an IETF standard way to distribute network configuration which you should be using: DHCP.

There’s even DHCP option 121, which provides a way to distribute CIDR information (modern static routes) to clients. Unfortunately this standard option isn’t supported out of the box on modern dhclient or ISC dhcpd, so you need to configure it and script it in.

First, on the client, /etc/dhclient-exit-hooks

#!/bin/bash
#
# /etc/dhclient-exit-hooks
#
# This file is called from /sbin/dhclient-script after a DHCP run.
#

#
# parse_option_121:
# @argv: the array contents of DHCP option 121, separated by spaces.
# @returns: a colon-separated list of arguments to pass to /sbin/ip route
#
function parse_option_121() {
        result=""

        while [ $# -ne 0 ]; do
                mask=$1
                shift

                # Is the destination a multicast group?
                if [ $1 -ge 224 -a $1 -lt 240 ]; then
                        multicast=1
                else
                        multicast=0
                fi

                # Parse the arguments into a CIDR net/mask string
                if [ $mask -gt 24 ]; then
                        destination="$1.$2.$3.$4/$mask"
                        shift; shift; shift; shift
                elif [ $mask -gt 16 ]; then
                        destination="$1.$2.$3.0/$mask"
                        shift; shift; shift
                elif [ $mask -gt 8 ]; then
                        destination="$1.$2.0.0/$mask"
                        shift; shift
                else
                        destination="$1.0.0.0/$mask"
                        shift
                fi

                # Read the gateway
                gateway="$1.$2.$3.$4"
                shift; shift; shift; shift

                # Multicast routing on Linux
                #  - If you set a next-hop address for a multicast group, this breaks with Cisco switches
                #  - If you simply leave it link-local and attach it to an interface, it works fine.
                if [ $multicast -eq 1 ]; then
                        temp_result="$destination dev $interface"
                else
                        temp_result="$destination via $gateway dev $interface"
                fi

                if [ -n "$result" ]; then
                        result="$result:$temp_result"
                else
                        result="$temp_result"
                fi
        done

        echo "$result"
}

function modify_routes() {
        action=$1
        route_list="$2"

        IFS=:
        for route in $route_list; do
                unset IFS
                /sbin/ip route $action $route
                IFS=:
        done
        unset IFS
}

if [ "$reason" = "BOUND" -o "$reason" = "REBOOT" -o "$reason" = "REBIND" -o "$reason" = "RENEW" ]; then
        # Delete old routes, if they exist
        if [ -n "$old_classless_routes" ]; then
                modify_routes delete "$(parse_option_121 $old_classless_routes)"
        fi

        # Add new routes, if they exist...
        if [ -n "$new_classless_routes" ]; then
                modify_routes add "$(parse_option_121 $new_classless_routes)"
        fi
fi

We use /etc/dhclient-exit-hooks because the RHEL5 dhclient-script only calls the up-hooks script on BOUND and REBOOT, so if you change your static routes on the server, your client won’t pick them up until the box reboots or the interface is otherwise cycled.

The obvious problem here is that it’s always deleting the old routes and adding the new routes in two stages, a worthwhile enhancement for this script is to diff the old and new routes and determine which ones actually need to be removed/added.

So that will not do anything at first, because dhclient doesn’t actually read option 121 until you tell it to. For that, you need to edit /etc/dhclient.conf, and tell it how to handle option 121 in a way that the script above can understand:

#
# dhclient.conf
#

option classless-routes code 121 = array of unsigned integer 8;
request;

This tells dhclient to read all options, parse option 121 into an array of numeric bytes, and provide that array as a space-separated string as the new_classless_routes and old_classless_routes variables.

So now we’ve gotten all that taken care of, we need to start distributing routes from the DHCP server. For that, you need to update your /etc/dhcpd.conf file:

#
# dhcpd.conf
#

option classless-routes code 121 = array of unsigned integer 8;

subnet 10.23.1.0 netmask 255.255.255.0 {
        [...]
        # Routes for 10.23.0.0/16 via 10.23.1.1, and 224.0.0.0/4 (all IP multicast) via same
        option classless-routes 16,10,23,10,23,1,1,4,224,10,23,1,1
        [...]
}

You can also put that option into a host stanza if you’re doing that. Finally, as I’m using cobbler, I wanted to be able to have the new “static-routes” interface option end up in my cobbler-managed DHCPd configuration. Here’s a bit of my template that puts that configuration option into the appropriate DHCP option:

#
# /etc/cobbler/dhcp.template
#

[...]

#for dhcp_tag in $dhcp_tags.keys()
group {
        #for mac in $dhcp_tags[$dhcp_tag].keys():
                #set iface = $dhcp_tags[$dhcp_tag][$mac]
                #if $iface.dns_name
        host $iface.dns_name {
                hardware ethernet $mac;
                        #if $iface.ip_address
                fixed-address $iface.dns_name;
                        #else
                ddns-hostname "${iface.dns_name.split('.')[0]}";
                        #end if
                        #if $iface.static_routes:
                                #set val121=""
                                #for routespec in $iface.static_routes:
                                        #set gateway=$routespec.split(':')[1]
                                        #set destcidr=$routespec.split(':')[0]
                                        #set destnet=$destcidr.split('/')[0]
                                        #set destmask=$destcidr.split('/')[1]
                                        #
                                        #if val121
                                                #set val121=$val121 + ",$destmask"
                                        #else
                                                #set val121=$destmask
                                        #end if
                                        #
                                        #if int($destmask) > 24
                                                #set val121=$val121 + "," + $destnet.replace('.', ',')
                                        #else if int($destmask) > 16
                                                #set val121=$val121 + "," + $destnet.split('.')[0] + "," + $destnet.split('.')[1] + "," + $destnet.split('.')[2]
                                        #else if int($destmask) > 8
                                                #set val121=$val121 + "," + $destnet.split('.')[0] + "," + $destnet.split('.')[1]
                                        #else
                                                #set val121=$val121 + "," + $destnet.split('.')[0]
                                        #end if
                                        #
                                        #set val121=$val121 + "," + $gateway.replace('.', ',')
                                #end for

                option classless-routes $val121
                        #end if
        }
                #end if
        #end for
}

Obviously, there are likely bugs in this script, and I’m only using it on a couple of boxes in my lab network, so feel free to point out any issues in the comments and I’ll update the above accordingly.

Advertisements

13 Comments

  1. Jim Snyder says:

    Looks like dhclient-(enter|exit)-hooks moved to /etc/dhcp in FC13.

    I run my firewall from /etc/dhclient-*-hooks.

    Imagine my surprise.

    I gather this change originates in ISC’s new dhcp release:

    http://www.mail-archive.com/debian-bugs-rc@lists.debian.org/msg225247.html

    I haven’t looked closely, but I think this bug report is relevant to your post:

    https://bugzilla.redhat.com/show_bug.cgi?id=516325

  2. Jim Snyder says:

    Incidentally, /etc/sysconfig/network-scripts/ifup-post checks for /sbin/ifup-local for (one presumes) site-specific post-“up” local customizations.

    It doesn’t exist, but one could always roll one’s own in the same way that one rolls /etc/…dhclient-*-hooks.

    And in /sbin/ifup, there’s a similarly futile reference to /sbin/ifup-pre-local.

    Probably just me, but I’d rather have site-specific scripts in /etc/sysconfig/network-scripts, or at least *not* /sbin.

  3. Ben says:

    I’ve been looking for more info on issues with Cisco and setting the next hop address for multicast addresses but have been unable to track it down. Do you potentially have some links or a let me google that for you smack in the face?

  4. James Cape says:

    @Ben,

    Unfortunately I don’t have a reference for it other than the fact that all the multicast route examples online don’t set a next-hop address, just the interface.

    I did just re-confirm that it still fails with my laptop here on em1/eth0 vs. wlan0. Em1 is the default interface (wired), and I added a route to 224/4 via <gwaddr> dev wlan0—it still fails.

    Quit the app, delete the route and re-add with 224/4 dev wlan0 (no via), restart the app, it works fine.

    Fired up wireshark, and Linux is doing what I remembered it doing since RHEL5 (at least): firing the IGMP join at the group IP address, but the gateway’s MAC address. My guess is that IGMP snooping on the switch (which is enabled by default on Catalysts) is watching on the MAC layer and not the IP layer. Since my join is not on the right Ethernet address at all, it won’t get caught by the snooper process and my port would not be added to the switch’s MFIB (in this scenario it’s AP1200 -> 2960 -> 3750 SVI, but the theory holds regardless).

    If you don’t specify a gateway address, then Linux does the standard thing and bitshifts the group address to derive the multicast MAC.

  5. Ben says:

    So sounds like I really just need to verify the behavior of my devices when configuring the static routes. In this case they are xDSL modems running some flavor of Linux, but I would guess that the behavior will be as you describe there as well.

  6. James Cape says:

    @Ben,

    Assuming the code is still enabled* and the kernel(s) in question haven’t been patched, it should exhibit the same behavior—but it’s definitely worth watching it with a sniffer.

    * According to some old howtos, there is a kconfig option to enable/disable multicast, I dunno if that’s been removed or not.

  7. James (author),

    I just wanted to say thank you very much for this guide!

    I’ve been looking for a way to hand out static routes to VPS’s on a Xen node without having to add a start-up script to work the routes out from the ifconfig command or adding in route-eth0 files, which means i’d have to make a new set of images for each Xen server (unmanageable much?)

    So when I make our VPS images for the OS’s all I have to include is the change to dhclient.conf and the server the VPS is created on will tell the client VPS what static route to use…

    This sort of information is a little hard to come by, bookmarking under “gold dust”!!

    Kind regards,
    Kevin

    1. James Cape says:

      @Kevin,

      Thanks for the kudos, though this is a bit behind the most recent stuff in Fedora 16: the up-to-date versions of dhcp and dhclient shipping with Fedora 16 both support this out-of-the box with “option classless-static-routes”.

  8. Steve says:

    Gold here, thanks muchly. Stole these scripts and put on our CentOS 5 vm’s, and static routes via dhcp is working great.

  9. Alan Edmonds says:

    This was a very helpful post. I’m running CentOS 5.9 and found that /sbin/dhclient seems to call /etc/dhclient-exit-hooks for more methods than just BOUND and REBOOT; in fact, it seems to always call it. This has the effect of deleting and adding routes every time the lease renews. Not desired behaviour. As mentioned in the text, the script could be enhanced. Here’s my “enhancement” to check the routes for differences during RENEW:

    Original:
    if [ “$reason” = “BOUND” -o “$reason” = “REBOOT” -o “$reason” = “REBIND” -o “$reason” = “RENEW” ]; then

    New:
    if [ “$reason” = “BOUND” -o “$reason” = “REBOOT” -o “$reason” = “REBIND” -o \( “$reason” = “RENEW” -a “$old_classless_static_routes” != “$new_classless_static_routes” \) ]; then

    One other comment. In the example dhcpd.conf file, the option line needs a terminating semicolon.

    # Routes for 10.23.0.0/16 via 10.23.1.1, and 224.0.0.0/4 (all IP multicast) via same
    option classless-routes 16,10,23,10,23,1,1,4,224,10,23,1,1;

    I have found this format slightly easier (for me) to read

    option classless-routes
    16,10,23, 10,23,1,1,
    4,224, 10,23,1,1;

    Hope this is useful.

    Alan

Comments are closed.