arp_ignore

general = { about, articles, links, projects }     meta = { date-posted: 2006-09-21 }

I ran into something interesting at work today.

Apparently it is default behavior for the Linux kernel to arp respond for any ip address it has assigned to it (any interface), out of any interface it receives the arp on.

What does this mean? Consider the following.

You have two interfaces, on different broadcast domains, and in different logical networks.

Here is an example..

Anyway. From host A, you would see the following. 

ARPING 10.0.0.3 from 10.0.0.2 eth0
Unicast reply from 10.0.0.3 [AA:AA:AA:AA:AA:AA]  0.941ms

So far so good.. Now try to arping 10.1.0.3 from host A, and send it out eth0. 

ARPING 10.5.5.5 from 10.0.0.2 eth0
Unicast reply from 10.5.5.5 [AA:AA:AA:AA:AA:AA]  0.941ms

Notice, that both arping responses came back from eth0 on host B. This means that if someone has firewall rules on that host, then eth0 will receive all the traffic, even for 10.5.5.5. This behavior can cause odd problems. Most of the time it is ok, but there are certain instances where it can be a problem.

If both of the NICs on host B are on the same broadcast domain, but still different logical networks, then one NIC will respond for all arps, for any addresses on eth0 or eth1 of host B. If you have specific rules on host B for traffic arriving through one of the NICs, then they will fail.

The same thing occurs if you have two NICs in the same broadcast domain and the same logical network. Only one NIC will respond for the ips on both NICs.

So, what is the resolution? arp_ignore

The ip-sysct contains an option for arp_ignore. This allows you to modify the arp response behavior.

From the linux kernel documentation for ip-sysctl: 

arp_ignore - INTEGER
    Define different modes for sending replies in response to
    received ARP requests that resolve local target IP addresses:
    0 - (default): reply for any local target IP address, configured
    on any interface
    1 - reply only if the target IP address is local address
    configured on the incoming interface
    2 - reply only if the target IP address is local address
    configured on the incoming interface and both with the
    sender's IP address are part from same subnet on this interface
    3 - do not reply for local addresses configured with scope host,
    only resolutions for global and link addresses are replied
    4-7 - reserved
    8 - do not reply for all local addresses

    The max value from conf/{all,interface}/arp_ignore is used
    when ARP request is received on the {interface}

To set it, either add the string "net.ipv4.conf.all.arp_ignore=2" to "/etc/sysctl.conf", or simply write that config manually. To get it to be persistent across reboots, go with sysctl or add the manual writing to a startup script.

I prefer to use the value 2, as apposed to value 1. Value 1 is a bit more permissive, and there are times when it would be the right choice. Most of the time though, 2 is a good choice.

So, what happens when we set arp_ignore to 2 in the above example? We don't get a response from the arping for the dummy interface ip, and you wouldn't get one from an address on a NIC on a different broadcast domain either.

This option was apparently added sometime in the 2.6 kernel, and backported to the 2.4 kernel (2.4.26?).

Later Addition:

You should also set the arp_announce value to 1 (net.ipv4.conf.all.arp_announce=1), so that an attempt is made, when sending arps out from an interface, to choose the best one for that interface.