Redundant Ethernet links from a Linux box
This could be an RTFM kind of question. If so, I'll gladly award the answer to whoever gives a link to TFM :)
In my racks, I've got redundancy all over the place. Two sets of switches, two upstream routers, two power paths, dual power supplies in the servers... you know the drill. However, now I'm wondering about dual paths from the servers. Each has two (or more) NICs. What we've done in the past was use LAGs (Cisco-speak: Etherchannel) to provide two links to the same switch. Actually, it's to two switches in a stack (so, one management interface, but two physical boxes). That means that we can survive failure of a cable, or of an interface (on the switch or on the box), or maybe even failure of a switch (if the stacking doesn't drag the other one with it).
Why not keep doing that? We're getting to the point where we need more than one switch stack. This means that, in a failure of a switch stack, we'll cut our reachable machines in half.
Assume that the switch stacks have cross connections with sufficient bandwidth to each other, to the routers, etc. What I'd like is something like Spanning Tree Protocol: run a line from each server to each switch, it picks one to use. If that link goes down, swap to the other one. I'm hoping that this will let us have a whole switch stack fail without problems (after everyone notices to switch over).
If it matters I'm running CentOS. Something that isn't switch specific is ideal, but I'd like to hear about solutions from vendors as well.
Solution 1:
Basically, you want to bond the NICs in the machine with mode=1, active-backup. This will help you set it up http://www.howtoforge.com/network_card_bonding_centos
Solution 2:
Take a look at the bonding driver. It has an active/passive mode that can do what you're describing. It shouldn't require any specific switch support. Here's an old document, but I think it should still be relevant:
http://www.kernel.org/pub/linux/kernel/people/marcelo/linux-2.4/Documentation/networking/bonding.txt