Loadbalancing outbound traffic while using openbgpd on freebsd

Solution 1:

I've not used openbgpd in years, so, I'll give you some theory which you can apply to your configuration.

A provider can send you routes. Typically they will send 'default', 'customers', or 'full'. A default route is just that, 0.0.0.0/0 is announced to you and you know to reach 0.0.0.0/0 (any IP) through them. Your situation makes it sound like both providers are handing you a default route, and most likely, the one with the lower ASN is getting all of the traffic (ASN being the tiebreaker when a packet can go to multiple destinations and the AS paths are equal length, etc.).

A 'customers' feed would have them announce every route that is directly connected to them. If you have a provider like Level3, you would likely see half the internet through this feed. If your other provider has a lower ASN, they would get any traffic that was not directly connected to the peer announcing their customer routes to you.

A 'full' feed will have route announcements for every site that a provider knows how to get to - not just those that are directly connected.

You should be able to do something like:

show ip bgp summary (IOS command)

which should list the number of prefixes received somewhere.

If both of your existing providers are sending you a default route, then your connection is currently set up for failover. Provider A goes down, you would then route to Provider B. If you take customer feeds from both along with the default route, traffic would flow to the provider that has the shortest ASpath to reach a given address, and would fall back to the default route if it couldn't find the address. If you got a full feed from both, depending on the overlap between the providers, you would send traffic out the 'best' path to each.

On one of our border routers, I show 310540 active routes, make sure you have enough RAM and have configured the machine to handle the routes from both providers if you take full routes. You might experiment with taking customer routes from both until you get more comfortable. Or take customer routes from the smaller provider of the two and see how things change.

This still doesn't 'load balance' or split the traffic evenly and depending on your traffic. You might find that one provider still gets the lion's share of traffic. At this point, you would need to do some prefix/route maps to adjust traffic to prefer one peer or the other.

Once you set up route maps or an ASN match to shift your traffic, if the destination site goes down, and their route is removed from that provider, your router would still send the traffic out the other connection. BGP is somewhat self-healing.

What we've done is we have a few maps defined that match based on the ASN.

PROVIDER_A_TO PROVIDER_A_THROUGH PROVIDER_B_TO PROVIDER_B_THROUGH

Juniper makes this easier than using communities, though, you can do either.

Our prefix lists see if the destination is a particular ASN, and if it is, it goes through the _TO map. If the ASN is listed in _THROUGH, it'll prefer that provider. Since one of our providers has a direct connection to AT&T, PROVIDER_B_THROUGH matches on 7018|7019 (and a few other ASNs) so that AT&T traffic defaults to Provider B. Even though AT&T is reachable from both, we prefer to send it though Provider B.

I'm sure openbgpd has similar methods.

Solution 2:

BGP is not a load balancing protocol.

Let me repeat that, BGP is NOT a load balancing protocol.

If ALL traffic goes out a single connection, my first guess is that you aren't getting a full feed from the provider that is being ignored. Routing is such that precedence is given to more specific routes, so if you have a route of 192.168.0.0/23 going to provider a, and two routes of 192.168.0.0/24 and 192.168.1.0/24 going to provider b, provider a will be completely ignored. I'm not familiar with OpenBGP. So I can't tell you the specifics of how to verify some things without looking. But the first thing you need to do is validate that BOTH providers are giving you a full feed and not just a default route.

If that is the case, then you need to look into route preference for outbound traffic and path padding for inbound traffic. Make sure to wait twenty four to forty eight hours between configuration changes just to ensure you have a large enough data set to determine the effectiveness of your changes.

Solution 3:

Build a bridge. Freebsd will actually load balance on the upstream features of current prefixes and will load balance quite well. BGP will not. You can install both upstream ISP and announce to both ISP per your /22 and the will likely reject your /30 wan prefixes but iBGP will pick that up. iBGP will run well on the bridge and then no real need for OSPF. You now have a loadbalancer and fully redundant BGP setup. We have several up and love them on gig links to the Internet.