Load Balancing a UDP server
I have a udp server, it is a central part in my business process. in order to handle the loads I'm expecting in the production environment I'll probably need 2 or 3 instances of the server. The server is almost entirely stateless, it mostly collects data, and the layer above it knows how to handle the minimal amount of stale data that can arise from the the multiple server instances.
My question is, how can I implement load balancing between the servers? I would prefer to distribute the requests as evenly as possible between the servers. I would also would like to have some fidelity, I mean if client X was routed to server y, then I want all of X's subsequent requests to go to server Y, as long as it is sensible and not overloads Y.
By the way it is a .NET system... what would you recommend?
the state is internal within the servers, not a transaction of some sort. the state is some data the servers aggregate from the data they receive, and is quaryable with a simple WCF WebService. The application is based on UDP, and though i dont agree with the decision, its "Above my pay grade"
I'm currently trying out MS's NLB, it works ok, it does the fidelity thing out of the box, but it generates noise on the entire network...
Also no DNS... Oh and it's a completely costume protocol.
Solution 1:
I have a udp server, [...] server is almost entirely stateless [..] have some fidelity, I mean if client X was routed to server y, then I want all of X's subsequent requests to go to server Y, as long as it is sensible and not overloads Y.
So, you're using an undisclosed application protocol that keeps some application state, and runs on top of UDP? You're kind of going in a difficult direction. UDP is not a reliable data transport, that's the whole point of it -- for reliable data transport see its popular friend TCP. The only way you can get your 'fidelity' is by having a load balancing proxy which understands your application layer protocol, and which knows what your current application state is and can act accordingly.
I see 3 approaches that come close to providing what you seek:
Statically spread incoming connections out over 3 IP addresses, based on source (end user) IP address. This way a given user will always be directed to the same server. Most professional firewalls can do this for you. You may have to make the 3 servers highly available yourself, as most firewalls won't do backend health checks for you.
Use DNS, and use DNS Round Robin, as already suggested by Matt Simmons.
Use Windows's built-in Network Load Balancing (NLB). Honestly, I don't know how the failover scenario would play out with NLB and your semi-stateful UDP based service -- you would have to investigate that yourself, based on how your application handles state. On the plus side, NLB is very easy to set up, free of charge with the Windows license, mature, and well performing.
Solution 2:
Linux Virtual Server is a highly scalable and highly available server built on a cluster of real servers. LVS supported UDP protocol and source hash algorithm(this is used when you want a client to always appear on the same realserver).
I use LVM to balancing DNS(rr),SIP(sh).
Solution 3:
Interesting. Most of the proxy software I've seen is admittedly TCP based.
Most of the UDP-specific load balancing that I've seen in my meager experience has been DNS-based (ie: time servers, DNS servers, etc). Is there any way to provide multiple A records? If that would work, the normal DNS Round Robin would ensure fair distribution of requests (probably fair enough, anyway) and client caching would ensure that the fidelity was retained (assuming you're using a cache-based platform on the client end).
Solution 4:
You can use any kind of load balancer to perform this, either hardware or software, you can choose from different load balancers based on what you need.
Level 3 load balancer: Will load balance only by looking at the incoming IP and the available backend IPs, this kind of load balancer will ensure stickiness by always sending the same incoming IP address to the same backend, although this kind of strategy can overload one of the backends if lots of clients com from the same IP (be it a proxy or a corporate gateway)
Level 7 load balancer: A level 7 load balancer will not only balance as a level 3 balancer but it'll also look at the content of the package, which will give you a lot more flexibility for your balancing policies.
Considering that you're using UDP both balancers should give a good performance, also deep packet inspection in UDP is a bit more limited than on TCP (just due to protocol reasons).
Depending on your budget you can start by using a software load balancer (linux + IPVS for example) and then start going up to hardware load balancers like the ones offered by Cisco or Netapp
Solution 5:
The open source NGINX and the application delivery platform, NGINX Plus now supporting UDP load balancing. The new capability builds on our existing TCP and HTTP capabilities, making NGINX a powerful, easy-to-use, and consistent frontend for an even wider range of Internet applications and devices.
Available in the release nginx-1.9.13