How to load balance sftp instances on AWS

Solution 1:

My comment could probably use some clarification. I spouted off with the eloquence of an inebriated yak:

I have never wanted to set myself on fire so much as I do now.

Why? Why would I say such a thing? Mostly because I'm an awful person. However, aside from that, I can explain my outburst by going over the original post piecemeal:

I like to know is it possible to load balance sftp servers in AWS.

Yes. Impossible is nothing. But know that unless you get a special SFTP package, the load balancing will be entirely up to you to build. The service being SFTP and being hosted in AWS is inconsequential.

I have 2 servers, and each of my servers are using s3fs-fuse to mount the same S3 bucket onto a mount point. Both of my ec2 instances are able to read/write to their mount points, and from S3, I can see the files from both servers.

You're off to a good start with a shared file system, the performance and reliability of the setup notwithstanding.

As for my next step, I like to know how can I load balance my sftp servers, so that when a user connects to a specific IP address, it will redirect them to one of my sftp servers.

The question is now: Why do you want to load balance. There is a fantastic amount of throughput and processing power afforded to the Amazon instance catalog and the need to load balance SFTP would mean you're approaching porn levels of network activity. Keep it simple, repeatable, and resilient wherever possible. Get an i2.xlarge with an SFTP daemon running on it and you should be fine no matter what. Build it with Puppet/Chef/$trendy-config-management-tool and you're in business. Moving on however...

I took a look at elastic load balancers, but they seem to only permit specific ports. I have also investigated HAProxy, but I am unsure how secure that solution will be.

HAproxy is exactly the kind of tool you need. Your uncertainty about security is easily dispelled with just a few hours of reading. My desire to self immolate is rising from this point on. If you're unsure about something, go become sure about it. HAProxy is the choice for many financial institutions, hospitals, and governments.

I have to take HIPAA compliance into consideration.

Totally understood, but compliance is not primarily the role of tools. You'll need to understand the concepts behind the HIPAA compliance requirements, and see how HAproxy can fulfill them. HAProxy is neither HIPAA compliant nor HIPAA non-compliant. No matter which tool you use, you'll need to independently verify the underlying assumptions and requirements of your compliance and regulatory needs. In fact, if anything, S3 and the use of Amazon instances should be inspected more carefully than the use of HAproxy.

The load balancer must be a static ip address as our vendors does not support DNS hostnames

This. This did it. Your vendor is bad and should feel bad. Now I want to jump into lava. Not supporting something basic like DNS resolution is entirely unrelated, but also it's like saying "A car must have an engine for me to use it." Well of course. Of course a load balancer is going to have the ability to use a static IP address. There are many more considerations that you need to be thinking about above simple static IP addresses.

TL;DR

Yes you can load balance SFTP with HAproxy. HIPAA compliance is up to you to discern and tool choice will not check boxes. You have some Googling to do and documentation to read.

I have some flames to put out.