How can I setup a VPN for remote users to connect to a AWS RDS server?

A very simple solution would be to just use SSH tunnels to carry the SQL traffic. You don't mention what OS you're running on your EC2 instances, so I'll assume you're running Linux (if you're not running Linux, just spin up a t1.micro instance for this purpose. That will provide plenty of horsepower for this type of traffic). So - with Linux server in hand, getting this set up will be quite easy. Each developer will need an account on that server, and they'll need to generate a keypair for themselves and provide you the public key to deploy on their server accounts.

If they were using a unixy OS, they'd run a command similar to this:

$ ssh user@ec2-host -L3306:a.b.c.d:3306

...where "a.b.c.d" is the IP address of the RDS instance. You'll just need to make sure that each user has appropriate grants on the RDS database to connect from the ec2 host they're ssh'ing through.

After doing this, the developers will connect to their localhost, port 3306, and that traffic will be tunneled through to the RDS instance.

(I've never actually used RDS, but being that it's built to be a drop-in MySQL replacement, I feel my assumption is correct that is uses port 3306. If it uses a different port, then change the port number on the end of the above command)

Yeah, you're not going to be able to do this without a bit of mucking around. The easiest solution would be to NAT the VPN traffic when the gateway sends it to RDS, so that RDS knows to send the traffic back to your gateway before it files off the NAT and sends it back down the VPN. The other way that might work is using a VPC; I've never used it with RDS (or, to be honest, much at all), but as I understand the way that VPC networking works, it's possible that RDS hooks into your VPC and hence might be able to use a gateway. I think, though, that NAT will end up being the easiest option.