Doing a mysql dump causes swapping issues
We dump much larger MySQL databases nightly without any swapping issues. Here is the command line that is executed:
mysqldump --host=$HOST -u $USER --password=$PASSWORD --max_allowed_packet=512M --port=3306 --single-transaction --skip-add-locks --quick -e databasename
Are you including the "--quick" option? This prevents mysqldump from retrieving large tables in a single query. This could be what is forcing your server to swap. It makes mysqldump get the large tables row by row.
If you leave out the --quick option for mysqldump, the server will buffer the entire response before sending it to the client, which could easily cause swapping with large tables.
Also, if you save the dump to the host running the server it may fight mysqld for cache. Try using --quick and running mysqldump over the network, if you're not doing these two things already.
Good luck!