Is there any ability to set my primary monitor?

Turn off the current Primary monitor in "Displays" application and apply.

The secondary monitor will become the Primary. Then simply turn the old primary monitor again to have both displays active again.

The only correct answer here is unfortunately a buried comment:

I've reported this at https://bugs.launchpad.net/unity/+bug/1425000 – mmalmeida

It seems that the primary one (at least in 14.04) is always the one closest to the top left. So, there's no way to make the "right" display the primary one. Editing ~/.config/monitors.xml makes no difference.

(As this question pre-dates 14.04, I'm guessing this has been an issue for a while now)