Adjust the other end of a visual selection
I'm not sure I'm actually understanding what you want, but perhaps it is this:
:help v_o
Basically, while text is visually highlighted, you can press o and it will jump to the other "end" of the visual region without modifying it.