Why does turn order shift?

Turn order that is listed is based on no further actions being taken. No movement, no using items, no attacking, no using skills. Each time you perform an action that will delay your next action, it will cause an appropriate shift to the order at the top right. Thus you can see where you'll end up as you go, decide "Hey, I can just end here and I'll still go before the enemy again". At least, unless you use a skill, then you won't know ahead of time. Each shift only is shifting the character whose turn it is, and won't affect where any other turns are in relation to each other.

Enemies follow the same logic - they appear to shift a lot more often because they both have weird delay counts and also because each individual attack they make counts as a separate action, and they make a lot of attacks.

I'm not positive on the exact calculation used for turns past the upcoming turn, but at the start of that character's turn, turn delay shifts back to "nothing has happened". This is usually less than what was predicted from earlier, and so turn order will shift even before you do nothing. This generally renders any turn action past the upcoming turn to be extremely inaccurate and so you should only plan in accordance to each character and enemy's immediately upcoming turn.