In Rock Band/Guitar Hero, what's the best approach to hand postion on the guitar for hard (and higher) difficulties?
The first tip I have is a bit cheap -- if you have a Rock Band Guitar controller, then you can use the higher frets (the ones closer to the guitar body), which are much closer together than the lower frets.
As far as hand positioning, this is a common issue that happens all the time when playing actual stringed instruments. Musicians on violin, for example, have some terminology associated with different positions of the hand.
First position is the natural resting position for your hand, where your 1st finger (the pointer finger) is on the 1st fret. Playing the first three frets is very easy, while playing the 4th fret (pinky position) is slightly harder and playing the 5th fret by stretching the pinky finger is very hard.
Second position is where your 1st finger is on the 2nd fret, with your pinky on the fifth fret. Playing frets 2 through 4 are easy, the 5th fret is a little hard, and playing the 1st fret by stretching your pointer finger backwards is very hard.
Third position is where your first finger is on the 3rd fret. Playing the 1st fret is impossible, the 2nd fret is difficult, and the rest are very easy.
In high level play, you have to learn how to shift between positions. You cannot stay in first position forever; stretching your pinky to hit the 5th fret is meant to be a stopgap solution and will wear your hand out on serious songs. Eventually with practice, after you are comfortable with shifting, it will become much easier than stretching you fingers.
For example, certain patterns are very obviously meant to be played in one position than any other. A fast 5-4-3-2 pattern can only be played in second position, and similarly a 4-3-2-1 can only really be played in first position. The game will frequently do things like make you play 5-4-3-2 4-3-2-1, forcing you to shift between second and first position quickly. This is why you have to learn how to shift.
Also note that shifting positions incurs a certain cost - it takes time to move your hand, you might miss the proper position while you are doing it, it takes brain power to remember which one you're in, etc. You have to weigh the advantages of using the best position with shifting too often.
So, returning to your original question: the correct position that you should be in depends highly on the song and the phrase you are playing. If the song most takes place in the first 4 frets and has an occasional 5th fret (like some medium songs), then it is not worth switching to another position. If it takes place mainly on the top 4 frets, then shift to second position. If it's a fast succession of two notes like frets 4 and 5 (a trill), switch to third position so that your pinky won't have to move too quickly.
I always move all the four fingers, since in most songs you'll get a stream of notes on that side.
Off course, when it's only for a couple of notes you should only move your index finger or pink to get the one/two notes at the edges.
However, this might vary between songs, but luckily you can practice what the best strategy is for each song.