Pandas: Shift down values by one row within a group
Newer versions of pandas can now perform a shift
on a group:
df['B_shifted'] = df.groupby(['A'])['B'].shift(1)
Note that when shifting down, it's the first row that has NaN.
Shift works on the output of the groupby clause:
>>> df = pandas.DataFrame(numpy.random.randint(1,3, (10,5)), columns=['a','b','c','d','e'])
>>> df
a b c d e
0 2 1 2 1 1
1 2 1 1 1 1
2 1 2 2 1 2
3 1 2 1 1 2
4 2 2 1 1 2
5 2 2 2 2 1
6 2 2 1 1 1
7 2 2 2 1 1
8 2 2 2 2 1
9 2 2 2 2 1
for k, v in df.groupby('a'):
print k
print 'normal'
print v
print 'shifted'
print v.shift(1)
1
normal
a b c d e
2 1 2 2 1 2
3 1 2 1 1 2
shifted
a b c d e
2 NaN NaN NaN NaN NaN
3 1 2 2 1 2
2
normal
a b c d e
0 2 1 2 1 1
1 2 1 1 1 1
4 2 2 1 1 2
5 2 2 2 2 1
6 2 2 1 1 1
7 2 2 2 1 1
8 2 2 2 2 1
9 2 2 2 2 1
shifted
a b c d e
0 NaN NaN NaN NaN NaN
1 2 1 2 1 1
4 2 1 1 1 1
5 2 2 1 1 2
6 2 2 2 2 1
7 2 2 1 1 1
8 2 2 2 1 1
9 2 2 2 2 1