select pandas rows by excluding index number
Not quite sure why I can't figure this out. I'm looking to slice a Pandas dataframe by using index numbers. I have a list/core index with the index numbers that i do NOT need, shown below
pandas.core.index.Int64Index
Int64Index([2340, 4840, 3163, 1597, 491 , 5010, 911 , 3085, 5486, 5475, 1417, 2663, 4204, 156 , 5058, 1990, 3200, 1218, 3280, 793 , 824 , 3625, 1726, 1971, 2845, 4668, 2973, 3039, 376 , 4394, 3749, 1610, 3892, 2527, 324 , 5245, 696 , 1239, 4601, 3219, 5138, 4832, 4762, 1256, 4437, 2475, 3732, 4063, 1193], dtype=int64)
How can I create a new dataframe excluding these index numbers. I tried
df.iloc[combined_index]
and obviously this just shows the rows with those index number (the opposite of what I want). any help will be greatly appreciated
Solution 1:
Not sure if that's what you are looking for, posting this as an answer, because it's too long for a comment:
In [31]: d = {'a':[1,2,3,4,5,6], 'b':[1,2,3,4,5,6]}
In [32]: df = pd.DataFrame(d)
In [33]: bad_df = df.index.isin([3,5])
In [34]: df[~bad_df]
Out[34]:
a b
0 1 1
1 2 2
2 3 3
4 5 5
Solution 2:
Just use .drop
and pass it the index list to exclude.
import pandas as pd
df = pd.DataFrame({"a": [10, 11, 12, 13, 14, 15]})
df.drop([1, 2, 3], axis=0)
Which outputs this.
a
0 10
4 14
5 15
Solution 3:
Probably an easier way is just to use a boolean index, and slice normally doing something like this:
df[~df.index.isin(list_to_exclude)]
Solution 4:
You could use pd.Int64Index(np.arange(len(df))).difference(index)
to form a new ordinal index. For example, if we want to remove the rows associated with ordinal index [1,3,5], then
import numpy as np
import pandas as pd
index = pd.Int64Index([1,3,5], dtype=np.int64)
df = pd.DataFrame(np.arange(6*2).reshape((6,2)), index=list('ABCDEF'))
# 0 1
# A 0 1
# B 2 3
# C 4 5
# D 6 7
# E 8 9
# F 10 11
new_index = pd.Int64Index(np.arange(len(df))).difference(index)
print(df.iloc[new_index])
yields
0 1
A 0 1
C 4 5
E 8 9