Filter dataframe rows if value in column is in a set list of values [duplicate]
I have a Python pandas DataFrame rpt
:
rpt
<class 'pandas.core.frame.DataFrame'>
MultiIndex: 47518 entries, ('000002', '20120331') to ('603366', '20091231')
Data columns:
STK_ID 47518 non-null values
STK_Name 47518 non-null values
RPT_Date 47518 non-null values
sales 47518 non-null values
I can filter the rows whose stock id is '600809'
like this: rpt[rpt['STK_ID'] == '600809']
<class 'pandas.core.frame.DataFrame'>
MultiIndex: 25 entries, ('600809', '20120331') to ('600809', '20060331')
Data columns:
STK_ID 25 non-null values
STK_Name 25 non-null values
RPT_Date 25 non-null values
sales 25 non-null values
and I want to get all the rows of some stocks together, such as ['600809','600141','600329']
. That means I want a syntax like this:
stk_list = ['600809','600141','600329']
rst = rpt[rpt['STK_ID'] in stk_list] # this does not works in pandas
Since pandas not accept above command, how to achieve the target?
Use the isin
method:
rpt[rpt['STK_ID'].isin(stk_list)]
isin()
is ideal if you have a list of exact matches, but if you have a list of partial matches or substrings to look for, you can filter using the str.contains
method and regular expressions.
For example, if we want to return a DataFrame where all of the stock IDs which begin with '600'
and then are followed by any three digits:
>>> rpt[rpt['STK_ID'].str.contains(r'^600[0-9]{3}$')] # ^ means start of string
... STK_ID ... # [0-9]{3} means any three digits
... '600809' ... # $ means end of string
... '600141' ...
... '600329' ...
... ... ...
Suppose now we have a list of strings which we want the values in 'STK_ID'
to end with, e.g.
endstrings = ['01$', '02$', '05$']
We can join these strings with the regex 'or' character |
and pass the string to str.contains
to filter the DataFrame:
>>> rpt[rpt['STK_ID'].str.contains('|'.join(endstrings)]
... STK_ID ...
... '155905' ...
... '633101' ...
... '210302' ...
... ... ...
Finally, contains
can ignore case (by setting case=False
), allowing you to be more general when specifying the strings you want to match.
For example,
str.contains('pandas', case=False)
would match PANDAS
, PanDAs
, paNdAs123
, and so on.
you can also use ranges by using:
b = df[(df['a'] > 1) & (df['a'] < 5)]