从 Pandas 数据框中删除重复项并保留原始数据
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/33042777/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Removing duplicates from Pandas dataFrame with condition for retaining original
提问by Rakesh Adhikesavan
Assuming I have the following DataFrame:
假设我有以下数据帧：
A | B
1 | Ms
1 | PhD
2 | Ms
2 | Bs
I want to remove the duplicate rows with respect to column A, and I want to retain the row with value 'PhD' in column B as the original, if I don't find a 'PhD', I want to retain the row with 'Bs' in column B.
我想删除关于 A 列的重复行，并且我想保留 B 列中值为 'PhD' 的行作为原始行，如果我没有找到'PhD'，我想保留该行B列中的“Bs”。
I am trying to use
我正在尝试使用
df.drop_duplicates('A')
with a condition
有条件
回答by hellpanderr
>>> df
A B
0 1 Ms
1 1 Ms
2 1 Ms
3 1 Ms
4 1 PhD
5 2 Ms
6 2 Ms
7 2 Bs
8 2 PhD
Sorting a dataframe with a custom function:
使用自定义函数对数据框进行排序：
def sort_df(df, column_idx, key):
'''Takes a dataframe, a column index and a custom function for sorting,
returns a dataframe sorted by that column using that function'''
col = df.ix[:,column_idx]
df = df.ix[[i[1] for i in sorted(zip(col,range(len(col))), key=key)]]
return df
Our function for sorting:
我们的排序功能：
cmp = lambda x:2 if 'PhD' in x else 1 if 'Bs' in x else 0
cmp = lambda x:2 if 'PhD' in x else 1 if 'Bs' in x else 0
In action:
在行动：
sort_df(df,'B',cmp).drop_duplicates('A', take_last=True)
sort_df(df,'B',cmp).drop_duplicates('A', take_last=True)
A B
4 1 PhD
8 2 PhD
回答by jgloves
Assuming uniqueness of B value given A value, and that each A value has a row with Bs in the B column:
假设给定 A 值的 B 值的唯一性，并且每个 A 值在 B 列中有一行 Bs：
df2 = df[df['B']=="PhD"]
will give you a dataframe with the PhD rows you want.
会给你一个包含你想要的博士行的数据框。
Then remove all the PhD and Ms from df:
然后从 df 中删除所有的博士和女士：
df = df[df['B']=="Bs"]
Then concatenate df and df2:
然后连接 df 和 df2：
df3 = concat([df2, df])
Then you can use drop_duplicates like you wanted:
然后你可以像你想要的那样使用 drop_duplicates ：
df3.drop_duplicates('A', inplace=True)
回答by mattvivier
Consider using Categoricals. They're a nice was to group / order text non-alphabetically (among other things.)
考虑使用Categoricals. 他们很高兴能够按非字母顺序对文本进行分组/排序（除其他外）。
import pandas as pd
df = pd.DataFrame([(1,'Ms'), (1, 'PhD'), (2, 'Ms'), (2, 'Bs'), (3, 'PhD'), (3, 'Bs'), (4, 'Ms'), (4, 'PhD'), (4, 'Bs')], columns=['A', 'B'])
df['B']=df['B'].astype('category')
# after setting the column's type to 'category', you can set the order
df['B']=df['B'].cat.set_categories(['PhD', 'Bs', 'Ms'], ordered=True)
df.sort(['A', 'B'], inplace=True)
df_unique = df.drop_duplicates('A')
