pandas drop_duplicates 在熊猫中不起作用？
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/46489695/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
drop_duplicates not working in pandas?
提问by A. Blackmagic
The purpose of my code is to import 2 Excel files, compare them, and print out the differences to a new Excel file.
我的代码的目的是导入 2 个 Excel 文件，比较它们，然后将差异打印到一个新的 Excel 文件中。
However, after concatenating all the data, and using the drop_duplicatesfunction, the code is accepted by the console. But, when printed to the new excel file, duplicates still remain within the day.
但是，在连接所有数据并使用该drop_duplicates函数后，该代码被控制台接受。但是，当打印到新的 excel 文件时，重复项仍然保留在一天内。
Am I missing something? Is something nullifying the drop_duplicatesfunction?
我错过了什么吗？有什么东西使drop_duplicates功能无效吗？
My code is as follows:
我的代码如下：
import datetime
import xlrd
import pandas as pd
#identify excel file paths
filepath = r"excel filepath"
filepath2 = r"excel filepath2"
#read relevant columns from the excel files
df1 = pd.read_excel(filepath, sheetname="Sheet1", parse_cols= "B, D, G, O")
df2 = pd.read_excel(filepath2, sheetname="Sheet1", parse_cols= "B, D, F, J")
#merge the columns from both excel files into one column each respectively
df4 = df1["Exchange Code"] + df1["Product Type"] + df1["Product Description"] + df1["Quantity"].apply(str)
df5 = df2["Exchange"] + df2["Product Type"] + df2["Product Description"] + df2["Quantity"].apply(str)
#concatenate both columns from each excel file, to make one big column containing all the data
df = pd.concat([df4, df5])
#remove all whitespace from each row of the column of data
df=df.str.strip()
df=["".join(x.split()) for x in df]
#convert the data to a dataframe from a series
df = pd.DataFrame({'Value': df})
#remove any duplicates
df.drop_duplicates(subset=None, keep="first", inplace=False)
#print to the console just as a visual aid
print(df)
#print the erroneous entries to an excel file
df.to_excel("Comparison19.xls")
回答by Keith
You've got inplace=Falseso you're not modifying df. You want either
你有inplace=False所以你不会修改df. 你想要
df.drop_duplicates(subset=None, keep="first", inplace=True)
or
或者
df = df.drop_duplicates(subset=None, keep="first", inplace=False)
回答by BAC83
I have just had this issue, and this was not the solution.
我刚刚遇到了这个问题，这不是解决方案。
It may be in the docs - I admittedly havent looked - and crucially this is only when dealing with date-based unique rows: the 'date' column must be formatted as such.
它可能在文档中 - 我承认我没有看过 - 关键是这仅在处理基于日期的唯一行时：“日期”列必须如此格式化。
If the datedata is a pandas objectdtype, the drop_duplicateswill not work - do a pd.to_datetimefirst.
如果date数据是 Pandas对象dtype，drop_duplicates则将不起作用 -pd.to_datetime首先执行。
回答by Mohamed Ali JAMAOUI
The use of inplace=Falsetells pandas to return a new dataframe with duplicates dropped, so you need to assign that back to df:
的使用inplace=False告诉Pandas返回一个删除重复项的新数据帧，因此您需要将其分配回df：
df = df.drop_duplicates(subset=None, keep="first", inplace=False)
or inplace=Trueto tell pandas to drop duplicates in the current dataframe
或inplace=True告诉Pandas删除当前数据框中的重复项
df.drop_duplicates(subset=None, keep="first", inplace=True)
