Python 对比两个数据集 datacompy包 及报错 TypeError: ‘NoneType‘ object is not iterable的解决
目标：对比两个数据集是否完全相同
数据集：df1,df2
方法一：pandas
两个数据集相减
#df2减df1
import pandas as pd
set_diff_df = pd.concat([df1, df2, df1]).drop_duplicates(keep=False)
print(set_diff_df)结果
Empty DataFrame表示两个数据集相同
方法二：datacompy包
需要安装
这个包的详细说明https://capitalone.github.io/datacompy/install.html
Windows10 Python3环境 anaconda进行安装
conda install datacompy成功后运行
import datacompy
compare=datacompy.Compare(df1,df2,abs_tol=0.000001)
print(compare.report())报错
TypeError: 'NoneType' object is not iterable后来发现是没有加入对比的连接列
以索引为连接列的代码如下
import datacompy
compare=datacompy.Compare(df1,df2,abs_tol=0.000001,on_index=True)
print(compare.report())成功！
结果如下
ataComPy Comparison
--------------------
DataFrame Summary
-----------------
DataFrame Columns Rows
0 df1 12 2000
1 df2 12 2000
Column Summary
--------------
Number of columns in common: 12
Number of columns in df1 but not in df2: 0
Number of columns in df2 but not in df1: 0
Row Summary
-----------
Matched on: index
Any duplicates on match values: No
Absolute Tolerance: 1e-06
Relative Tolerance: 0
Number of rows in common: 2,000
Number of rows in df1 but not in df2: 0
Number of rows in df2 but not in df1: 0
Number of rows with some compared columns unequal: 0
Number of rows with all compared columns equal: 2,000
Column Comparison
-----------------
Number of columns compared with some values unequal: 0
Number of columns compared with all values equal: 12
Total number of values which compare unequal: 0也可以以数据的某个变量为连接列
语句中加入：
join_columns=['变量名']