数据筛选

df_all_cities是我们建立的一个包含所有数据的Pandas Dataframe，考虑到我们的分析目标，我们可能会需要提取部分数据来针对我们感兴趣的具体问题进行分析。为了方便大家对数据进行探索，在下面我们定义了一个filter_data和reading_stats的函数，通过输入不同的条件(conditions),该函数可以帮助我们筛选出这部分的数据。

def filter_data(data, condition):
"""
Remove elements that do not match the condition provided.
Takes a data list as input and returns a filtered list.
Conditions should be a list of strings of the following format:
' '
where the following operations are valid: >, <, >=, <=, ==, !=

Example: ["duration < 15", "start_city == 'San Francisco'"]
"""

# Only want to split on first two spaces separating field from operator and
# operator from value: spaces within value should be retained.
field, op, value = condition.split(" ", 2)

# check if field is valid
if field not in data.columns.values :
    raise Exception("'{}' is not a feature of the dataframe. Did you spell something wrong?".format(field))

# convert value into number or strip excess quotes if string
try:
    value = float(value)
except:
    value = value.strip("\'\"")

# get booleans for filtering
if op == ">":
    matches = data[field] > value
elif op == "<":
    matches = data[field] < value
elif op == ">=":
    matches = data[field] >= value
elif op == "<=":
    matches = data[field] <= value
elif op == "==":
    matches = data[field] == value
elif op == "!=":
    matches = data[field] != value
else: # catch invalid operation codes
    raise Exception("Invalid comparison operator. Only >, <, >=, <=, ==, != allowed.")

# filter data and outcomes
data = data[matches].reset_index(drop = True)
return data

def reading_stats(data, filters = [], verbose = True):
"""
Report number of readings and average PM2.5 readings for data points that meet
specified filtering criteria.
"""

n_data_all = data.shape[0]

# Apply filters to data
for condition in filters:
    data = filter_data(data, condition)

# Compute number of data points that met the filter criteria.
n_data = data.shape[0]

# Compute statistics for PM 2.5 readings.
pm_mean = data['PM_US_Post'].mean()
pm_qtiles = data['PM_US_Post'].quantile([.25, .5, .75]).as_matrix()

# Report computed statistics if verbosity is set to True (default).
if verbose:
    if filters:
        print('There are {:d} readings ({:.2f}%) matching the filter criteria.'.format(n_data, 100. * n_data / n_data_all))
    else:
        print('There are {:d} reading in the dataset.'.format(n_data))

    print('The average readings of PM 2.5 is {:.2f} ug/m^3.'.format(pm_mean))
    print('The median readings of PM 2.5 is {:.2f} ug/m^3.'.format(pm_qtiles[1]))
    print('25% of readings of PM 2.5 are smaller than {:.2f} ug/m^3.'.format(pm_qtiles[0]))
    print('25% of readings of PM 2.5 are larger than {:.2f} ug/m^3.'.format(pm_qtiles[2]))
    seaborn.boxplot(data['PM_US_Post'], showfliers=False)
    plt.title('Boxplot of PM 2.5 of filtered data')
    plt.xlabel('PM_US Post (ug/m^3)')

# Return three-number summary
return data

在使用中，我们只需要调用reading_stats即可，我们在这个函数中调用了filter_data函数，因此并不需要我们直接操作filter_data函数。下面是对于该函数的一些提示。

reading_stats函数中包含有3个参数：

第一个参数（必须）：需要被加载的 dataframe，数据将从这里开始分析。
第二个参数（可选）：数据过滤器，可以根据一系列输入的条件(conditions)来过滤将要被分析的数据点。过滤器应作为一系列条件提供，每个条件之间使用逗号进行分割，并在外侧使用""将其定义为字符串格式，所有的条件使用[]包裹。每个单独的条件应该为包含三个元素的一个字符串：' '（元素与元素之间需要有一个空格字符来作为间隔），可以使用以下任意一个运算符：>、<、>=、<=、==、!=。数据点必须满足所有条件才能计算在内。例如，["city == 'Beijing'", "season == 'Spring'"] 仅保留北京市，季节为春天的数据。在第一个条件中, 是city，是 ==, 是'Beijing',因为北京为字符串，所以加了单引号，它们三个元素之间分别添加一个空格。最后，这个条件需要使用双引号引用起来。这个例子中使用了两个条件，条件与条件之间使用逗号进行分割，这两个条件最后被放在[]之中。
第三个参数（可选）：详细数据，该参数决定我们是否打印被选择的数据的详细统计信息。如果verbose = True，会自动打印数据的条数，以及四分位点，并绘制箱线图。如果verbose = False, 则只会返回筛选后的dataframe，不进行打印。