python 置信区间_关于置信区间的完整指南和Python示例

python 置信区间

Confidence Interval (CI) is essential in statistics and very important for data scientists. In this article, I will explain it thoroughly with necessary formulas and also demonstrate how to calculate it using python.

置信区间(CI)在统计中至关重要，对数据科学家而言非常重要。在本文中，我将用必要的公式彻底解释它，并演示如何使用python计算它。

置信区间 (Confidence Interval)

As it sounds, the confidence interval is a range of values. In the ideal condition, it should contain the best estimate of a statistical parameter. It is expressed as a percentage. 95% confidence interval is the most common. You can use other values like 97%, 90%, 75%, or even 99% confidence interval if your research demands. Let’s understand it by an example:

听起来，置信区间是一个值范围。在理想条件下，它应包含统计参数的最佳估计。用百分比表示。 95％的置信区间是最常见的。如果您的研究需要，可以使用其他值，例如97％，90％，75％甚至99％置信区间。让我们通过一个例子来理解它：

Here is a statement:

这是一条声明：

“In a sample of 659 parents with toddlers, about 85%, stated they use a car seat for all travel with their toddler. From these results, a 95% confidence interval was provided, going from about 82.3% up to 87.7%.”

“在659位有学步的父母的样本中，大约85％的受访者表示，他们在学步时都使用汽车安全座椅。根据这些结果，提供了95％的置信区间，从大约82.3％上升到87.7％。”

This statement means, we are 95% certain that the population proportion who use a car seat for all travel with their toddler will fall between 82.3% and 87.7%. If we take a different sample or a subsample of these 659 people, 95% of the time, the percentage of the population who use a car seat in all travel with their toddlers will be in between 82.3% and 87.7%.

该声明意味着，我们有95％的把握确定，带小孩旅行的所有年龄段的孩子所占的比例将介于82.3％和87.7％之间。如果我们对这659名人群进行不同的抽样或子抽样，则有95％的时间，在与孩子一起旅行的所有时间中使用汽车安全座椅的人口比例将在82.3％至87.7％之间。

Remember, 95% confidence interval does not mean 95% probability

请记住，95％的置信区间并不意味着95％的概率

The reason confidence interval is so popular and useful is, we cannot take data from all populations. Like the example above, we could not get the information from all the parents with toddlers. We had to calculate the result from 659 parents. From that result, we tried to get an estimate of the overall population. So, it is reasonable to consider a margin of error and take a range. That’s why we take a confidence interval which is a range.

置信区间如此受欢迎且有用的原因是，我们无法从所有人群中获取数据。像上面的示例一样，我们无法从所有有学步的父母那里获得信息。我们必须计算659位父母的结果。根据该结果，我们试图获得总体人口的估计值。因此，考虑误差幅度并取一个范围是合理的。这就是为什么我们将置信区间设为一个范围。

We want a simple random sample and a normal distribution to construct a confidence interval. But if the sample size is large enough (30 or more) normal distribution is not necessary.

我们想要一个简单的随机样本和一个正态分布来构造一个置信区间。但是，如果样本大小足够大(30个或更多)，则不需要正态分布。

GIPHY GIPHY

如何计算置信区间 (How to Calculate the Confidence Interval)

The calculation of the confidence interval involves the best estimate which is obtained by the sample and a margin of error. So, we take the best estimate and add a margin of error to it. Here is the formula for the confidence interval and the margin of error:

置信区间的计算涉及通过样本获得的最佳估计值和误差范围。因此，我们采用最佳估计，并在其中增加误差范围。这是置信区间和误差范围的公式：

Here, SE is the standard error.

在此，SE是标准误差 。

Normally, CI is calculated for two statistical parameters: the proportion and the mean.

通常，CI是针对两个统计参数计算的：比例和平均值。

Combining these two formulas above, we can elaborate the formula for CI as follows:

结合以上两个公式，我们可以将CI的公式阐述如下：

Population proportion or the mean is calculated from the sample. In the example of “the parents with toddlers”, the best estimate or the population proportion of parents that uses car seats in all travel with their toddlers is 85%. So, the best estimate (population proportion) is 85. z-score is fixed for the confidence level (CL).

人口比例或平均值是从样本中计算得出的。在“有小孩的父母”的示例中，最佳估计或在与小孩一起旅行的所有时间中使用汽车安全座椅的父母的人口比例为85％。因此，最佳估计值(人口比例)为85。z值固定为置信度(CL)。

A z-score for a 95% confidence interval for a large enough sample size(30 or more) is 1.96.

95％置信区间的z评分对于足够大的样本量(30个或更多)为1.96。

Here are the z-scores for some commonly used confidence levels:

以下是一些常用置信度水平的z得分：

The method to calculate the standard error is different for population proportion and mean. The formula to calculate standard error of population proportion is:

总体比例和均值的标准误差的计算方法不同。计算人口比例标准误的公式为：

The formula to calculate the standard error of the sample mean is:

计算样本平均值的标准误的公式为：

As per the statement, the population proportion that uses a car seat for all travel with their toddlers is 85%. So, this is our best estimate. We need to add the margin of error to it. To calculate the margin of error we need the z-score and the standard error. I am going to calculate a 95% CI. The z-score should be 1.96 and I already mentioned the formula of standard error for the population proportion. Plugging in all the values:

根据声明，在所有带小孩的旅行中使用汽车安全座椅的人口比例为85％。因此，这是我们的最佳估计。我们需要添加误差范围。要计算误差幅度，我们需要z分数和标准误差。我要计算95％CI。 z分数应为1.96，我已经提到了人口比例的标准误公式。插入所有值：

The confidence interval is 82.3% and 87.7% as we saw in the statement before.

正如我们在之前的声明中所看到的，置信区间为82.3％和87.7％。

演示地址

GIPHY GIPHY

Python中的置信区间 (Confidence interval in Python)

I am assuming that you are already a python user. But even if you are not a python user you should be able to get the concept of the calculation and use your own tools to calculate the same. The tools I used for this exercise are:

我假设您已经是python用户。但是，即使您不是python用户，您也应该能够获得计算的概念，并使用自己的工具进行计算。我用于此练习的工具是：

Numpy Library
脾气暴躁的图书馆
Pandas Library
熊猫图书馆
Statsmodels Library
统计模型库
Jupyter Notebook environment.
Jupyter Notebook环境。

If you install an anaconda package, you will get a Jupyter Notebook and the other tools as well. There are some good youtube videos to demonstrate how to install anaconda package if you do not have that already.

如果安装了Python包装，则还将获得Jupyter Notebook和其他工具。有一些不错的youtube视频，以演示如何安装anaconda软件包(如果尚未安装的话)。

CI代表Python中的人口比例 (CI for the population Proportion in Python)

I am going to use the Heart dataset from Kaggle. Please click on the link to download the dataset. First, I imported the packages and the dataset:

我将使用Kaggle的Heart数据集。请单击链接下载数据集。首先，我导入了软件包和数据集：

import pandas as pd
import numpy as np
df = pd.read_csv('Heart.csv')
df

The last column of the data is ‘AHD’. It says if a person has heart disease or not. In the beginning, we have a ‘Sex’ column as well.

数据的最后一列是“ AHD”。它说一个人是否患有心脏病。一开始，我们也有一个“性别”列。

We are going to construct a CI for the female population proportion that has heart disease.

我们将为患有心脏病的女性人口比例构建一个CI。

First, replace 1 and 0 with ‘Male’ and ‘Female’ in a new column ‘Sex1’.

首先，在新列“ Sex1”中将“ 1”和“ 0”分别替换为“ Male”和“ Female”。

df['Sex1'] = df.Sex.replace({1: "Male", 0: "Female"})

We do not need all the columns in the dataset. We will only use the ‘AHD’ column as that contains if a person has heart disease or not and the Sex1 column we just created. Make a DataFrame with only these two columns and drop all the null values.

我们不需要数据集中的所有列。我们将仅使用“ AHD”列，因为其中包含一个人是否患有心脏病以及我们刚刚创建的Sex1列。 创建一个仅包含这两列的DataFrame并删除所有空值。

dx = df[["AHD", "Sex1"]].dropna()

We need the number of females who have heart disease. The line of code below will give the number of males and females with heart disease and with no heart disease.

我们需要患有心脏病的女性人数。 下面的代码行将给出患有心脏病但没有心脏病的男性和女性的数量。

pd.crosstab(dx.AHD, dx.Sex1)

Here is the output table:

这是输出表：

The number of females who have heart disease is 25. Calculate the female population proportion with heart disease.

患有心脏病的女性人数为25。 计算患有心脏病的女性人口比例。

p_fm = 25/(72+25)

The ‘p_fm’ is 0.26. The size of the female population:

'p_fm'为0.26。 女性人数：

n = 72+25

The size of the female population is 97. Calculate the standard error

女性人口的总数为97。 计算标准误

se_female = np.sqrt(p_fm * (1 - p_fm) / n)

The standard error is 0.044.

标准误差为0.044。

Now construct the CI using the formulas above. The z-score is 1.96 for a 95% confidence interval.

现在，使用以上公式构造CI 。对于95％的置信区间，z得分是1.96。

z_score = 1.96
lcb = p_fm - z_score* se_female #lower limit of the CI
ucb = p_fm + z_score* se_female #upper limit of the CI

The confidence interval is 0.17 and 0.344.

置信区间为0.17和0.344。

You can calculate it using the library ‘statsmodels’.

您可以使用“ statsmodels”库进行计算。

import statsmodels.api as sm
sm.stats.proportion_confint(n * p_fm, n)

The confidence interval comes out to be the same as above.

置信区间与上述相同。

演示地址

GIPHY GIPHY

人口比例差异的CI (CI for the Difference in Population Proportion)

Is the population proportion of females with heart disease the same as the population proportion of males with heart disease? If they are the same, then the difference in both the population proportions will be zero.

患有心脏病的女性人口比例是否与患有心脏病的男性人口比例相同？如果它们相同，则两个人口比例之差将为零。

We will calculate a confidence interval of the difference in the population proportion of females and males with heart disease.

我们将计算患有心脏病的女性和男性人口比例差异的置信区间。

Here is the step by step process:

这是逐步的过程：

Calculate the male population proportion with heart disease and standard error using the same procedure.

使用相同的程序计算患有心脏病和标准误的男性人口比例。

p_male = 114/(114+92)  #male population proportion
n = 114+92             #total male population

The male population proportion with heart disease is 0.55 and the male population size is 206. Calculate the standard error for the male population proportion.

患有心脏病的男性人口比例为0.55，男性人口规模为206。 计算男性人口比例的标准误。

se_male = np.sqrt(p_male * (1 - p_male) / n)

The standard error for the male population is 0.034. Calculate the difference in standard error.

男性人群的标准误差是0.034。 计算标准误差的差。

The difference in standard error is not just subtraction. Use proper formula.

标准误差的差异不仅仅是减法。使用正确的公式。

Here is the formula to calculate the difference in two standard errors:

这是计算两个标准误差之差的公式：

Let’s use this formula to calculate the difference in the standard error of male and female population with heart disease.

让我们使用此公式来计算患有心脏病的男性和女性人口的标准误之差。

se_diff = np.sqrt(se_female**2 + se_male**2)

Use this standard error to calculate the difference in the population proportion of males and females with heart disease and construct the CI of the difference.

使用此标准误差来计算患有心脏病的男性和女性人口比例的差异，并构建差异的CI。

d = 0.55 - 0.26
lcb = d - 1.96 * se_diff  #lower limit of the CI
ucb = d + 1.96 * se_diff  #upper limit of the CI

The CI is 0.18 and 0.4. This range does not have 0 in it. Both the numbers are above zero. So, We cannot make any conclusion that the population proportion of females with heart disease is the same as the population proportion of males with heart disease. If the CI would be -0.12 and 0.1, we could say that the male and female population proportion with heart disease is the same.

CI为0.18和0.4。此范围内没有0。两个数字都大于零。因此，我们无法得出任何结论：患有心脏病的女性人口比例与患有心脏病的男性人口比例相同。如果CI为-0.12和0.1，则可以说患有心脏病的男性和女性人口比例相同。

演示地址

GIPHY GIPHY

均值CI的计算 (Calculation of CI of mean)

We will use the same heart disease dataset. The dataset has a ‘chol’ column that contains the cholesterol level. For this demonstration,

我们将使用相同的心脏病数据集。数据集有一个“胆固醇”列，其中包含胆固醇水平。为了这个示范

we will calculate the confidence interval of the mean cholesterol level of the female population.

我们将计算女性人群平均胆固醇水平的置信区间。

Let’s find the mean, standard deviation, and population size for the female population. I want to get the same parameters for the male population as well. Because it will be useful for our next exercise. Use pandas groupby and aggregate methods for this purpose. If you need a refresher on pandas groupby and aggregate method, please check out this article:

让我们找到女性人口的均值，标准差和人口规模。我也想为男性人口获得相同的参数。因为它将对我们的下一个练习有用。为此，请使用pandas groupby和aggregate方法。如果您需要对pandas groupby和aggregate方法进行复习，请查看本文：

Here is the code to get the mean, standard deviation, and population size of the male and female population:

这是获取男性和女性人口的均值，标准差和人口规模的代码：

df.groupby("Sex1").agg({"Chol": [np.mean, np.std, np.size]})

If we extract the necessary parameters for the female population only:

如果仅提取女性人口的必要参数：

mean_fe = 261.75  #mean cholesterol of female
sd = 64.9         #standard deviation for female population
n = 97            #Total number of female
z = 1.96          #z-score from the z table mentioned before

Here 1.96 is the z-score for a 95% confidence level.

这里1.96是95％置信水平的z得分。

Calculate the standard error using the formula for the standard error of the mean

使用公式计算平均值的标准误差

se = sd /np.sqrt(n)

Now we have everything to construct a CI for mean cholesterol in the female population.

现在，我们已掌握了构建女性人群平均胆固醇CI的一切方法。

Construct the CI

构建CI

lcb = mean_fe - z* se  #lower limit of the CI
ucb = mean_fe + z* se  #upper limit of the CI
(lcb, ucb)

The CI came out to be 248.83 and 274.67.

CI分别为248.83和274.67。

That means the true mean of the cholesterol of the female population will fall between 248.83 and 274.67

这意味着女性人群中胆固醇的真实平均值将介于248.83和274.67之间

演示地址

GIPHY GIPHY

均值差的CI的计算 (Calculation of CI of The Difference in Mean)

There are two approaches to calculate the CI for the difference in the mean of two populations.

有两种方法可以计算两个总体的均值之差的CI。

Pooled approach and unpooled approach

合并方法和非合并方法

As mentioned earlier, we need a simple random sample and a normal distribution. If the sample is large, a normal distribution is not necessary.

如前所述，我们需要一个简单的随机样本和一个正态分布。如果样本很大，则不需要正态分布。

There is one more assumption for a pooled approach. That is, the variance of the two populations is the same or almost the same.

合并方法还有一个假设。也就是说，两个总体的方差相同或几乎相同。

If the variance is not the same, the unpooled approach is more appropriate.

如果方差不相同，则非池化方法更为合适。

The formula of the standard error for the pooled approach is:

合并方法的标准误差公式为：

Here, s1 and s2 are the standard error for the population1 and population2. In the same way, n1 and n2 are the population size of population1 and population2.

s1和s2是总体1和总体2的标准误差。同样，n1和n2是人口1和人口2的人口规模。

The formula of the standard error for the unpooled approach is:

无池方法的标准误差公式为：

Here, we will construct the CI for the difference in mean of the cholesterol level of the male and female population.

在这里，我们将针对男性和女性人群中胆固醇水平平均值的差异构建CI。

We already derived all the necessary parameters from the dataset in the previous example. Here they are:

在前面的示例中，我们已经从数据集中导出了所有必要的参数。他们来了：

n1 = 97
n2 = 206
mean_female = 261.75
mean_male = 239.6
sd_female = 64.9
sd_male = 42.65

As we can see, the standard deviation of the two target populations is different. So. the variance must be different as well.

我们可以看到，两个目标人群的标准差是不同的。所以。方差也必须不同。

So, for this example, the unpooled approach will be more appropriate.

因此，对于此示例，非池化方法将更合适。

Calculate the standard error for male and female population using the formula we used in the previous example

使用上一个示例中使用的公式计算男性和女性人口的标准误差

sem_female = sd_female / np.sqrt(97)
sem_male = sd_male / np.sqrt(206)

The difference in mean of the two samples

两个样本的均值之差

mean_d = mean_female - mean_male

The difference in mean ‘mean_d’ is 22.15.

平均值“ mean_d”之差为22.15。

Using the formula for the unpooled approach, calculate the difference in standard error:

使用非池化方法的公式，计算标准误差的差：

sem_d = (np.sqrt((n1-1)*se_female**2 + (n2-1)*se_male**2)/(n1+n2-2))*(np.sqrt(1/n1 + 1/n2))

Finally, construct the CI for the difference in mean

最后，针对均值差构造CI

lcb = mean_d - 1.96*sem_d  #lower limit of the CI
ucb = mean_d + 1.96*sem_d  #upper limit of the CI
(lcb, ucb)

The lower and upper limit of the confidence interval came out to be 22.1494 and 22.15. They are almost the same. That means the mean cholesterol of the female population is not different than the mean cholesterol of the male population.

置信区间的上限和下限分别为22.1494和22.15。他们几乎一样。这意味着女性人口的平均胆固醇与男性人口的平均胆固醇没有差异。

结论 (Conclusion)

In this article, I tried to explain the confidence interval in detail with the calculation process in python. Python code I used here is simple enough for anyone to understand. Even if you are not a python user you should be able to understand the process and apply it in your way.

在本文中，我试图用python中的计算过程详细解释置信区间。我在这里使用的Python代码非常简单，任何人都可以理解。即使您不是python用户，您也应该能够理解该过程并以您的方式应用它。

演示地址

GIPHY GIPHY

python 置信区间_关于置信区间的完整指南和Python示例

置信区间 (Confidence Interval)

如何计算置信区间 (How to Calculate the Confidence Interval)

Python中的置信区间 (Confidence interval in Python)

CI代表Python中的人口比例 (CI for the population Proportion in Python)

人口比例差异的CI (CI for the Difference in Population Proportion)

均值CI的计算 (Calculation of CI of mean)

均值差的CI的计算 (Calculation of CI of The Difference in Mean)

结论 (Conclusion)

你可能感兴趣的:(python)