Paper intensive reading (十三)：Removing batch effects in analysis of expression microarray data

论文题目：Removing batch effects in analysis of expression microarray data: an evaluation of six batch adjustment methods

scholar 引用：258

页数：10

发表时间：February 28, 2011

发表刊物：PLOS ONE

作者：Chao Chen1,2, Kay Grennan2, Judith Badner2, Dandan Zhang3, Elliot Gershon2, Li Jin1, Chunyu Liu 

复旦大学

摘要：

The expression microarray is a frequently used approach to study gene expression on a genome-wide scale. However, the data produced by the thousands of microarray studies published annually are confounded by ‘‘batch effects,’’ the systematic error introduced when samples are processed in multiple batches. Although batch effects can be reduced by careful experimental design, they cannot be eliminated unless the whole study is done in a single batch. A number of programs are now available to adjust microarray data for batch effects prior to analysis. We systematically evaluated six of these programs using multiple measures of precision, accuracy and overall performance. ComBat, an Empirical Bayes method, outperformed the other five programs by most metrics. We also showed that it is essential to standardize expression data at the probe level when testing for correlation of expression profiles, due to a sizeable probe effect in microarray data that can inflate the correlation among replicates and unrelated samples.

ComBat（2007年提出的方法）是六个方法中效果最好的。

Discussion：

• we now know that the causes of batch effects include variables simply not under the control of the researcher. 批次效应一定存在，很多因素无法控制，再好的实验设计也没用。
• In the SMRI brain expression microarray data set, batch effects accounted for nearly 50% of the observed variation in expression, to which site effects contributed 42% and date effects 7.3%.batch effects主要包含site effects和date effects
• 1.ComBat outperformed other methods overall. Combat性能最佳！
• Its parametric and non-parametric algorithms both worked well in both kinds of data sets, controlling the variation attributable to batch effects, increasing the correlation among replicates, and producing the largest AUC in our assessment of overall performance. ComBat_p和ComBat_n性能指标评估都佳！
• We also confirmed another advantage of ComBat: it can robustly manage high-dimensional data when sample sizes are small, which is important for experiments with limited sample size, meta-analyses and clinical diagnostics. 在高维低样本量的数据集中，combat也是使用，这对于样本量局限、荟萃分析和临床诊断的情况下都很适用。
• Moreover, ComBat not only worked well on data generated on the Affymetrix platform, but has also been reported to work well with Illumina BeadChips data [28].
• 2.DWD方法的缺点：did not perform well in our analyses when batch sizes were small；it can only analyze two batches at a time. the standardization or normalization in DWD can change the scale between cases and controls；
• 3.SVA方法的缺陷：SVA is based on SVD. 首先，it is not necessarily a simple matter to identify the batch effect eigenvector(特征向量)，Batch effects may actually contribute substantially to several of the top eigenvectors, so SVD may not identify and remove all the batch effects, and may remove other effects not related to batch. 其次，a basic assumption of SVD is that the eigenvectors have Gaussian distributions. Batch effects, however, may be due to changes in technician, reagents, environmental conditions, scanner effects and/or other variables; this complicated situation may result in batch effects not being distributed in a Gaussian manner. 最后，这个方法的鲁棒性不强。
• 4.Ratio_G 有研究表明这个方法outperforms other methods in adjusting data for use in a predictive model, and reasoned that it is because non-ratio- based methods can confound batch and biological effects when one batch has a reverse negative/positive ratio compared to another batch.但是在本研究中，不怎么滴。accuracy and ROC-AUC results indicated that Ratio_G performed worse than ComBat_p and ComBat_n. Also, Ratio_G performed worst in removing batch effects from the SMRI data.
• 5.PAMR It did very well in our measures of accuracy because of this simple transformation. Again, though, batch effects are compli- cated, and do not affect all samples equally. PAMR does treat all samples equally, so it can over- or under-correct particular samples and came in second to ComBat in our measures of precision. 可以说仅次于ComBat
• 对于不良的实验设计，再好的方法也没有用。
• 介绍了一些同行的评估方法工作MAQC-II project
• 讨论了本文工作较同行工作的优点。
• we took only date and site effects into consideration, since platform-, channel- or tissue-dependent variations are avoidable with careful experimental design.
• 解释MAQC-II project中认为Ratio_G效果最好，而本研究中结论不一致的原因。
• Our evaluation makes clear that adjustment for batch effects is a mandatory step in the analysis of microarray data when the sample size is too large to fit in a single batch.
• ComBat was best able to reduce and remove batch effects while increasing precision and accuracy. 
• PAMR was a close second, but its performance suffered when batch size was small; only ComBat performed robustly when adjusting small batches. 

Introduction：

• Gene expression microarray technology is Promising

• 引出batch effect的定义：the term ‘‘batch’’ refers to microarrays processed at one site over a short period of time using the same platform. 同样的平台，同一个地点，短时间内

• The cumulative(累积) error introduced by these time and place-dependent experimental variations is referred to as ‘‘batch effects."

• Batch effects are almost inevitable; largely because most of the available microarray platforms can assay fewer than 24 samples at a time (the latest technology may process 96 samples in each batch). 平台处理能力有限，不得不分多个batch

• 选了六种方法，ComBat分有参数和无参数两种。

1. Distance-weighted discrimination (DWD)[11], based on the Support Vector Ma- chines (SVM) algorithm, is a two-class discrimination analysis for high-dimension low sample size data. 

2. Mean-centering (PAMR)[12] is a gene-wise one-way analysis of variance (ANOVA).

3. Surrogate variable analysis (SVA)[13], combines singular value decomposition (SVD) and a linear model analysis to estimate the eigenvalues from a residual expression matrix from which biological variation has already been removed. 

4. Geometric ratio-based method (Ratio_G) scales sample measurements by the geometric mean of a group of reference measurements [14].

5. An Empirical Bayes method, called Combating Batch Effects When Combining Batches of Gene Expression Microarray Data (ComBat)[15], estimates parameters for location and scale adjustment of each batch for each gene independently[16]; ComBat includes two methods, a parametric prior method (ComBat_p) and a non-parametric method (ComBat_n), based on the prior distributions of the estimated parameters. 

• 排除了一些方法比如说SVD，Ra- tio_A等是因为先前的很多研究已经表明了它们不如上述六种方法某一种，或者是上述六种方法某一种的微变形。

• since they are based on different statistical models, their accuracy, precision and overall effectiveness vary.

• 用模拟数据预估，因为知道batch effect的初始值，再用实验数据进行验证。

• 两个模拟数据集：

1. Variation Assessment Simulated (VAS), comprising 100 samples, 65 of which were assigned to Profile 1 and 35 of which were assigned to Profile 2, where profile was a generic random variable. 

2. Accuracy Assessment Simulated (AAS) ：consisted of 100 cases and 100 controls, with 1,200 out of 10,000 genes being differentially expressed with 12 different fold change values ranging from 23 to 3.

• 实验数据集：Stanley Medical Research Institute (SMRI) data set ： 62 individuals, with each replicate processed in one of three laboratories [18] Based on place and date of processing, the samples were run in 23 batches, averaging eight samples with at least one case and one control per batch. 

• The VAS and SMRI data were used for variation and precision assessment; the AAS and spike-in data were used for accuracy and overall performance evaluation. 

• All the batch adjustment methods were applied after experimental data were pre-processed by robust multiarray analysis (RMA) 通用处理

• 评估方法：

1. We first measured how much each program reduces the variation caused by batch effects. 

2. To test the programs’ precision, we assessed whether the expression values of the technical replicates correlated better before or after batch adjustment, 

3. As a measure of accuracy, the programs’ abilities to accurately quantify fold change in expression were assessed using the correlation between nominal fold changes and observed fold changes. 

4. To assess the overall detection ability of each program, we used a receiver operator characteristic (ROC) curve. 

• Our ultimate goal was to identify the batch adjustment method that best prepares data from multiple batches for analysis or meta- analysis to be integrated, as measured by batch effects reduction, accuracy, precision and overall performance.最终，ComBat获胜，而这个又包含两种方法，倒是可以分析一下两种方法的优劣。

正文组织架构：

1. Introduction

2. Results

2.1 Proportion of variation attributable to batch effects

2.2 Precision

2.3 Accuracy

2.4 Overall performance

3. Discussion

4. Materials and Methods

4.1 Samples

4.2 Expression microarray data simulations

4.3 Measuring source of variation

5. Supporting information 

正文部分内容摘录：

• Signal detection slopes were calculated using the spkTools[21] R package.

• The significances of differences between slopes were assessed with a test for homogeneity of slope[32], which was done with the NCStats R package[33].

• The evaluation of overall performance was performed using the ROCR[34] R package from Bioconductor[35].