Comparison of linear discriminant analysis methods for the classification of cancer based on gene expression data
 Desheng Huang^{1, 2},
 Yu Quan^{3},
 Miao He^{4} and
 Baosen Zhou^{2, 5}Email author
DOI: 10.1186/1756996628149
© Huang et al; licensee BioMed Central Ltd. 2009
Received: 8 November 2009
Accepted: 10 December 2009
Published: 10 December 2009
Abstract
Background
More studies based on gene expression data have been reported in great detail, however, one major challenge for the methodologists is the choice of classification methods. The main purpose of this research was to compare the performance of linear discriminant analysis (LDA) and its modification methods for the classification of cancer based on gene expression data.
Methods
The classification performance of linear discriminant analysis (LDA) and its modification methods was evaluated by applying these methods to six public cancer gene expression datasets. These methods included linear discriminant analysis (LDA), prediction analysis for microarrays (PAM), shrinkage centroid regularized discriminant analysis (SCRDA), shrinkage linear discriminant analysis (SLDA) and shrinkage diagonal discriminant analysis (SDDA). The procedures were performed by software R 2.80.
Results
PAM picked out fewer feature genes than other methods from most datasets except from Brain dataset. For the two methods of shrinkage discriminant analysis, SLDA selected more genes than SDDA from most datasets except from 2class lung cancer dataset. When comparing SLDA with SCRDA, SLDA selected more genes than SCRDA from 2class lung cancer, SRBCT and Brain dataset, the result was opposite for the rest datasets. The average test error of LDA modification methods was lower than LDA method.
Conclusions
The classification performance of LDA modification methods was superior to that of traditional LDA with respect to the average error and there was no significant difference between theses modification methods.
Background
Conventional diagnosis of cancer has been based on the examination of the morphological appearance of stained tissue specimens in the light microscope, which is subjective and depends on highly trained pathologists. Thus, the diagnostic problems may occur due to interobserver variability. Microarrays offer the hope that cancer classification can be objective and accurate. DNA microarrays measure thousands to millions of gene expressions at the same time, which could provide the clinicians with the information to choose the most appropriate forms of treatment.
Studies on the diagnosis of cancer based on gene expression data have been reported in great detail, however, one major challenge for the methodologists is the choice of classification methods. Proposals to solve this problem have utilized many innovations including the introduction of sophisticated algorithms for support vector machines [1] and the proposal of ensemble methods such as random forests [2]. The conceptually simple approach of linear discriminant analysis (LDA) and its sibling, diagonal discriminant analysis (DDA) [3–5], remain among the most effective procedures also in the domain of highdimensional prediction. In the present study, our main focus will be solely put on the LDA part and henceforth the term "discriminant analysis" will stand for the meaning of LDA unless otherwise emphasized. The traditional way of doing discriminant analysis is introduced by R. Fisher, known as the linear discriminant analysis (LDA). Recently some modification of LDA have been advanced and gotten good performance, such as prediction analysis for microarrays (PAM), shrinkage centroid regularized discriminant analysis(SCRDA), shrinkage linear discriminant analysis(SLDA) and shrinkage diagonal discriminant analysis(SDDA). So, the main purpose of this research was to describe the performance of LDA and its modification methods for the classification of cancer based on gene expression data.
Cancer is not a single disease, there are many different kinds of cancer, arising in different organs and tissues through the accumulated mutation of multiple genes. Many previous studies only focused on one method or single dataset and gene selection is much more difficult in multiclass situations [6, 7]. Evaluation of the most commonly employed methods may give more accurate results if it is based on the collection of multiple databases from the statistical point of view.
In summary, we investigate the performance of LDA and its modification methods for the classification of cancer based on multiple gene expression datasets.
Methods
Datasets
Characteristics of the six microarray datasets used
Dataset  No. of samples  Classes (No. of samples)  No. of genes  Original ref.  Website 

Twoclass lung cancer  181  MPM(31), adenocarcinoma(150)  12533  [8]  
Colon  62  normal(22), tumor(40)  2000  [9]  http://microarray.princeton.edu/oncology/affydata/index.html 
Prostate  102  normal(50), tumor(52)  6033  [10]  http://microarray.princeton.edu/oncology/affydata/index.html 
Multiclass lung cancer  68(66) ^{a}  adenocarcinoma(37), combined(1), normal(5), small cell(4), squamous cell(10), fetal(1), large cell(4), lymph node(6)  3171  
SRBCT  88(83) ^{b}  Burkitt lymphoma (29), Ewing sarcoma (11), neuroblastoma (18), rhabdomyosarcoma (25), nonSRBCTs(5)  2308  [13]  
Brain  42(38) ^{c}  medulloblastomas(10), CNS AT/RTs(5), rhabdoid renal and extrarenal rhabdoid tumours(5), supratentorial PNETs(8), nonembryonal brain tumours (malignant glioma) (10), normal human cerebella(4)  5597  [14] 
Data preprocessing
To avoid the noise of the dataset, preprocessing was necessary in the analysis. Absolute transformation was first performed on the original data. The data was transformed to have a mean of 0 and standard deviation of 1 after logarithmic transformation and normalization. When the original data had already experienced the above transformation, it entered next step directly.
Algorithms for feature gene selection
Notation
Let x_{ij} be the expression level of gene j in the sample i, and y_{i} be the cancer type for sample i, j = 1,...,p and response y_{i}∈{1,...,K}. Denote Y = (y_{1},...,y_{n})^{T} and x_{i} = (x_{i1},...,x_{ip})^{T}, i = 1,...,n. Gene expression data on p genes for n mRNA samples may be summarized by an n × p matrix X = (x_{ij})_{n × p}. Let C_{k} be indices of the n_{k} samples in class k, where n_{k} denotes the number of observations belonging to class k, n = n_{1}+...+n_{K}. A predictor or classifier for K tumor classes can be built from a learning set L by C(.,L); the predicted class for an observation x* is C(x*,L). The jth component of the centroid for class k is , the jth component of the overall centroid is .
Prediction analysis for microarrays/nearest shrunken centroid method, PAM/NSC
and , s_{0} is a positive constant and usually equal to the median value of the s_{j} over the set of genes.
where + means positive part (t_{+} = t if t>0 and zero otherwise). For a gene j, if d_{kj} is shrunken to zero for all classes k, then the centroid for gene j is , the same for all classes. Thus gene j does not contribute to the nearestcentroid computation. Soft threshold Δ was chosen by crossvalidation.
Shrinkage discriminant analysis, SDA
In SDA, Feature selection is controlled using higher criticism threshold (HCT) or false nondiscovery rates (FNDR) [5]. The HCT is the order statistic of the Zscore corresponding to index i maximizing , π_{i} is the pvalue associated with the ith Zscore and π_{(i)} is the i th order statistic of the collection of pvalues(1 ≤ i ≤ p). The ideal threshold optimizes the classification error. SDA consists of Shrinkage linear discriminant analysis (SLDA) and Shrinkage diagonal discriminant analysis (SDDA) [15, 16].
Shrunken centroids regularized discriminant analysis, SCRDA
There are two parameters in SCRDA [4], one is α (0<α<1), the other is soft threshold Δ. The choosing the optimal tuning parameter pairs (α, Δ) is based on crossvalidation. A "MinMin" rule was followed to identify the optimal parameter pair (α, Δ):
First, all the pairs (α, Δ) that corresponded to the minimal crossvalidation error from training samples were found.
Second, the pair or pairs that used the minimal number of genes were selected.
When there was more than one optimal pair, the average test error based on all the pairs chosen would be calculated. As traditional LDA is not suitable to deal with the "large p, small N" paradigm, so we did not adopt it to select feature genes.
Algorithms of LDA and its modification methods for classification
Linear discriminant analysis, LDA
Where B and W denote the matrices of betweengroup and withingroup sums of squares and crossproducts.
where , v_{ l }is eigenvector, s is the number of feature genes.
When numbers of classes K = 2, FLDA yields the same classifier as the maximum likelihood (ML) discriminant rule for multivariate normal class densities with the same covariance matrix.
Prediction analysis for microarrays/nearest shrunken centroid method, PAM/NSC
Here was the diagonal matrix taking the diagonal elements of . If the smallest distances are close and hence ambiguous, the prior correction gives a preference for larger classes, because they potentially account for more errors.
Shrinkage discriminant analysis, SDA
Where , P = (ρ_{ij}) and
Algorithm of SCRDA
where 0 ≤ α ≤ 1
In the same way, sample correlation matrix was substituted by .
Then the regularized sample covariance matrix was computed by
Study design and program realization
We used 10fold crossvalidation (CV) to divide the preprocessed dataset into 10 approximately equalsize parts by random sampling. It worked as follows: we fit the model on 90% of the samples and then predicted the class labels of the remaining 10% (the test samples). This procedure was repeated 10 times to avoid overlapping test sets, with each part playing the role of the test samples and the errors on all 10 parts added together to compute the overall error [18]. R software (version 2.80) with packages MASS, pamr, RDA, SDA was used for the realization of the above described methods [19]. A tolerance value was set to decide if a matrix is singular. If variable had withingroup variance less than tol^2, LDA fitting iteration would stop and report the variable as constant. In practice, we set a very small tolerance value 1 × 10^{14}, and no singular was detected.
Results
Feature genes selection
Numbers of feature genes selected by 4 methods for each dataset
Dataset  PAM  SDDA  SLDA  SCRDA 

2class lung cancer  7.98  422.74  407.83  118.72 
Colon  25.72  65.67  117.08  214.87 
Prostate  83.13  120.53  187.91  217.47 
Multiclass lung cancer  45.26  57.98  97.27  1015.00 
SRBCT  30.87  114.32  131.24  86.22 
Brain  69.11  115.04  182.01  26.83 
Performance comparison for methods based on different datasets
The performance of the methods described above was compared by average test error using 10fold cross validation. We ran 10 cycles of 10fold cross validation. The average test errors were calculated based on the incorrectness of the classification of each testing samples. For example, for the 2class lung cancer dataset, using the LDA method based on PAM as the feature gene method, 30 samples out of 100 sample test sets were incorrectly classified, resulting in an average test error of 0.30.
Average test error of LDA and its modification methods (10 cycles of 10fold cross validation)
Dataset  Gene selection methods  Performance  

LDA  PAM  SDDA  SLDA  SCRDA  
2class Lung cancer data(n = 181, p = 12533, K = 2)  PAM  0.30  0.26  0.15  0.16  0.42 
SDDA  0.17  0.11  0.1  0.11  0.1  
SLDA  0.47  0.3  0.3  0.3  0.32  
SCRDA  0.73  0.20  0.19  0.17  0.19  
Colon data(n = 62, p = 2000, K = 2)  PAM  1.30  0.82  0.8  0.86  0.86 
SDDA  2.25  2.09  1.33  1.29  1.25  
SLDA  1.12  0.74  0.75  0.77  0.80  
SCRDA  1.19  0.77  0.77  0.75  0.78  
Prostate data(n = 102, p = 6033, K = 2)  PAM  2.87  0.89  0.82  0.81  1.00 
SDDA  2.53  0.71  0.72  0.68  0.74  
SLDA  1.75  0.7  0.64  0.64  0.70  
SCRDA  2.15  0.57  0.59  0.57  0.61  
Multiclass lung cancer data(n = 66, p = 3171, K = 6)  PAM  2.13  1.16  1.21  1.28  1.19 
SDDA  1.62  1.32  1.32  1.31  1.30  
SLDA  1.62  1.31  1.32  1.26  1.34  
SCRDA  1.63  1.43  1.45  1.58  1.35  
SRBCT data(n = 83, p = 2308, K = 4)  PAM  0.17  0.01  0.01  0.03  0.01 
SDDA  2.45  0.03  0.02  0  0.03  
SLDA  2.87  0  0  0  0  
SCRDA  2.32  0.03  0.03  0.02  0.03  
Brain data(n = 38, p = 5597, K = 4)  PAM  1.14  0.57  0.57  0.58  0.61 
SDDA  1.09  0.61  0.62  0.63  0.55  
SLDA  0.89  0.60  0.60  0.57  0.58  
SCRDA  0.84  0.56  0.54  0.54  0.57 
Discussion
Microarrays are capable of determining the expression levels of thousands of genes simultaneously and hold great promise to facilitate the discovery of new biological knowledge [20]. One feature of microarray data is that the number of variables p (genes) far exceeds the number of samples N. In statistical terms, it is called 'large p, small N' problem. Standard statistical methods in classification do not work well or even at all, so improvement or modification of existing statistical methods is needed to prevent overfitting and produce more reliable estimations. Some adhoc shrinkage methods have been proposed to utilize the shrinkage ideas and prove to be useful in empirical studies [21–23]. Distinguishing normal samples from tumor samples is essential for successful diagnosis or treatment of cancer. And, another important problem is in characterizing multiple types of tumors. The problem of multiple classifications has recently received more attention in the context of DNA microarrays. In the present study, we first presented an evaluation of the performance of LDA and its modification methods for classification with 6 public microarray datasets.
The gene selection method [6, 24, 25], the number of selected genes and the classification method are three critical issues for the performance of a sample classification. Feature selection techniques can be organized into three categories, filter methods, wrapper methods and embedded methods. LDA and its modification methods belong to wrapper methods which embed the model hypothesis search within the feature subset search. In the present study, different numbers of gene have been selected by different LDA modification methods. There is no theoretical estimation of the optimal number of selected genes and the optimal gene set can vary from data to data [26]. So we did not focus on the combination of the optimal gene set by one feature gene selection method and one classification algorithm. In this paper we just describe the performance of LDA and its modification methods under the same selection method in different microarray dataset.
Various statistical and machine learning methods have been used to analyze the high dimensional data for cancer classification. These methods have been shown to have statistical and clinical relevance in cancer detection for a variety of tumor types. In this study, it has been shown that LDA modification methods have better performance than traditional LDA under the same gene selection criterion. Dudoit also reported that simple classifiers such as DLDA and Nearest Neighbor performed remarkably well compared with more sophisticated ones, such as aggregated classification trees [27]. It indicates that LDA modification methods did a good job in some situations. Zhang et al[28] developed a fast algorithm of generalized linear discriminant analysis (GLDA) and applied it to seven public cancer datasets. Their study included 4 same datasets (Colon, Prostate, SRBCT and Brain) as those in our study and adopted a 3fold crossvalidation design. The average test errors of our study were less than those of their study, while there was no statistical significance of the difference. The results reported by Guo et al[4] are of concordance with ours except for the colon dataset. Their study also included the above mentioned 4 same datasets and they found that in the colon dataset the average test error of SCRDA was as same as PAM, while in the present study we found that the average test error of SCRDA was slightly less than that of PAM.
There are several interesting problems that remain to be addressed. A question is raised that when comparing the predictive performance of different classification methods on different microarray data, is there any difference between various methods, such as leaveoneout crossvalidation and bootstrap [29, 30]? And another interesting further step might be a preanalysis of the data to choose a suitable gene selection method. Despite the great promise of discriminant analysis in the field of microarray technology, the complexity and the multiple choices of the available methods are quite difficult to the bench clinicians. This may influence the clinicians' adoption of microarray data based results when making decision on diagnosis or treatment. Microarray data's widespread clinical relevance and applicability still need to be resolved.
Conclusions
An extensive survey in building classification models from microarray data with LDA and its modification methods has been conducted in the present study. The study showed that the modification methods are superior to LDA in the prediction accuracy.
List of abbreviations
 CV:

Crossvalidation
 DDA:

diagonal discriminant analysis
 FNDR:

False nondiscovery rates
 GLDA:

generalized linear discriminant analysis
 HCT:

Higher criticism threshold
 LDA:

linear discriminant analysis
 NSC:

nearest shrunken centroid method
 PAM:

prediction analysis for microarrays
 SCRDA:

Shrinkage centroid regularized discriminant analysis
 SDA:

Shrinkage discriminant analysis
 SDDA:

Shrinkage diagonal discriminant analysis
 SLDA:

Shrinkage linear discriminant analysis.
Declarations
Acknowledgements
This study was partially supported by Provincial Education Department of Liaoning (No.2008S232), Natural Science Foundation of Liaoning province (No.20072103) and China Medical Board (No.00726.). The authors are most grateful to the contributors of the datasets and R statistical software. The authors thank the two reviewers for their insightful comments which led to an improved version of the manuscript.
Authors’ Affiliations
References
 Guyon I, Weston J, Barnhill , Vapnik V: Gene Selection for Cancer Classification using Support Vector Machines. Mach Learn. 2002, 46: 389422. 10.1023/A:1012487302797.View ArticleGoogle Scholar
 Breiman L: Random Forests. Mach Learn. 2001, 45: 532. 10.1023/A:1010933404324.View ArticleGoogle Scholar
 Tusher VG, Tibshirani R, Chu G: Significance analysis of microarrays applied to the ionizing radiation response. Proc Natl Acad Sci USA. 2001, 98: 51165121. 10.1073/pnas.091062498.View ArticleGoogle Scholar
 Guo Y, Hastie T, Tibshirani R: Regularized linear discriminant analysis and its application in microarrays. Biostatistics. 2005, 8: 86100. 10.1093/biostatistics/kxj035.View ArticleGoogle Scholar
 Schäfer J, Strimmer K: A shrinkage approach to largescale covariance matrix estimation and implications for functional genomics. Stat Appl Genet Mol Biol. 2005, 4:Google Scholar
 Yeung KY, Bumgarner RE, Raftery AE: Bayesian model averaging: development of an improved multiclass, gene selection and classification tool for microarray data. Bioinformatics. 2005, 21: 23942402. 10.1093/bioinformatics/bti319.View ArticleGoogle Scholar
 Li T, Zhang C, Ogihara M: A comparative study of feature selection and multiclass classification methods for tissue classification based on gene expression. Bioinformatics. 2004, 20: 24292437. 10.1093/bioinformatics/bth267.View ArticleGoogle Scholar
 Gordon GJ, Jensen RV, Hsiao LL, Gullans SR, Blumenstock JE, Ramaswamy S, Richards WG, Sugarbaker DJ, Bueno R: Translation of microarray data into clinically relevant cancer diagnostic tests using gene expression ratios in lung cancer and mesothelioma. Cancer Res. 2002, 62: 49634967.Google Scholar
 Alon U, Barkai N, Notterman DA, Gish K, Ybarra S, Mack D, Levine AJ: Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proc Natl Acad Sci USA. 1999, 96: 67456750. 10.1073/pnas.96.12.6745.View ArticleGoogle Scholar
 Singh D, Febbo PG, Ross K, Jackson DG, Manola J, Ladd C, Tamayo P, Renshaw AA, D'Amico AV, Richie JP, Lander ES, Loda M, Kantoff PW, Golub TR, Sellers WR: Gene expression correlates of clinical prostate cancer behavior. Cancer Cell. 2002, 1: 203209. 10.1016/S15356108(02)000302.View ArticleGoogle Scholar
 Bhattacharjee A, Richards WG, Staunton J, Li C, Monti S, Vasa P, Ladd C, Beheshti J, Bueno R, Gillette M, Loda M, Weber G, Mark EJ, Lander ES, Wong W, Johnson BE, Golub TR, Sugarbaker DJ, Meyerson M: Classification of human lung carcinomas by mRNA expressionprofiling reveals distinct adenocarcinoma subclasses. Proc Natl Acad Sci USA. 2001, 98: 1379013795. 10.1073/pnas.191502998.View ArticleGoogle Scholar
 Parmigiani G, GarrettMayer ES, Anbazhagan R, Gabrielson E: A crossstudy comparison of gene expression studies for the molecular classification of lung cancer. Clin Cancer Res. 2004, 10: 29222927. 10.1158/10780432.CCR030490.View ArticleGoogle Scholar
 Khan J, Wei JS, Ringnér M, Saal LH, Ladanyi M, Westermann F, Berthold F, Schwab M, Antonescu CR, Peterson C, Meltzer PS: Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks. Nat Med. 2001, 7: 673679. 10.1038/89044.View ArticleGoogle Scholar
 Pomeroy SL, Tamayo P, Gaasenbeek M, Sturla LM, Angelo M, McLaughlin ME, Kim JY, Goumnerova LC, Black PM, Lau C, Allen JC, Zagzag D, Olson JM, Curran T, Wetmore C, Biegel JA, Poggio T, Mukherjee S, Rifkin R, Califano A, Stolovitzky G, Louis DN, Mesirov JP, Lander ES, Golub TR: Prediction of central nervous system embryonal tumour outcome based on gene expression. Nature. 2002, 415: 436442. 10.1038/415436a.View ArticleGoogle Scholar
 OpgenRhein R, Strimmer K: Accurate ranking of differentially expressed genes by a distributionfree shrinkage approach. Stat Appl Genet Mol Biol. 2007, 6: Article9Google Scholar
 Schäfer J, Strimmer K: A shrinkage approach to largescale covariance matrix estimation and implications for functional genomics. Stat Appl Genet Mol Biol. 2005, 4: Article32Google Scholar
 Fisher RA: The Use of Multiple Measurements in Taxonomic Problems. Annuals of Eugenics. 1936, 7: 179188.View ArticleGoogle Scholar
 Hastie T, Tibshirani R, Friedman J: The elements of statistical learning; data mining, inference and prediction. 2001, New York: Springer, 193224.Google Scholar
 R Development Core Team R: A language and environment forstatistical computing. 2009, R Foundation for StatisticalComputing, Vienna, Austria, ISBN 3900051070, [http://www.Rproject.org]Google Scholar
 Campioni M, Ambrogi V, Pompeo E, Citro G, Castelli M, Spugnini EP, Gatti A, Cardelli P, Lorenzon L, Baldi A, Mineo TC: Identification of genes downregulated during lung cancer progression: a cDNA array study. J Exp Clin Cancer Res. 2008, 27: 3810.1186/175699662738.View ArticleGoogle Scholar
 Tusher VG, Tibshirani R, Chu G: Significance analysis of microarrays applied to the ionizing radiation response. Proc Natl Acad Sci USA. 2001, 98: 51165121. 10.1073/pnas.091062498.View ArticleGoogle Scholar
 Tibshirani R: Regression shrinkage and selection via the lasso. J Royal Statist Soc B. 1996, 58: 267288.Google Scholar
 Xie Y, Pan W, Jeong KS, Khodursky A: Incorporating prior information via shrinkage: a combined analysis of genomewide location data and gene expression data. Stat Med. 2007, 26: 22582275. 10.1002/sim.2703.View ArticleGoogle Scholar
 Li Y, Campbell C, Tipping M: Bayesian automatic relevance determination algorithms for classifying gene expression data. Bioinformatics. 2002, 18: 13321339. 10.1093/bioinformatics/18.10.1332.View ArticleGoogle Scholar
 DiazUriarte R: Supervised methods with genomic data: a review and cautionary view. Data analysis and visualization in genomics and proteomics. Edited by: Francisco Azuaje, Joaquín Dopazo. 2005, Hoboken: John Wiley & Sons, Ltd, 193214. full_text.View ArticleGoogle Scholar
 Tsai CA, Chen CH, Lee TC, Ho IC, Yang UC, Chen JJ: Gene selection for sample classifications in microarray experiments. DNA Cell Biol. 2004, 23: 607614. 10.1089/dna.2004.23.607.View ArticleGoogle Scholar
 Dudoit S, Fridlyand J, Speed TP: Comparison of Discrimination Methods for the Classification o Tumors Using Gene Expression Data. J Am Stat Assoc. 2002, 97: 7787. 10.1198/016214502753479248.View ArticleGoogle Scholar
 Li H, Zhang K, Jiang T: Robust and accurate cancer classification with gene expression profiling. Proc IEEE Comput Syst Bioinform Conf: 811 August 2005; California. 2005, 310321.Google Scholar
 Breiman L, Spector P: Submodel selection and evaluation in regression: the xrandom case. Int Stat Rev. 1992, 60: 291319. 10.2307/1403680.View ArticleGoogle Scholar
 Efron B: Bootstrap methods: Another look at the jackknife. Ann Stat. 1979, 7: 126. 10.1214/aos/1176344552.View ArticleGoogle Scholar
Copyright
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.