 Research
 Open Access
 Published:
Comparison of linear discriminant analysis methods for the classification of cancer based on gene expression data
Journal of Experimental & Clinical Cancer Researchvolume 28, Article number: 149 (2009)
Abstract
Background
More studies based on gene expression data have been reported in great detail, however, one major challenge for the methodologists is the choice of classification methods. The main purpose of this research was to compare the performance of linear discriminant analysis (LDA) and its modification methods for the classification of cancer based on gene expression data.
Methods
The classification performance of linear discriminant analysis (LDA) and its modification methods was evaluated by applying these methods to six public cancer gene expression datasets. These methods included linear discriminant analysis (LDA), prediction analysis for microarrays (PAM), shrinkage centroid regularized discriminant analysis (SCRDA), shrinkage linear discriminant analysis (SLDA) and shrinkage diagonal discriminant analysis (SDDA). The procedures were performed by software R 2.80.
Results
PAM picked out fewer feature genes than other methods from most datasets except from Brain dataset. For the two methods of shrinkage discriminant analysis, SLDA selected more genes than SDDA from most datasets except from 2class lung cancer dataset. When comparing SLDA with SCRDA, SLDA selected more genes than SCRDA from 2class lung cancer, SRBCT and Brain dataset, the result was opposite for the rest datasets. The average test error of LDA modification methods was lower than LDA method.
Conclusions
The classification performance of LDA modification methods was superior to that of traditional LDA with respect to the average error and there was no significant difference between theses modification methods.
Background
Conventional diagnosis of cancer has been based on the examination of the morphological appearance of stained tissue specimens in the light microscope, which is subjective and depends on highly trained pathologists. Thus, the diagnostic problems may occur due to interobserver variability. Microarrays offer the hope that cancer classification can be objective and accurate. DNA microarrays measure thousands to millions of gene expressions at the same time, which could provide the clinicians with the information to choose the most appropriate forms of treatment.
Studies on the diagnosis of cancer based on gene expression data have been reported in great detail, however, one major challenge for the methodologists is the choice of classification methods. Proposals to solve this problem have utilized many innovations including the introduction of sophisticated algorithms for support vector machines [1] and the proposal of ensemble methods such as random forests [2]. The conceptually simple approach of linear discriminant analysis (LDA) and its sibling, diagonal discriminant analysis (DDA) [3–5], remain among the most effective procedures also in the domain of highdimensional prediction. In the present study, our main focus will be solely put on the LDA part and henceforth the term "discriminant analysis" will stand for the meaning of LDA unless otherwise emphasized. The traditional way of doing discriminant analysis is introduced by R. Fisher, known as the linear discriminant analysis (LDA). Recently some modification of LDA have been advanced and gotten good performance, such as prediction analysis for microarrays (PAM), shrinkage centroid regularized discriminant analysis(SCRDA), shrinkage linear discriminant analysis(SLDA) and shrinkage diagonal discriminant analysis(SDDA). So, the main purpose of this research was to describe the performance of LDA and its modification methods for the classification of cancer based on gene expression data.
Cancer is not a single disease, there are many different kinds of cancer, arising in different organs and tissues through the accumulated mutation of multiple genes. Many previous studies only focused on one method or single dataset and gene selection is much more difficult in multiclass situations [6, 7]. Evaluation of the most commonly employed methods may give more accurate results if it is based on the collection of multiple databases from the statistical point of view.
In summary, we investigate the performance of LDA and its modification methods for the classification of cancer based on multiple gene expression datasets.
Methods
Procedure for the classification of cancer is shown as follows. First, a classifier is trained on a subset (training set) of gene expression dataset. Then, the mature classifier is used for unknown subset (test set) and predicting each observation's class. The detailed information about classification procedure is shown in Figure 1.
Datasets
Six publicly available microarray datasets [8–14] were used to test the above described methods and we call them 2class lung cancer, colon, prostate, multiclass lung cancer, SRBCT and brain following the naming there. Due to the fact that microarraybased studies may report findings that are not reproducible, after reviewing literature we selected these above public datasets with the consideration of our research topic and crosscomparison with other similar studies. The main features of these datasets are summarized in Table 1.
Data preprocessing
To avoid the noise of the dataset, preprocessing was necessary in the analysis. Absolute transformation was first performed on the original data. The data was transformed to have a mean of 0 and standard deviation of 1 after logarithmic transformation and normalization. When the original data had already experienced the above transformation, it entered next step directly.
Algorithms for feature gene selection
Notation
Let x_{ij} be the expression level of gene j in the sample i, and y_{i} be the cancer type for sample i, j = 1,...,p and response y_{i}∈{1,...,K}. Denote Y = (y_{1},...,y_{n})^{T} and x_{i} = (x_{i1},...,x_{ip})^{T}, i = 1,...,n. Gene expression data on p genes for n mRNA samples may be summarized by an n × p matrix X = (x_{ij})_{n × p}. Let C_{k} be indices of the n_{k} samples in class k, where n_{k} denotes the number of observations belonging to class k, n = n_{1}+...+n_{K}. A predictor or classifier for K tumor classes can be built from a learning set L by C(.,L); the predicted class for an observation x* is C(x*,L). The jth component of the centroid for class k is , the jth component of the overall centroid is .
Prediction analysis for microarrays/nearest shrunken centroid method, PAM/NSC
PAM [3] algorithm tries to shrink the class centroids () towards the overall centroid .
where d_{kj} is a t statistic for gene j, comparing class k to the overall centroid, and s_{j} is the pooled withinclass standard deviation for gene j:
and , s_{0} is a positive constant and usually equal to the median value of the s_{j} over the set of genes.
Equation(1) can be transformed to
PAM method shrinks each d_{kj} toward zero, and giving yielding shrunken centroids
Soft thresholding is defined by
where + means positive part (t_{+} = t if t>0 and zero otherwise). For a gene j, if d_{kj} is shrunken to zero for all classes k, then the centroid for gene j is , the same for all classes. Thus gene j does not contribute to the nearestcentroid computation. Soft threshold Δ was chosen by crossvalidation.
Shrinkage discriminant analysis, SDA
In SDA, Feature selection is controlled using higher criticism threshold (HCT) or false nondiscovery rates (FNDR) [5]. The HCT is the order statistic of the Zscore corresponding to index i maximizing , π_{i} is the pvalue associated with the ith Zscore and π_{(i)} is the i th order statistic of the collection of pvalues(1 ≤ i ≤ p). The ideal threshold optimizes the classification error. SDA consists of Shrinkage linear discriminant analysis (SLDA) and Shrinkage diagonal discriminant analysis (SDDA) [15, 16].
Shrunken centroids regularized discriminant analysis, SCRDA
There are two parameters in SCRDA [4], one is α (0<α<1), the other is soft threshold Δ. The choosing the optimal tuning parameter pairs (α, Δ) is based on crossvalidation. A "MinMin" rule was followed to identify the optimal parameter pair (α, Δ):
First, all the pairs (α, Δ) that corresponded to the minimal crossvalidation error from training samples were found.
Second, the pair or pairs that used the minimal number of genes were selected.
When there was more than one optimal pair, the average test error based on all the pairs chosen would be calculated. As traditional LDA is not suitable to deal with the "large p, small N" paradigm, so we did not adopt it to select feature genes.
Algorithms of LDA and its modification methods for classification
Linear discriminant analysis, LDA
Fisher linear discriminant analysis (FLDA, or for short, LDA) [17] projects high dimension data x into one dimension axle to find linear combinations xa with large ratios of betweengroup to withingroup sums of squares. Fisher's criteria can be defined as:
Where B and W denote the matrices of betweengroup and withingroup sums of squares and crossproducts.
Class k sample means can be gotten from learning set L, and for a new tumor sample with gene expression x*, the predicted class for x* is the class whose mean vector is closest to x* in the space of discriminant variables, that is
where , v_{ l }is eigenvector, s is the number of feature genes.
When numbers of classes K = 2, FLDA yields the same classifier as the maximum likelihood (ML) discriminant rule for multivariate normal class densities with the same covariance matrix.
Prediction analysis for microarrays/nearest shrunken centroid method, PAM/NSC
PAM [3] assumes that genes are independent, the target classes correspond to individual (single) clusters and classify test samples to the nearest shrunken centroid, again standardizing by s_{j} +s_{0}. The relative number of samples in each class is corrected at the same time. For a test sample (a vector) with expression levels x*, the discriminant score for class k was defined by,
where π_{k} = n_{k}/n or π_{k} = 1/K is class prior probability, . This prior probability gives the overall frequency of class k in the population. The classification rule is
Here was the diagonal matrix taking the diagonal elements of . If the smallest distances are close and hence ambiguous, the prior correction gives a preference for larger classes, because they potentially account for more errors.
Shrinkage discriminant analysis, SDA
The corresponding discriminant score [5] was defined by
Where , P = (ρ_{ij}) and
Algorithm of SCRDA
A new test sample was classified by regularized discriminant function [4],
Covariance was estimated by
where 0 ≤ α ≤ 1
In the same way, sample correlation matrix was substituted by .
Then the regularized sample covariance matrix was computed by
Study design and program realization
We used 10fold crossvalidation (CV) to divide the preprocessed dataset into 10 approximately equalsize parts by random sampling. It worked as follows: we fit the model on 90% of the samples and then predicted the class labels of the remaining 10% (the test samples). This procedure was repeated 10 times to avoid overlapping test sets, with each part playing the role of the test samples and the errors on all 10 parts added together to compute the overall error [18]. R software (version 2.80) with packages MASS, pamr, RDA, SDA was used for the realization of the above described methods [19]. A tolerance value was set to decide if a matrix is singular. If variable had withingroup variance less than tol^2, LDA fitting iteration would stop and report the variable as constant. In practice, we set a very small tolerance value 1 × 10^{14}, and no singular was detected.
Results
Feature genes selection
As shown in Table 2, PAM picked out fewer feature genes than other methods from most datasets except from Brain dataset. For the two methods of shrinkage discriminant analysis, SLDA selected more genes than SDDA from most datasets except from 2class lung cancer dataset. When comparing SLDA with SCRDA, SLDA selected more genes than SCRDA from 2class lung cancer, SRBCT and Brain dataset, the result was opposite for the rest datasets.
Performance comparison for methods based on different datasets
The performance of the methods described above was compared by average test error using 10fold cross validation. We ran 10 cycles of 10fold cross validation. The average test errors were calculated based on the incorrectness of the classification of each testing samples. For example, for the 2class lung cancer dataset, using the LDA method based on PAM as the feature gene method, 30 samples out of 100 sample test sets were incorrectly classified, resulting in an average test error of 0.30.
The significance of the performance difference between these methods was judged depending on whether or not their 95% confidence intervals of accuracy overlapped. Here, if the upper limit was greater than 100%, it was treated as 100%. If two methods had nonoverlapping confidence intervals, their performances were significantly different. The bold fonts in Table 3 shows the performances of PAM, SDDA, SLDA and SCRDA, when they were used both for feature gene selection and classification. As shown in Table 3, the performance of LDA modification methods is superior to traditional LDA method, while there is no significant difference between theses modification methods (Figure 2).
Discussion
Microarrays are capable of determining the expression levels of thousands of genes simultaneously and hold great promise to facilitate the discovery of new biological knowledge [20]. One feature of microarray data is that the number of variables p (genes) far exceeds the number of samples N. In statistical terms, it is called 'large p, small N' problem. Standard statistical methods in classification do not work well or even at all, so improvement or modification of existing statistical methods is needed to prevent overfitting and produce more reliable estimations. Some adhoc shrinkage methods have been proposed to utilize the shrinkage ideas and prove to be useful in empirical studies [21–23]. Distinguishing normal samples from tumor samples is essential for successful diagnosis or treatment of cancer. And, another important problem is in characterizing multiple types of tumors. The problem of multiple classifications has recently received more attention in the context of DNA microarrays. In the present study, we first presented an evaluation of the performance of LDA and its modification methods for classification with 6 public microarray datasets.
The gene selection method [6, 24, 25], the number of selected genes and the classification method are three critical issues for the performance of a sample classification. Feature selection techniques can be organized into three categories, filter methods, wrapper methods and embedded methods. LDA and its modification methods belong to wrapper methods which embed the model hypothesis search within the feature subset search. In the present study, different numbers of gene have been selected by different LDA modification methods. There is no theoretical estimation of the optimal number of selected genes and the optimal gene set can vary from data to data [26]. So we did not focus on the combination of the optimal gene set by one feature gene selection method and one classification algorithm. In this paper we just describe the performance of LDA and its modification methods under the same selection method in different microarray dataset.
Various statistical and machine learning methods have been used to analyze the high dimensional data for cancer classification. These methods have been shown to have statistical and clinical relevance in cancer detection for a variety of tumor types. In this study, it has been shown that LDA modification methods have better performance than traditional LDA under the same gene selection criterion. Dudoit also reported that simple classifiers such as DLDA and Nearest Neighbor performed remarkably well compared with more sophisticated ones, such as aggregated classification trees [27]. It indicates that LDA modification methods did a good job in some situations. Zhang et al[28] developed a fast algorithm of generalized linear discriminant analysis (GLDA) and applied it to seven public cancer datasets. Their study included 4 same datasets (Colon, Prostate, SRBCT and Brain) as those in our study and adopted a 3fold crossvalidation design. The average test errors of our study were less than those of their study, while there was no statistical significance of the difference. The results reported by Guo et al[4] are of concordance with ours except for the colon dataset. Their study also included the above mentioned 4 same datasets and they found that in the colon dataset the average test error of SCRDA was as same as PAM, while in the present study we found that the average test error of SCRDA was slightly less than that of PAM.
There are several interesting problems that remain to be addressed. A question is raised that when comparing the predictive performance of different classification methods on different microarray data, is there any difference between various methods, such as leaveoneout crossvalidation and bootstrap [29, 30]? And another interesting further step might be a preanalysis of the data to choose a suitable gene selection method. Despite the great promise of discriminant analysis in the field of microarray technology, the complexity and the multiple choices of the available methods are quite difficult to the bench clinicians. This may influence the clinicians' adoption of microarray data based results when making decision on diagnosis or treatment. Microarray data's widespread clinical relevance and applicability still need to be resolved.
Conclusions
An extensive survey in building classification models from microarray data with LDA and its modification methods has been conducted in the present study. The study showed that the modification methods are superior to LDA in the prediction accuracy.
Abbreviations
 CV:

Crossvalidation
 DDA:

diagonal discriminant analysis
 FNDR:

False nondiscovery rates
 GLDA:

generalized linear discriminant analysis
 HCT:

Higher criticism threshold
 LDA:

linear discriminant analysis
 NSC:

nearest shrunken centroid method
 PAM:

prediction analysis for microarrays
 SCRDA:

Shrinkage centroid regularized discriminant analysis
 SDA:

Shrinkage discriminant analysis
 SDDA:

Shrinkage diagonal discriminant analysis
 SLDA:

Shrinkage linear discriminant analysis.
References
 1.
Guyon I, Weston J, Barnhill , Vapnik V: Gene Selection for Cancer Classification using Support Vector Machines. Mach Learn. 2002, 46: 389422. 10.1023/A:1012487302797.
 2.
Breiman L: Random Forests. Mach Learn. 2001, 45: 532. 10.1023/A:1010933404324.
 3.
Tusher VG, Tibshirani R, Chu G: Significance analysis of microarrays applied to the ionizing radiation response. Proc Natl Acad Sci USA. 2001, 98: 51165121. 10.1073/pnas.091062498.
 4.
Guo Y, Hastie T, Tibshirani R: Regularized linear discriminant analysis and its application in microarrays. Biostatistics. 2005, 8: 86100. 10.1093/biostatistics/kxj035.
 5.
Schäfer J, Strimmer K: A shrinkage approach to largescale covariance matrix estimation and implications for functional genomics. Stat Appl Genet Mol Biol. 2005, 4:
 6.
Yeung KY, Bumgarner RE, Raftery AE: Bayesian model averaging: development of an improved multiclass, gene selection and classification tool for microarray data. Bioinformatics. 2005, 21: 23942402. 10.1093/bioinformatics/bti319.
 7.
Li T, Zhang C, Ogihara M: A comparative study of feature selection and multiclass classification methods for tissue classification based on gene expression. Bioinformatics. 2004, 20: 24292437. 10.1093/bioinformatics/bth267.
 8.
Gordon GJ, Jensen RV, Hsiao LL, Gullans SR, Blumenstock JE, Ramaswamy S, Richards WG, Sugarbaker DJ, Bueno R: Translation of microarray data into clinically relevant cancer diagnostic tests using gene expression ratios in lung cancer and mesothelioma. Cancer Res. 2002, 62: 49634967.
 9.
Alon U, Barkai N, Notterman DA, Gish K, Ybarra S, Mack D, Levine AJ: Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proc Natl Acad Sci USA. 1999, 96: 67456750. 10.1073/pnas.96.12.6745.
 10.
Singh D, Febbo PG, Ross K, Jackson DG, Manola J, Ladd C, Tamayo P, Renshaw AA, D'Amico AV, Richie JP, Lander ES, Loda M, Kantoff PW, Golub TR, Sellers WR: Gene expression correlates of clinical prostate cancer behavior. Cancer Cell. 2002, 1: 203209. 10.1016/S15356108(02)000302.
 11.
Bhattacharjee A, Richards WG, Staunton J, Li C, Monti S, Vasa P, Ladd C, Beheshti J, Bueno R, Gillette M, Loda M, Weber G, Mark EJ, Lander ES, Wong W, Johnson BE, Golub TR, Sugarbaker DJ, Meyerson M: Classification of human lung carcinomas by mRNA expressionprofiling reveals distinct adenocarcinoma subclasses. Proc Natl Acad Sci USA. 2001, 98: 1379013795. 10.1073/pnas.191502998.
 12.
Parmigiani G, GarrettMayer ES, Anbazhagan R, Gabrielson E: A crossstudy comparison of gene expression studies for the molecular classification of lung cancer. Clin Cancer Res. 2004, 10: 29222927. 10.1158/10780432.CCR030490.
 13.
Khan J, Wei JS, Ringnér M, Saal LH, Ladanyi M, Westermann F, Berthold F, Schwab M, Antonescu CR, Peterson C, Meltzer PS: Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks. Nat Med. 2001, 7: 673679. 10.1038/89044.
 14.
Pomeroy SL, Tamayo P, Gaasenbeek M, Sturla LM, Angelo M, McLaughlin ME, Kim JY, Goumnerova LC, Black PM, Lau C, Allen JC, Zagzag D, Olson JM, Curran T, Wetmore C, Biegel JA, Poggio T, Mukherjee S, Rifkin R, Califano A, Stolovitzky G, Louis DN, Mesirov JP, Lander ES, Golub TR: Prediction of central nervous system embryonal tumour outcome based on gene expression. Nature. 2002, 415: 436442. 10.1038/415436a.
 15.
OpgenRhein R, Strimmer K: Accurate ranking of differentially expressed genes by a distributionfree shrinkage approach. Stat Appl Genet Mol Biol. 2007, 6: Article9
 16.
Schäfer J, Strimmer K: A shrinkage approach to largescale covariance matrix estimation and implications for functional genomics. Stat Appl Genet Mol Biol. 2005, 4: Article32
 17.
Fisher RA: The Use of Multiple Measurements in Taxonomic Problems. Annuals of Eugenics. 1936, 7: 179188.
 18.
Hastie T, Tibshirani R, Friedman J: The elements of statistical learning; data mining, inference and prediction. 2001, New York: Springer, 193224.
 19.
R Development Core Team R: A language and environment forstatistical computing. 2009, R Foundation for StatisticalComputing, Vienna, Austria, ISBN 3900051070, [http://www.Rproject.org]
 20.
Campioni M, Ambrogi V, Pompeo E, Citro G, Castelli M, Spugnini EP, Gatti A, Cardelli P, Lorenzon L, Baldi A, Mineo TC: Identification of genes downregulated during lung cancer progression: a cDNA array study. J Exp Clin Cancer Res. 2008, 27: 3810.1186/175699662738.
 21.
Tusher VG, Tibshirani R, Chu G: Significance analysis of microarrays applied to the ionizing radiation response. Proc Natl Acad Sci USA. 2001, 98: 51165121. 10.1073/pnas.091062498.
 22.
Tibshirani R: Regression shrinkage and selection via the lasso. J Royal Statist Soc B. 1996, 58: 267288.
 23.
Xie Y, Pan W, Jeong KS, Khodursky A: Incorporating prior information via shrinkage: a combined analysis of genomewide location data and gene expression data. Stat Med. 2007, 26: 22582275. 10.1002/sim.2703.
 24.
Li Y, Campbell C, Tipping M: Bayesian automatic relevance determination algorithms for classifying gene expression data. Bioinformatics. 2002, 18: 13321339. 10.1093/bioinformatics/18.10.1332.
 25.
DiazUriarte R: Supervised methods with genomic data: a review and cautionary view. Data analysis and visualization in genomics and proteomics. Edited by: Francisco Azuaje, Joaquín Dopazo. 2005, Hoboken: John Wiley & Sons, Ltd, 193214. full_text.
 26.
Tsai CA, Chen CH, Lee TC, Ho IC, Yang UC, Chen JJ: Gene selection for sample classifications in microarray experiments. DNA Cell Biol. 2004, 23: 607614. 10.1089/dna.2004.23.607.
 27.
Dudoit S, Fridlyand J, Speed TP: Comparison of Discrimination Methods for the Classification o Tumors Using Gene Expression Data. J Am Stat Assoc. 2002, 97: 7787. 10.1198/016214502753479248.
 28.
Li H, Zhang K, Jiang T: Robust and accurate cancer classification with gene expression profiling. Proc IEEE Comput Syst Bioinform Conf: 811 August 2005; California. 2005, 310321.
 29.
Breiman L, Spector P: Submodel selection and evaluation in regression: the xrandom case. Int Stat Rev. 1992, 60: 291319. 10.2307/1403680.
 30.
Efron B: Bootstrap methods: Another look at the jackknife. Ann Stat. 1979, 7: 126. 10.1214/aos/1176344552.
Acknowledgements
This study was partially supported by Provincial Education Department of Liaoning (No.2008S232), Natural Science Foundation of Liaoning province (No.20072103) and China Medical Board (No.00726.). The authors are most grateful to the contributors of the datasets and R statistical software. The authors thank the two reviewers for their insightful comments which led to an improved version of the manuscript.
Author information
Additional information
Competing interests
The authors declare that they have no competing interests.
Authors' contributions
DH conceived the study and drafted the manuscript. DH and YQ performed the analyses. MH provided guidance and discussion on the methodology. BZ attracted partial funding and participated in the design of the analysis strategy. All authors read and approved the final version of this manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
Rights and permissions
About this article
Received
Accepted
Published
DOI
Keywords
 Linear Discriminant Analysis
 Modification Method
 Gene Expression Dataset
 Gene Selection Method
 Fisher Linear Discriminant Analysis