Skip to main content

Comparison of linear discriminant analysis methods for the classification of cancer based on gene expression data



More studies based on gene expression data have been reported in great detail, however, one major challenge for the methodologists is the choice of classification methods. The main purpose of this research was to compare the performance of linear discriminant analysis (LDA) and its modification methods for the classification of cancer based on gene expression data.


The classification performance of linear discriminant analysis (LDA) and its modification methods was evaluated by applying these methods to six public cancer gene expression datasets. These methods included linear discriminant analysis (LDA), prediction analysis for microarrays (PAM), shrinkage centroid regularized discriminant analysis (SCRDA), shrinkage linear discriminant analysis (SLDA) and shrinkage diagonal discriminant analysis (SDDA). The procedures were performed by software R 2.80.


PAM picked out fewer feature genes than other methods from most datasets except from Brain dataset. For the two methods of shrinkage discriminant analysis, SLDA selected more genes than SDDA from most datasets except from 2-class lung cancer dataset. When comparing SLDA with SCRDA, SLDA selected more genes than SCRDA from 2-class lung cancer, SRBCT and Brain dataset, the result was opposite for the rest datasets. The average test error of LDA modification methods was lower than LDA method.


The classification performance of LDA modification methods was superior to that of traditional LDA with respect to the average error and there was no significant difference between theses modification methods.


Conventional diagnosis of cancer has been based on the examination of the morphological appearance of stained tissue specimens in the light microscope, which is subjective and depends on highly trained pathologists. Thus, the diagnostic problems may occur due to inter-observer variability. Microarrays offer the hope that cancer classification can be objective and accurate. DNA microarrays measure thousands to millions of gene expressions at the same time, which could provide the clinicians with the information to choose the most appropriate forms of treatment.

Studies on the diagnosis of cancer based on gene expression data have been reported in great detail, however, one major challenge for the methodologists is the choice of classification methods. Proposals to solve this problem have utilized many innovations including the introduction of sophisticated algorithms for support vector machines [1] and the proposal of ensemble methods such as random forests [2]. The conceptually simple approach of linear discriminant analysis (LDA) and its sibling, diagonal discriminant analysis (DDA) [35], remain among the most effective procedures also in the domain of high-dimensional prediction. In the present study, our main focus will be solely put on the LDA part and henceforth the term "discriminant analysis" will stand for the meaning of LDA unless otherwise emphasized. The traditional way of doing discriminant analysis is introduced by R. Fisher, known as the linear discriminant analysis (LDA). Recently some modification of LDA have been advanced and gotten good performance, such as prediction analysis for microarrays (PAM), shrinkage centroid regularized discriminant analysis(SCRDA), shrinkage linear discriminant analysis(SLDA) and shrinkage diagonal discriminant analysis(SDDA). So, the main purpose of this research was to describe the performance of LDA and its modification methods for the classification of cancer based on gene expression data.

Cancer is not a single disease, there are many different kinds of cancer, arising in different organs and tissues through the accumulated mutation of multiple genes. Many previous studies only focused on one method or single dataset and gene selection is much more difficult in multi-class situations [6, 7]. Evaluation of the most commonly employed methods may give more accurate results if it is based on the collection of multiple databases from the statistical point of view.

In summary, we investigate the performance of LDA and its modification methods for the classification of cancer based on multiple gene expression datasets.


Procedure for the classification of cancer is shown as follows. First, a classifier is trained on a subset (training set) of gene expression dataset. Then, the mature classifier is used for unknown subset (test set) and predicting each observation's class. The detailed information about classification procedure is shown in Figure 1.

Figure 1

Framework for the procedure of classification.


Six publicly available microarray datasets [814] were used to test the above described methods and we call them 2-class lung cancer, colon, prostate, multi-class lung cancer, SRBCT and brain following the naming there. Due to the fact that microarray-based studies may report findings that are not reproducible, after reviewing literature we selected these above public datasets with the consideration of our research topic and cross-comparison with other similar studies. The main features of these datasets are summarized in Table 1.

Table 1 Characteristics of the six microarray datasets used

Data pre-processing

To avoid the noise of the dataset, pre-processing was necessary in the analysis. Absolute transformation was first performed on the original data. The data was transformed to have a mean of 0 and standard deviation of 1 after logarithmic transformation and normalization. When the original data had already experienced the above transformation, it entered next step directly.

Algorithms for feature gene selection


Let xij be the expression level of gene j in the sample i, and yi be the cancer type for sample i, j = 1,...,p and response yi{1,...,K}. Denote Y = (y1,...,yn)T and xi = (xi1,...,xip)T, i = 1,...,n. Gene expression data on p genes for n mRNA samples may be summarized by an n × p matrix X = (xij)n × p. Let Ck be indices of the nk samples in class k, where nk denotes the number of observations belonging to class k, n = n1+...+nK. A predictor or classifier for K tumor classes can be built from a learning set L by C(.,L); the predicted class for an observation x* is C(x*,L). The jth component of the centroid for class k is , the jth component of the overall centroid is .

Prediction analysis for microarrays/nearest shrunken centroid method, PAM/NSC

PAM [3] algorithm tries to shrink the class centroids () towards the overall centroid .


where dkj is a t statistic for gene j, comparing class k to the overall centroid, and sj is the pooled within-class standard deviation for gene j:


and , s0 is a positive constant and usually equal to the median value of the sj over the set of genes.

Equation(1) can be transformed to


PAM method shrinks each dkj toward zero, and giving yielding shrunken centroids


Soft thresholding is defined by


where + means positive part (t+ = t if t>0 and zero otherwise). For a gene j, if dkj is shrunken to zero for all classes k, then the centroid for gene j is , the same for all classes. Thus gene j does not contribute to the nearest-centroid computation. Soft threshold Δ was chosen by cross-validation.

Shrinkage discriminant analysis, SDA

In SDA, Feature selection is controlled using higher criticism threshold (HCT) or false non-discovery rates (FNDR) [5]. The HCT is the order statistic of the Z-score corresponding to index i maximizing , πi is the p-value associated with the ith Z-score and π(i) is the i th order statistic of the collection of p-values(1 ≤ i ≤ p). The ideal threshold optimizes the classification error. SDA consists of Shrinkage linear discriminant analysis (SLDA) and Shrinkage diagonal discriminant analysis (SDDA) [15, 16].

Shrunken centroids regularized discriminant analysis, SCRDA

There are two parameters in SCRDA [4], one is α (0<α<1), the other is soft threshold Δ. The choosing the optimal tuning parameter pairs (α, Δ) is based on cross-validation. A "Min-Min" rule was followed to identify the optimal parameter pair (α, Δ):

First, all the pairs (α, Δ) that corresponded to the minimal cross-validation error from training samples were found.

Second, the pair or pairs that used the minimal number of genes were selected.

When there was more than one optimal pair, the average test error based on all the pairs chosen would be calculated. As traditional LDA is not suitable to deal with the "large p, small N" paradigm, so we did not adopt it to select feature genes.

Algorithms of LDA and its modification methods for classification

Linear discriminant analysis, LDA

Fisher linear discriminant analysis (FLDA, or for short, LDA) [17] projects high dimension data x into one dimension axle to find linear combinations xa with large ratios of between-group to within-group sums of squares. Fisher's criteria can be defined as:


Where B and W denote the matrices of between-group and within-group sums of squares and cross-products.

Class k sample means can be gotten from learning set L, and for a new tumor sample with gene expression x*, the predicted class for x* is the class whose mean vector is closest to x* in the space of discriminant variables, that is


where , v l is eigenvector, s is the number of feature genes.

When numbers of classes K = 2, FLDA yields the same classifier as the maximum likelihood (ML) discriminant rule for multivariate normal class densities with the same covariance matrix.

Prediction analysis for microarrays/nearest shrunken centroid method, PAM/NSC

PAM [3] assumes that genes are independent, the target classes correspond to individual (single) clusters and classify test samples to the nearest shrunken centroid, again standardizing by sj +s0. The relative number of samples in each class is corrected at the same time. For a test sample (a vector) with expression levels x*, the discriminant score for class k was defined by,


where πk = nk/n or πk = 1/K is class prior probability, . This prior probability gives the overall frequency of class k in the population. The classification rule is


Here was the diagonal matrix taking the diagonal elements of . If the smallest distances are close and hence ambiguous, the prior correction gives a preference for larger classes, because they potentially account for more errors.

Shrinkage discriminant analysis, SDA

The corresponding discriminant score [5] was defined by


Where , P = (ρij) and

Algorithm of SCRDA

A new test sample was classified by regularized discriminant function [4],


Covariance was estimated by


where 0 ≤ α ≤ 1

In the same way, sample correlation matrix was substituted by .

Then the regularized sample covariance matrix was computed by

Study design and program realization

We used 10-fold cross-validation (CV) to divide the pre-processed dataset into 10 approximately equal-size parts by random sampling. It worked as follows: we fit the model on 90% of the samples and then predicted the class labels of the remaining 10% (the test samples). This procedure was repeated 10 times to avoid overlapping test sets, with each part playing the role of the test samples and the errors on all 10 parts added together to compute the overall error [18]. R software (version 2.80) with packages MASS, pamr, RDA, SDA was used for the realization of the above described methods [19]. A tolerance value was set to decide if a matrix is singular. If variable had within-group variance less than tol^2, LDA fitting iteration would stop and report the variable as constant. In practice, we set a very small tolerance value 1 × 10-14, and no singular was detected.


Feature genes selection

As shown in Table 2, PAM picked out fewer feature genes than other methods from most datasets except from Brain dataset. For the two methods of shrinkage discriminant analysis, SLDA selected more genes than SDDA from most datasets except from 2-class lung cancer dataset. When comparing SLDA with SCRDA, SLDA selected more genes than SCRDA from 2-class lung cancer, SRBCT and Brain dataset, the result was opposite for the rest datasets.

Table 2 Numbers of feature genes selected by 4 methods for each dataset

Performance comparison for methods based on different datasets

The performance of the methods described above was compared by average test error using 10-fold cross validation. We ran 10 cycles of 10-fold cross validation. The average test errors were calculated based on the incorrectness of the classification of each testing samples. For example, for the 2-class lung cancer dataset, using the LDA method based on PAM as the feature gene method, 30 samples out of 100 sample test sets were incorrectly classified, resulting in an average test error of 0.30.

The significance of the performance difference between these methods was judged depending on whether or not their 95% confidence intervals of accuracy overlapped. Here, if the upper limit was greater than 100%, it was treated as 100%. If two methods had non-overlapping confidence intervals, their performances were significantly different. The bold fonts in Table 3 shows the performances of PAM, SDDA, SLDA and SCRDA, when they were used both for feature gene selection and classification. As shown in Table 3, the performance of LDA modification methods is superior to traditional LDA method, while there is no significant difference between theses modification methods (Figure 2).

Table 3 Average test error of LDA and its modification methods (10 cycles of 10-fold cross validation)
Figure 2

Comparison of classification performance for different datasets. The y-axis shows the average error and the x-axis indicates the gene selection methods: PAM, SDDA, SLDA and SCRDA. Error bars (± 1.96 SE) are provided for the classification methods.


Microarrays are capable of determining the expression levels of thousands of genes simultaneously and hold great promise to facilitate the discovery of new biological knowledge [20]. One feature of microarray data is that the number of variables p (genes) far exceeds the number of samples N. In statistical terms, it is called 'large p, small N' problem. Standard statistical methods in classification do not work well or even at all, so improvement or modification of existing statistical methods is needed to prevent over-fitting and produce more reliable estimations. Some ad-hoc shrinkage methods have been proposed to utilize the shrinkage ideas and prove to be useful in empirical studies [2123]. Distinguishing normal samples from tumor samples is essential for successful diagnosis or treatment of cancer. And, another important problem is in characterizing multiple types of tumors. The problem of multiple classifications has recently received more attention in the context of DNA microarrays. In the present study, we first presented an evaluation of the performance of LDA and its modification methods for classification with 6 public microarray datasets.

The gene selection method [6, 24, 25], the number of selected genes and the classification method are three critical issues for the performance of a sample classification. Feature selection techniques can be organized into three categories, filter methods, wrapper methods and embedded methods. LDA and its modification methods belong to wrapper methods which embed the model hypothesis search within the feature subset search. In the present study, different numbers of gene have been selected by different LDA modification methods. There is no theoretical estimation of the optimal number of selected genes and the optimal gene set can vary from data to data [26]. So we did not focus on the combination of the optimal gene set by one feature gene selection method and one classification algorithm. In this paper we just describe the performance of LDA and its modification methods under the same selection method in different microarray dataset.

Various statistical and machine learning methods have been used to analyze the high dimensional data for cancer classification. These methods have been shown to have statistical and clinical relevance in cancer detection for a variety of tumor types. In this study, it has been shown that LDA modification methods have better performance than traditional LDA under the same gene selection criterion. Dudoit also reported that simple classifiers such as DLDA and Nearest Neighbor performed remarkably well compared with more sophisticated ones, such as aggregated classification trees [27]. It indicates that LDA modification methods did a good job in some situations. Zhang et al[28] developed a fast algorithm of generalized linear discriminant analysis (GLDA) and applied it to seven public cancer datasets. Their study included 4 same datasets (Colon, Prostate, SRBCT and Brain) as those in our study and adopted a 3-fold cross-validation design. The average test errors of our study were less than those of their study, while there was no statistical significance of the difference. The results reported by Guo et al[4] are of concordance with ours except for the colon dataset. Their study also included the above mentioned 4 same datasets and they found that in the colon dataset the average test error of SCRDA was as same as PAM, while in the present study we found that the average test error of SCRDA was slightly less than that of PAM.

There are several interesting problems that remain to be addressed. A question is raised that when comparing the predictive performance of different classification methods on different microarray data, is there any difference between various methods, such as leave-one-out cross-validation and bootstrap [29, 30]? And another interesting further step might be a pre-analysis of the data to choose a suitable gene selection method. Despite the great promise of discriminant analysis in the field of microarray technology, the complexity and the multiple choices of the available methods are quite difficult to the bench clinicians. This may influence the clinicians' adoption of microarray data based results when making decision on diagnosis or treatment. Microarray data's widespread clinical relevance and applicability still need to be resolved.


An extensive survey in building classification models from microarray data with LDA and its modification methods has been conducted in the present study. The study showed that the modification methods are superior to LDA in the prediction accuracy.





diagonal discriminant analysis


False non-discovery rates


generalized linear discriminant analysis


Higher criticism threshold


linear discriminant analysis


nearest shrunken centroid method


prediction analysis for microarrays


Shrinkage centroid regularized discriminant analysis


Shrinkage discriminant analysis


Shrinkage diagonal discriminant analysis


Shrinkage linear discriminant analysis.


  1. 1.

    Guyon I, Weston J, Barnhill , Vapnik V: Gene Selection for Cancer Classification using Support Vector Machines. Mach Learn. 2002, 46: 389-422. 10.1023/A:1012487302797.

    Article  Google Scholar 

  2. 2.

    Breiman L: Random Forests. Mach Learn. 2001, 45: 5-32. 10.1023/A:1010933404324.

    Article  Google Scholar 

  3. 3.

    Tusher VG, Tibshirani R, Chu G: Significance analysis of microarrays applied to the ionizing radiation response. Proc Natl Acad Sci USA. 2001, 98: 5116-5121. 10.1073/pnas.091062498.

    CAS  Article  Google Scholar 

  4. 4.

    Guo Y, Hastie T, Tibshirani R: Regularized linear discriminant analysis and its application in microarrays. Biostatistics. 2005, 8: 86-100. 10.1093/biostatistics/kxj035.

    Article  Google Scholar 

  5. 5.

    Schäfer J, Strimmer K: A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics. Stat Appl Genet Mol Biol. 2005, 4:

    Google Scholar 

  6. 6.

    Yeung KY, Bumgarner RE, Raftery AE: Bayesian model averaging: development of an improved multi-class, gene selection and classification tool for microarray data. Bioinformatics. 2005, 21: 2394-2402. 10.1093/bioinformatics/bti319.

    CAS  Article  Google Scholar 

  7. 7.

    Li T, Zhang C, Ogihara M: A comparative study of feature selection and multiclass classification methods for tissue classification based on gene expression. Bioinformatics. 2004, 20: 2429-2437. 10.1093/bioinformatics/bth267.

    CAS  Article  Google Scholar 

  8. 8.

    Gordon GJ, Jensen RV, Hsiao LL, Gullans SR, Blumenstock JE, Ramaswamy S, Richards WG, Sugarbaker DJ, Bueno R: Translation of microarray data into clinically relevant cancer diagnostic tests using gene expression ratios in lung cancer and mesothelioma. Cancer Res. 2002, 62: 4963-4967.

    CAS  Google Scholar 

  9. 9.

    Alon U, Barkai N, Notterman DA, Gish K, Ybarra S, Mack D, Levine AJ: Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proc Natl Acad Sci USA. 1999, 96: 6745-6750. 10.1073/pnas.96.12.6745.

    CAS  Article  Google Scholar 

  10. 10.

    Singh D, Febbo PG, Ross K, Jackson DG, Manola J, Ladd C, Tamayo P, Renshaw AA, D'Amico AV, Richie JP, Lander ES, Loda M, Kantoff PW, Golub TR, Sellers WR: Gene expression correlates of clinical prostate cancer behavior. Cancer Cell. 2002, 1: 203-209. 10.1016/S1535-6108(02)00030-2.

    CAS  Article  Google Scholar 

  11. 11.

    Bhattacharjee A, Richards WG, Staunton J, Li C, Monti S, Vasa P, Ladd C, Beheshti J, Bueno R, Gillette M, Loda M, Weber G, Mark EJ, Lander ES, Wong W, Johnson BE, Golub TR, Sugarbaker DJ, Meyerson M: Classification of human lung carcinomas by mRNA expressionprofiling reveals distinct adenocarcinoma subclasses. Proc Natl Acad Sci USA. 2001, 98: 13790-13795. 10.1073/pnas.191502998.

    CAS  Article  Google Scholar 

  12. 12.

    Parmigiani G, Garrett-Mayer ES, Anbazhagan R, Gabrielson E: A cross-study comparison of gene expression studies for the molecular classification of lung cancer. Clin Cancer Res. 2004, 10: 2922-2927. 10.1158/1078-0432.CCR-03-0490.

    CAS  Article  Google Scholar 

  13. 13.

    Khan J, Wei JS, Ringnér M, Saal LH, Ladanyi M, Westermann F, Berthold F, Schwab M, Antonescu CR, Peterson C, Meltzer PS: Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks. Nat Med. 2001, 7: 673-679. 10.1038/89044.

    CAS  Article  Google Scholar 

  14. 14.

    Pomeroy SL, Tamayo P, Gaasenbeek M, Sturla LM, Angelo M, McLaughlin ME, Kim JY, Goumnerova LC, Black PM, Lau C, Allen JC, Zagzag D, Olson JM, Curran T, Wetmore C, Biegel JA, Poggio T, Mukherjee S, Rifkin R, Califano A, Stolovitzky G, Louis DN, Mesirov JP, Lander ES, Golub TR: Prediction of central nervous system embryonal tumour outcome based on gene expression. Nature. 2002, 415: 436-442. 10.1038/415436a.

    CAS  Article  Google Scholar 

  15. 15.

    Opgen-Rhein R, Strimmer K: Accurate ranking of differentially expressed genes by a distribution-free shrinkage approach. Stat Appl Genet Mol Biol. 2007, 6: Article9-

    Google Scholar 

  16. 16.

    Schäfer J, Strimmer K: A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics. Stat Appl Genet Mol Biol. 2005, 4: Article32-

    Google Scholar 

  17. 17.

    Fisher RA: The Use of Multiple Measurements in Taxonomic Problems. Annuals of Eugenics. 1936, 7: 179-188.

    Article  Google Scholar 

  18. 18.

    Hastie T, Tibshirani R, Friedman J: The elements of statistical learning; data mining, inference and prediction. 2001, New York: Springer, 193-224.

    Google Scholar 

  19. 19.

    R Development Core Team R: A language and environment forstatistical computing. 2009, R Foundation for StatisticalComputing, Vienna, Austria, ISBN 3-900051-07-0, []

    Google Scholar 

  20. 20.

    Campioni M, Ambrogi V, Pompeo E, Citro G, Castelli M, Spugnini EP, Gatti A, Cardelli P, Lorenzon L, Baldi A, Mineo TC: Identification of genes down-regulated during lung cancer progression: a cDNA array study. J Exp Clin Cancer Res. 2008, 27: 38-10.1186/1756-9966-27-38.

    Article  Google Scholar 

  21. 21.

    Tusher VG, Tibshirani R, Chu G: Significance analysis of microarrays applied to the ionizing radiation response. Proc Natl Acad Sci USA. 2001, 98: 5116-5121. 10.1073/pnas.091062498.

    CAS  Article  Google Scholar 

  22. 22.

    Tibshirani R: Regression shrinkage and selection via the lasso. J Royal Statist Soc B. 1996, 58: 267-288.

    Google Scholar 

  23. 23.

    Xie Y, Pan W, Jeong KS, Khodursky A: Incorporating prior information via shrinkage: a combined analysis of genome-wide location data and gene expression data. Stat Med. 2007, 26: 2258-2275. 10.1002/sim.2703.

    Article  Google Scholar 

  24. 24.

    Li Y, Campbell C, Tipping M: Bayesian automatic relevance determination algorithms for classifying gene expression data. Bioinformatics. 2002, 18: 1332-1339. 10.1093/bioinformatics/18.10.1332.

    CAS  Article  Google Scholar 

  25. 25.

    Diaz-Uriarte R: Supervised methods with genomic data: a review and cautionary view. Data analysis and visualization in genomics and proteomics. Edited by: Francisco Azuaje, Joaquín Dopazo. 2005, Hoboken: John Wiley & Sons, Ltd, 193-214. full_text.

    Google Scholar 

  26. 26.

    Tsai CA, Chen CH, Lee TC, Ho IC, Yang UC, Chen JJ: Gene selection for sample classifications in microarray experiments. DNA Cell Biol. 2004, 23: 607-614. 10.1089/dna.2004.23.607.

    CAS  Article  Google Scholar 

  27. 27.

    Dudoit S, Fridlyand J, Speed TP: Comparison of Discrimination Methods for the Classification o Tumors Using Gene Expression Data. J Am Stat Assoc. 2002, 97: 77-87. 10.1198/016214502753479248.

    CAS  Article  Google Scholar 

  28. 28.

    Li H, Zhang K, Jiang T: Robust and accurate cancer classification with gene expression profiling. Proc IEEE Comput Syst Bioinform Conf: 8-11 August 2005; California. 2005, 310-321.

    Google Scholar 

  29. 29.

    Breiman L, Spector P: Submodel selection and evaluation in regression: the x-random case. Int Stat Rev. 1992, 60: 291-319. 10.2307/1403680.

    Article  Google Scholar 

  30. 30.

    Efron B: Bootstrap methods: Another look at the jackknife. Ann Stat. 1979, 7: 1-26. 10.1214/aos/1176344552.

    Article  Google Scholar 

Download references


This study was partially supported by Provincial Education Department of Liaoning (No.2008S232), Natural Science Foundation of Liaoning province (No.20072103) and China Medical Board (No.00726.). The authors are most grateful to the contributors of the datasets and R statistical software. The authors thank the two reviewers for their insightful comments which led to an improved version of the manuscript.

Author information



Corresponding author

Correspondence to Baosen Zhou.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

DH conceived the study and drafted the manuscript. DH and YQ performed the analyses. MH provided guidance and discussion on the methodology. BZ attracted partial funding and participated in the design of the analysis strategy. All authors read and approved the final version of this manuscript.

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Authors’ original file for figure 2

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Huang, D., Quan, Y., He, M. et al. Comparison of linear discriminant analysis methods for the classification of cancer based on gene expression data. J Exp Clin Cancer Res 28, 149 (2009).

Download citation


  • Linear Discriminant Analysis
  • Modification Method
  • Gene Expression Dataset
  • Gene Selection Method
  • Fisher Linear Discriminant Analysis