How many biological replicates are needed in an RNA-seq experiment and which differential expression tool should you use?

Nicholas J Schurch, Pietá Schofield, Marek Gierliński, Christian Cole, Alexander Sherstnev, Vijender Singh, Nicola Wrobel, Karim Gharbi, Gordon G Simpson (Lead / Corresponding author), Tom Owen-Hughes (Lead / Corresponding author), Mark Blaxter (Lead / Corresponding author), Geoffrey J Barton (Lead / Corresponding author)

Research output: Contribution to journalArticle

151 Citations (Scopus)
113 Downloads (Pure)

Abstract

RNA-seq is now the technology of choice for genome-wide differential gene expression experiments, but it is not clear how many biological replicates are needed to ensure valid biological interpretation of the results or which statistical tools are best for analyzing the data. An RNA-seq experiment with 48 biological replicates in each of two conditions was performed to answer these questions and provide guidelines for experimental design. With three biological replicates, nine of the 11 tools evaluated found only 20%-40% of the significantly differentially expressed (SDE) genes identified with the full set of 42 clean replicates. This rises to >85% for the subset of SDE genes changing in expression by more than fourfold. To achieve >85% for all SDE genes regardless of fold change requires more than 20 biological replicates. The same nine tools successfully control their false discovery rate at ≲5% for all numbers of replicates, while the remaining two tools fail to control their FDR adequately, particularly for low numbers of replicates. For future RNA-seq experiments, these results suggest that at least six biological replicates should be used, rising to at least 12 when it is important to identify SDE genes for all fold changes. If fewer than 12 replicates are used, a superior combination of true positive and false positive performances makes edgeR and DESeq2 the leading tools. For higher replicate numbers, minimizing false positives is more important and DESeq marginally outperforms the other tools.

Original languageEnglish
Pages (from-to)839-851
Number of pages13
JournalRNA: a Publication of the RNA Society
Volume22
Issue number6
Early online date28 Mar 2016
DOIs
Publication statusPublished - Jun 2016

Fingerprint

RNA
Genes
Research Design
Genome
Guidelines
Technology
Gene Expression

Keywords

  • RNA-seq
  • benchmarking
  • differential expression
  • replication
  • yeast
  • experimental design
  • statistical power

Cite this

Schurch, Nicholas J ; Schofield, Pietá ; Gierliński, Marek ; Cole, Christian ; Sherstnev, Alexander ; Singh, Vijender ; Wrobel, Nicola ; Gharbi, Karim ; Simpson, Gordon G ; Owen-Hughes, Tom ; Blaxter, Mark ; Barton, Geoffrey J. / How many biological replicates are needed in an RNA-seq experiment and which differential expression tool should you use?. In: RNA: a Publication of the RNA Society. 2016 ; Vol. 22, No. 6. pp. 839-851.
@article{aa81598553b145b5bb062fd9e29f108e,
title = "How many biological replicates are needed in an RNA-seq experiment and which differential expression tool should you use?",
abstract = "RNA-seq is now the technology of choice for genome-wide differential gene expression experiments, but it is not clear how many biological replicates are needed to ensure valid biological interpretation of the results or which statistical tools are best for analyzing the data. An RNA-seq experiment with 48 biological replicates in each of two conditions was performed to answer these questions and provide guidelines for experimental design. With three biological replicates, nine of the 11 tools evaluated found only 20{\%}-40{\%} of the significantly differentially expressed (SDE) genes identified with the full set of 42 clean replicates. This rises to >85{\%} for the subset of SDE genes changing in expression by more than fourfold. To achieve >85{\%} for all SDE genes regardless of fold change requires more than 20 biological replicates. The same nine tools successfully control their false discovery rate at ≲5{\%} for all numbers of replicates, while the remaining two tools fail to control their FDR adequately, particularly for low numbers of replicates. For future RNA-seq experiments, these results suggest that at least six biological replicates should be used, rising to at least 12 when it is important to identify SDE genes for all fold changes. If fewer than 12 replicates are used, a superior combination of true positive and false positive performances makes edgeR and DESeq2 the leading tools. For higher replicate numbers, minimizing false positives is more important and DESeq marginally outperforms the other tools.",
keywords = "RNA-seq, benchmarking, differential expression, replication, yeast, experimental design, statistical power",
author = "Schurch, {Nicholas J} and Piet{\'a} Schofield and Marek Gierliński and Christian Cole and Alexander Sherstnev and Vijender Singh and Nicola Wrobel and Karim Gharbi and Simpson, {Gordon G} and Tom Owen-Hughes and Mark Blaxter and Barton, {Geoffrey J}",
note = "We acknowledge funding from the Wellcome Trust (095062; 098439/Z/12; 92530/Z/10/Z; WT097945; WT092340; WT083481), the Medical Research Council (MR/K001744/1), and the Biotechnology and Biological Sciences Research Council (BB/M004155/1; BB/H002286/1).",
year = "2016",
month = "6",
doi = "10.1261/rna.053959.115",
language = "English",
volume = "22",
pages = "839--851",
journal = "RNA: a Publication of the RNA Society",
issn = "1355-8382",
publisher = "Cold Spring Harbor Laboratory Press",
number = "6",

}

How many biological replicates are needed in an RNA-seq experiment and which differential expression tool should you use? / Schurch, Nicholas J; Schofield, Pietá; Gierliński, Marek; Cole, Christian; Sherstnev, Alexander; Singh, Vijender; Wrobel, Nicola; Gharbi, Karim; Simpson, Gordon G (Lead / Corresponding author); Owen-Hughes, Tom (Lead / Corresponding author); Blaxter, Mark (Lead / Corresponding author); Barton, Geoffrey J (Lead / Corresponding author).

In: RNA: a Publication of the RNA Society, Vol. 22, No. 6, 06.2016, p. 839-851.

Research output: Contribution to journalArticle

TY - JOUR

T1 - How many biological replicates are needed in an RNA-seq experiment and which differential expression tool should you use?

AU - Schurch, Nicholas J

AU - Schofield, Pietá

AU - Gierliński, Marek

AU - Cole, Christian

AU - Sherstnev, Alexander

AU - Singh, Vijender

AU - Wrobel, Nicola

AU - Gharbi, Karim

AU - Simpson, Gordon G

AU - Owen-Hughes, Tom

AU - Blaxter, Mark

AU - Barton, Geoffrey J

N1 - We acknowledge funding from the Wellcome Trust (095062; 098439/Z/12; 92530/Z/10/Z; WT097945; WT092340; WT083481), the Medical Research Council (MR/K001744/1), and the Biotechnology and Biological Sciences Research Council (BB/M004155/1; BB/H002286/1).

PY - 2016/6

Y1 - 2016/6

N2 - RNA-seq is now the technology of choice for genome-wide differential gene expression experiments, but it is not clear how many biological replicates are needed to ensure valid biological interpretation of the results or which statistical tools are best for analyzing the data. An RNA-seq experiment with 48 biological replicates in each of two conditions was performed to answer these questions and provide guidelines for experimental design. With three biological replicates, nine of the 11 tools evaluated found only 20%-40% of the significantly differentially expressed (SDE) genes identified with the full set of 42 clean replicates. This rises to >85% for the subset of SDE genes changing in expression by more than fourfold. To achieve >85% for all SDE genes regardless of fold change requires more than 20 biological replicates. The same nine tools successfully control their false discovery rate at ≲5% for all numbers of replicates, while the remaining two tools fail to control their FDR adequately, particularly for low numbers of replicates. For future RNA-seq experiments, these results suggest that at least six biological replicates should be used, rising to at least 12 when it is important to identify SDE genes for all fold changes. If fewer than 12 replicates are used, a superior combination of true positive and false positive performances makes edgeR and DESeq2 the leading tools. For higher replicate numbers, minimizing false positives is more important and DESeq marginally outperforms the other tools.

AB - RNA-seq is now the technology of choice for genome-wide differential gene expression experiments, but it is not clear how many biological replicates are needed to ensure valid biological interpretation of the results or which statistical tools are best for analyzing the data. An RNA-seq experiment with 48 biological replicates in each of two conditions was performed to answer these questions and provide guidelines for experimental design. With three biological replicates, nine of the 11 tools evaluated found only 20%-40% of the significantly differentially expressed (SDE) genes identified with the full set of 42 clean replicates. This rises to >85% for the subset of SDE genes changing in expression by more than fourfold. To achieve >85% for all SDE genes regardless of fold change requires more than 20 biological replicates. The same nine tools successfully control their false discovery rate at ≲5% for all numbers of replicates, while the remaining two tools fail to control their FDR adequately, particularly for low numbers of replicates. For future RNA-seq experiments, these results suggest that at least six biological replicates should be used, rising to at least 12 when it is important to identify SDE genes for all fold changes. If fewer than 12 replicates are used, a superior combination of true positive and false positive performances makes edgeR and DESeq2 the leading tools. For higher replicate numbers, minimizing false positives is more important and DESeq marginally outperforms the other tools.

KW - RNA-seq

KW - benchmarking

KW - differential expression

KW - replication

KW - yeast

KW - experimental design

KW - statistical power

U2 - 10.1261/rna.053959.115

DO - 10.1261/rna.053959.115

M3 - Article

VL - 22

SP - 839

EP - 851

JO - RNA: a Publication of the RNA Society

JF - RNA: a Publication of the RNA Society

SN - 1355-8382

IS - 6

ER -