The ENCODE Imputation Challenge: a critical assessment of methods for cross-cell type imputation of epigenomic profiles

Jacob  Schreiber,; Carles Boix; Jin wook Lee; Hongyang  Li; Yuanfang Guan; Chun-Chieh Chang; Jen-Chien   Chang; Alex Hawkins-Hooker; Bernhard Schölkopf; Gabriele Schweikert; Mateo Rojas Carulla; Arif Canakoglu; Francesco Guzzo; Luca Nanni; Marco Masseroli; Mark James Carman; Pietro Pinoli; Chenyang Hong; Kevin Y. Yip; Jeffrey P. Spence; Sanjit Singh Batra; Yun S. Song; Shaun Mahony; Zheng Zhang; Wuwei Tan; Yang Shen; Yuanfei Sun; Minyi Shi; Jessika Adrian; Richard Sandstrom; Nina Farrell; Jessica Halow; Kristen Lee; Lixia Jiang; Xinqiong Yang; Charles Epstein; J. Seth Strattan; Bradley Bernstein; Michael Snyder; Manolis Kellis; William Stafford; Anshul Kundaje

doi:10.1186/s13059-023-02915-y

The ENCODE Imputation Challenge: a critical assessment of methods for cross-cell type imputation of epigenomic profiles

, Jacob Schreiber, (Lead / Corresponding author), Carles Boix (Lead / Corresponding author), Jin wook Lee, Hongyang Li, Yuanfang Guan, Chun-Chieh Chang, Jen-Chien Chang, Alex Hawkins-Hooker, Bernhard Schölkopf, Gabriele Schweikert, Mateo Rojas Carulla, Arif Canakoglu, Francesco Guzzo, Luca Nanni, Marco Masseroli, Mark James Carman, Pietro Pinoli, Chenyang Hong, Kevin Y. YipJeffrey P. Spence, Sanjit Singh Batra, Yun S. Song, Shaun Mahony, Zheng Zhang, Wuwei Tan, Yang Shen, Yuanfei Sun, Minyi Shi, Jessika Adrian, Richard Sandstrom, Nina Farrell, Jessica Halow, Kristen Lee, Lixia Jiang, Xinqiong Yang, Charles Epstein, J. Seth Strattan, Bradley Bernstein, Michael Snyder, Manolis Kellis, William Stafford, Anshul Kundaje

Computational Biology

Research output: Contribution to journal › Article › peer-review

7 Citations (Scopus)

76 Downloads (Pure)

Abstract

A promising alternative to comprehensively performing genomics experiments is to, instead, perform a subset of experiments and use computational methods to impute the remainder. However, identifying the best imputation methods and what measures meaningfully evaluate performance are open questions. We address these questions by comprehensively analyzing 23 methods from the ENCODE Imputation Challenge. We find that imputation evaluations are challenging and confounded by distributional shifts from differences in data collection and processing over time, the amount of available data, and redundancy among performance measures. Our analyses suggest simple steps for overcoming these issues and promising directions for more robust research.

Original language	English
Article number	79
Number of pages	22
Journal	Genome Biology
Volume	24
Issue number	1
DOIs	https://doi.org/10.1186/s13059-023-02915-y
Publication status	Published - 18 Apr 2023

Access to Document

10.1186/s13059-023-02915-yLicence: CC BY

Final Published VersionFinal published version, 1.1 MBLicence: CC BY

Cite this

, Schreiber, J., Boix, C., Lee, J. W., Li, H., Guan, Y., Chang, C.-C., Chang, J.-C., Hawkins-Hooker, A., Schölkopf, B., Schweikert, G., Carulla, M. R., Canakoglu, A., Guzzo, F., Nanni, L., Masseroli, M., Carman, M. J., Pinoli, P., Hong, C., ... Kundaje, A. (2023). The ENCODE Imputation Challenge: a critical assessment of methods for cross-cell type imputation of epigenomic profiles. Genome Biology, 24(1), Article 79. https://doi.org/10.1186/s13059-023-02915-y

@article{96c35fbae2d34a7fbedf1c7a7592af39,

title = "The ENCODE Imputation Challenge: a critical assessment of methods for cross-cell type imputation of epigenomic profiles",

abstract = "A promising alternative to comprehensively performing genomics experiments is to, instead, perform a subset of experiments and use computational methods to impute the remainder. However, identifying the best imputation methods and what measures meaningfully evaluate performance are open questions. We address these questions by comprehensively analyzing 23 methods from the ENCODE Imputation Challenge. We find that imputation evaluations are challenging and confounded by distributional shifts from differences in data collection and processing over time, the amount of available data, and redundancy among performance measures. Our analyses suggest simple steps for overcoming these issues and promising directions for more robust research.",

author = "Jacob Schreiber, and Carles Boix and Lee, {Jin wook} and Hongyang Li and Yuanfang Guan and Chun-Chieh Chang and Jen-Chien Chang and Alex Hawkins-Hooker and Bernhard Sch{\"o}lkopf and Gabriele Schweikert and Carulla, {Mateo Rojas} and Arif Canakoglu and Francesco Guzzo and Luca Nanni and Marco Masseroli and Carman, {Mark James} and Pietro Pinoli and Chenyang Hong and Yip, {Kevin Y.} and Spence, {Jeffrey P.} and Batra, {Sanjit Singh} and Song, {Yun S.} and Shaun Mahony and Zheng Zhang and Wuwei Tan and Yang Shen and Yuanfei Sun and Minyi Shi and Jessika Adrian and Richard Sandstrom and Nina Farrell and Jessica Halow and Kristen Lee and Lixia Jiang and Xinqiong Yang and Charles Epstein and Strattan, {J. Seth} and Bradley Bernstein and Michael Snyder and Manolis Kellis and William Stafford and Anshul Kundaje",

year = "2023",

month = apr,

day = "18",

doi = "10.1186/s13059-023-02915-y",

language = "English",

volume = "24",

journal = "Genome Biology",

issn = "1474-7596",

publisher = "BioMed Central",

number = "1",

}

Schreiber, J, Boix, C, Lee, JW, Li, H, Guan, Y, Chang, C-C, Chang, J-C, Hawkins-Hooker, A, Schölkopf, B, Schweikert, G, Carulla, MR, Canakoglu, A, Guzzo, F, Nanni, L, Masseroli, M, Carman, MJ, Pinoli, P, Hong, C, Yip, KY, Spence, JP, Batra, SS, Song, YS, Mahony, S, Zhang, Z, Tan, W, Shen, Y, Sun, Y, Shi, M, Adrian, J, Sandstrom, R, Farrell, N, Halow, J, Lee, K, Jiang, L, Yang, X, Epstein, C, Strattan, JS, Bernstein, B, Snyder, M, Kellis, M & Stafford, W & Kundaje, A 2023, 'The ENCODE Imputation Challenge: a critical assessment of methods for cross-cell type imputation of epigenomic profiles', Genome Biology, vol. 24, no. 1, 79. https://doi.org/10.1186/s13059-023-02915-y

TY - JOUR

T1 - The ENCODE Imputation Challenge: a critical assessment of methods for cross-cell type imputation of epigenomic profiles

AU - Schreiber,, Jacob

AU - Boix, Carles

AU - Lee, Jin wook

AU - Li, Hongyang

AU - Guan, Yuanfang

AU - Chang, Chun-Chieh

AU - Chang, Jen-Chien

AU - Hawkins-Hooker, Alex

AU - Schölkopf, Bernhard

AU - Schweikert, Gabriele

AU - Carulla, Mateo Rojas

AU - Canakoglu, Arif

AU - Guzzo, Francesco

AU - Nanni, Luca

AU - Masseroli, Marco

AU - Carman, Mark James

AU - Pinoli, Pietro

AU - Hong, Chenyang

AU - Yip, Kevin Y.

AU - Spence, Jeffrey P.

AU - Batra, Sanjit Singh

AU - Song, Yun S.

AU - Mahony, Shaun

AU - Zhang, Zheng

AU - Tan, Wuwei

AU - Shen, Yang

AU - Sun, Yuanfei

AU - Shi, Minyi

AU - Adrian, Jessika

AU - Sandstrom, Richard

AU - Farrell, Nina

AU - Halow, Jessica

AU - Lee, Kristen

AU - Jiang, Lixia

AU - Yang, Xinqiong

AU - Epstein, Charles

AU - Strattan, J. Seth

AU - Bernstein, Bradley

AU - Snyder, Michael

AU - Kellis, Manolis

AU - Stafford, William

AU - Kundaje, Anshul

PY - 2023/4/18

Y1 - 2023/4/18

N2 - A promising alternative to comprehensively performing genomics experiments is to, instead, perform a subset of experiments and use computational methods to impute the remainder. However, identifying the best imputation methods and what measures meaningfully evaluate performance are open questions. We address these questions by comprehensively analyzing 23 methods from the ENCODE Imputation Challenge. We find that imputation evaluations are challenging and confounded by distributional shifts from differences in data collection and processing over time, the amount of available data, and redundancy among performance measures. Our analyses suggest simple steps for overcoming these issues and promising directions for more robust research.

AB - A promising alternative to comprehensively performing genomics experiments is to, instead, perform a subset of experiments and use computational methods to impute the remainder. However, identifying the best imputation methods and what measures meaningfully evaluate performance are open questions. We address these questions by comprehensively analyzing 23 methods from the ENCODE Imputation Challenge. We find that imputation evaluations are challenging and confounded by distributional shifts from differences in data collection and processing over time, the amount of available data, and redundancy among performance measures. Our analyses suggest simple steps for overcoming these issues and promising directions for more robust research.

U2 - 10.1186/s13059-023-02915-y

DO - 10.1186/s13059-023-02915-y

M3 - Article

C2 - 37072822

SN - 1474-7596

VL - 24

JO - Genome Biology

JF - Genome Biology

IS - 1

M1 - 79

ER -

The ENCODE Imputation Challenge: a critical assessment of methods for cross-cell type imputation of epigenomic profiles

Abstract

Access to Document

Fingerprint

Cite this