Transparency and reproducibility in artificial intelligence

Benjamin Haibe-Kains; George Alexandru Adam; Ahmed Hosny; Farnoosh Khodakarami; Thakkar Shraddha; Thakkar Shraddha; Rebecca Kusko; Susanna-Assunta Sansone; Weida Tong; Russ D. Wolfinger; Christopher E. Mason; Wendell Jones; Joaquin Dopazo; Cesare Furlanello; Levi Waldron; Bo Wang; Chris McIntosh; Anna Goldenberg; Anshul Kundaje; Casey S. Greene; Tamara Broderick; Michael M. Hoffman; Jeffrey T. Leek; Keegan Korthauer; Wolfgang Huber; Alvis Brazma; Joelle Pineau; Robert Tibshirani; Trevor Hastie; John P. A. Ioannidis; John Quackenbush; Hugo J. W. L. Aerts

doi:10.1038/s41586-020-2766-y

Transparency and reproducibility in artificial intelligence

Benjamin Haibe-Kains^*, George Alexandru Adam, Ahmed Hosny, Farnoosh Khodakarami, Thakkar Shraddha, Thakkar Shraddha, Rebecca Kusko, Susanna-Assunta Sansone, Weida Tong, Russ D. Wolfinger, Christopher E. Mason, Wendell Jones, Joaquin Dopazo, Cesare Furlanello, Levi Waldron, Bo Wang, Chris McIntosh, Anna Goldenberg, Anshul Kundaje, Casey S. GreeneTamara Broderick, Michael M. Hoffman, Jeffrey T. Leek, Keegan Korthauer, Wolfgang Huber, Alvis Brazma, Joelle Pineau, Robert Tibshirani, Trevor Hastie, John P. A. Ioannidis, John Quackenbush, Hugo J. W. L. Aerts

^*Corresponding author for this work

Research output: Contribution to journal › Comment/Letter to the editor › Academic › peer-review

604 Downloads (Pure)

Abstract

Breakthroughs in artificial intelligence (AI) hold enormous potential as it can automate complex tasks and go even beyond human performance. In their study, McKinney et al. showed the high potential of AI for breast cancer screening. However, the lack of methods’ details and algorithm code undermines its scientific value. Here, we identify obstacles hindering transparent and reproducible AI research as faced by McKinney et al., and provide solutions to these obstacles with implications for the broader field.

Original language	English
Pages (from-to)	E14-E16
Number of pages	7
Journal	Nature
Volume	586
Issue number	7829
DOIs	https://doi.org/10.1038/s41586-020-2766-y
Publication status	Published - 15 Oct 2020

Keywords

AI

Access to Document

10.1038/s41586-020-2766-y

Full textFinal published version, 1.16 MBLicence: Taverne

Cite this

Haibe-Kains, B., Adam, G. A., Hosny, A., Khodakarami, F., Shraddha, T., Shraddha, T., Kusko, R., Sansone, S.-A., Tong, W., Wolfinger, R. D., Mason, C. E., Jones, W., Dopazo, J., Furlanello, C., Waldron, L., Wang, B., McIntosh, C., Goldenberg, A., Kundaje, A., ... Aerts, H. J. W. L. (2020). Transparency and reproducibility in artificial intelligence. Nature, 586(7829), E14-E16. https://doi.org/10.1038/s41586-020-2766-y

@article{e950535dde7640149c8ef17827480c66,

title = "Transparency and reproducibility in artificial intelligence",

abstract = "Breakthroughs in artificial intelligence (AI) hold enormous potential as it can automate complex tasks and go even beyond human performance. In their study, McKinney et al. showed the high potential of AI for breast cancer screening. However, the lack of methods{\textquoteright} details and algorithm code undermines its scientific value. Here, we identify obstacles hindering transparent and reproducible AI research as faced by McKinney et al., and provide solutions to these obstacles with implications for the broader field.",

keywords = "AI",

author = "Benjamin Haibe-Kains and Adam, {George Alexandru} and Ahmed Hosny and Farnoosh Khodakarami and Thakkar Shraddha and Thakkar Shraddha and Rebecca Kusko and Susanna-Assunta Sansone and Weida Tong and Wolfinger, {Russ D.} and Mason, {Christopher E.} and Wendell Jones and Joaquin Dopazo and Cesare Furlanello and Levi Waldron and Bo Wang and Chris McIntosh and Anna Goldenberg and Anshul Kundaje and Greene, {Casey S.} and Tamara Broderick and Hoffman, {Michael M.} and Leek, {Jeffrey T.} and Keegan Korthauer and Wolfgang Huber and Alvis Brazma and Joelle Pineau and Robert Tibshirani and Trevor Hastie and Ioannidis, {John P. A.} and John Quackenbush and Aerts, {Hugo J. W. L.}",

note = "Funding Information: Acknowledgements We thank S. McKinney and colleagues for their prompt and open communication regarding the materials and methods of their study. This work was supported in part by the National Cancer Institute (R01 CA237170). Funding Information: Competing interests This study was funded by Google LLC. S.M.M., A.K., D.T., C.J.K, Y.L., G.S.C. and S.S. are employees of Google and own stock as part of the standard compensation package. The authors have no other competing interests to disclose.",

year = "2020",

month = oct,

day = "15",

doi = "10.1038/s41586-020-2766-y",

language = "English",

volume = "586",

pages = "E14--E16",

journal = "Nature",

issn = "0028-0836",

publisher = "Nature Research",

number = "7829",

}

Haibe-Kains, B, Adam, GA, Hosny, A, Khodakarami, F, Shraddha, T, Shraddha, T, Kusko, R, Sansone, S-A, Tong, W, Wolfinger, RD, Mason, CE, Jones, W, Dopazo, J, Furlanello, C, Waldron, L, Wang, B, McIntosh, C, Goldenberg, A, Kundaje, A, Greene, CS, Broderick, T, Hoffman, MM, Leek, JT, Korthauer, K, Huber, W, Brazma, A, Pineau, J, Tibshirani, R, Hastie, T, Ioannidis, JPA, Quackenbush, J & Aerts, HJWL 2020, 'Transparency and reproducibility in artificial intelligence', Nature, vol. 586, no. 7829, pp. E14-E16. https://doi.org/10.1038/s41586-020-2766-y

TY - JOUR

T1 - Transparency and reproducibility in artificial intelligence

AU - Haibe-Kains, Benjamin

AU - Adam, George Alexandru

AU - Hosny, Ahmed

AU - Khodakarami, Farnoosh

AU - Shraddha, Thakkar

AU - Kusko, Rebecca

AU - Sansone, Susanna-Assunta

AU - Tong, Weida

AU - Wolfinger, Russ D.

AU - Mason, Christopher E.

AU - Jones, Wendell

AU - Dopazo, Joaquin

AU - Furlanello, Cesare

AU - Waldron, Levi

AU - Wang, Bo

AU - McIntosh, Chris

AU - Goldenberg, Anna

AU - Kundaje, Anshul

AU - Greene, Casey S.

AU - Broderick, Tamara

AU - Hoffman, Michael M.

AU - Leek, Jeffrey T.

AU - Korthauer, Keegan

AU - Huber, Wolfgang

AU - Brazma, Alvis

AU - Pineau, Joelle

AU - Tibshirani, Robert

AU - Hastie, Trevor

AU - Ioannidis, John P. A.

AU - Quackenbush, John

AU - Aerts, Hugo J. W. L.

N1 - Funding Information: Acknowledgements We thank S. McKinney and colleagues for their prompt and open communication regarding the materials and methods of their study. This work was supported in part by the National Cancer Institute (R01 CA237170). Funding Information: Competing interests This study was funded by Google LLC. S.M.M., A.K., D.T., C.J.K, Y.L., G.S.C. and S.S. are employees of Google and own stock as part of the standard compensation package. The authors have no other competing interests to disclose.

PY - 2020/10/15

Y1 - 2020/10/15

N2 - Breakthroughs in artificial intelligence (AI) hold enormous potential as it can automate complex tasks and go even beyond human performance. In their study, McKinney et al. showed the high potential of AI for breast cancer screening. However, the lack of methods’ details and algorithm code undermines its scientific value. Here, we identify obstacles hindering transparent and reproducible AI research as faced by McKinney et al., and provide solutions to these obstacles with implications for the broader field.

AB - Breakthroughs in artificial intelligence (AI) hold enormous potential as it can automate complex tasks and go even beyond human performance. In their study, McKinney et al. showed the high potential of AI for breast cancer screening. However, the lack of methods’ details and algorithm code undermines its scientific value. Here, we identify obstacles hindering transparent and reproducible AI research as faced by McKinney et al., and provide solutions to these obstacles with implications for the broader field.

KW - AI

U2 - 10.1038/s41586-020-2766-y

DO - 10.1038/s41586-020-2766-y

M3 - Comment/Letter to the editor

C2 - 33057217

SN - 0028-0836

VL - 586

SP - E14-E16

JO - Nature

JF - Nature

IS - 7829

ER -

Transparency and reproducibility in artificial intelligence

Abstract

Keywords

Access to Document

Fingerprint

Cite this