Crowdsourcing digital health measures to predict Parkinson’s disease severity: the Parkinson’s Disease Digital Biomarker DREAM Challenge

Solveig K. Sieberts; Jennifer Schaff; Marlena Duda; Bálint Ármin Pataki; Ming Sun; Phil Snyder; Jean-francois Daneault; Federico Parisi; Gianluca Costante; Udi Rubin; Peter Banda; Yooree Chae; Elias Chaibub neto; E. Ray Dorsey; Zafer Aydın; Aipeng Chen; Laura L. Elo; Carlos Espino; Enrico Glaab; Ethan Goan; Fatemeh Noushin Golabchi; Yasin Görmez; Maria K. Jaakkola; Jitendra Jonnagaddala; Riku Klén; Dongmei Li; Christian Mcdaniel; Dimitri Perrin; Thanneer M. Perumal; Nastaran Mohammadian Rad; Erin Rainaldi; Stefano Sapienza; Patrick Schwab; Nikolai Shokhirev; Mikko S. Venäläinen; Gloria Vergara-Diaz; Yuqian Zhang; Avner Abrami; Aditya Adhikary; Carla Agurto; Sherry Bhalla; Halil Bilgin; Vittorio Caggiano; Jun Cheng; Eden Deng; Qiwei Gan; Rajan Girsa; Zhi Han; Stephen Heisig; Kun Huang; Samad Jahandideh; Wolfgang Kopp; Christoph F. Kurz; Gregor Lichtner; Raquel Norel; G. P. S. Raghava; Tavpritesh Sethi; Nicholas Shawen; Vaibhav Tripathi; Matthew Tsai; Tongxin Wang; Yi Wu; Jie Zhang; Xinyu Zhang; Yuanjia Wang; Yuanfang Guan; Daniela Brunner; Paolo Bonato; Lara M. Mangravite; Larsson Omberg

doi:10.1038/s41746-021-00414-7

Crowdsourcing digital health measures to predict Parkinson’s disease severity: the Parkinson’s Disease Digital Biomarker DREAM Challenge

Solveig K. Sieberts^*
, Jennifer Schaff
, Marlena Duda
, Bálint Ármin Pataki
, Ming Sun
, Phil Snyder
, Jean-francois Daneault
, Federico Parisi
, Gianluca Costante
, Udi Rubin
, Peter Banda
, Yooree Chae
, Elias Chaibub neto
, E. Ray Dorsey
, Zafer Aydın
, Aipeng Chen
, Laura L. Elo
, Carlos Espino
, Enrico Glaab
, Ethan Goan

Fatemeh Noushin Golabchi, Yasin Görmez, Maria K. Jaakkola, Jitendra Jonnagaddala, Riku Klén, Dongmei Li, Christian Mcdaniel, Dimitri Perrin, Thanneer M. Perumal, Nastaran Mohammadian Rad, Erin Rainaldi, Stefano Sapienza, Patrick Schwab, Nikolai Shokhirev, Mikko S. Venäläinen, Gloria Vergara-Diaz, Yuqian Zhang, Avner Abrami, Aditya Adhikary, Carla Agurto, Sherry Bhalla, Halil Bilgin, Vittorio Caggiano, Jun Cheng, Eden Deng, Qiwei Gan, Rajan Girsa, Zhi Han, Stephen Heisig, Kun Huang, Samad Jahandideh, Wolfgang Kopp, Christoph F. Kurz, Gregor Lichtner, Raquel Norel, G. P. S. Raghava, Tavpritesh Sethi, Nicholas Shawen, Vaibhav Tripathi, Matthew Tsai, Tongxin Wang, Yi Wu, Jie Zhang, Xinyu Zhang, Yuanjia Wang, Yuanfang Guan, Daniela Brunner, Paolo Bonato, Lara M. Mangravite, Larsson Omberg^*

^*Corresponding author for this work

Research output: Contribution to journal › Article › Academic › peer-review

Abstract

Consumer wearables and sensors are a rich source of data about patients' daily disease and symptom burden, particularly in the case of movement disorders like Parkinson's disease (PD). However, interpreting these complex data into so-called digital biomarkers requires complicated analytical approaches, and validating these biomarkers requires sufficient data and unbiased evaluation methods. Here we describe the use of crowdsourcing to specifically evaluate and benchmark features derived from accelerometer and gyroscope data in two different datasets to predict the presence of PD and severity of three PD symptoms: tremor, dyskinesia, and bradykinesia. Forty teams from around the world submitted features, and achieved drastically improved predictive performance for PD status (best AUROC = 0.87), as well as tremor- (best AUPR = 0.75), dyskinesia- (best AUPR = 0.48) and bradykinesia-severity (best AUPR = 0.95).

Original language	English
Article number	53
Number of pages	12
Journal	npj Digital Medicine
Volume	4
Issue number	1
DOIs	https://doi.org/10.1038/s41746-021-00414-7
Publication status	Published - 19 Mar 2021
Externally published	Yes

Access to Document

10.1038/s41746-021-00414-7Licence: CC BY

https://www.nature.com/articles/s41746-021-00414-7

Cite this

Sieberts, S. K., Schaff, J., Duda, M., Pataki, B. Á., Sun, M., Snyder, P., Daneault, J., Parisi, F., Costante, G., Rubin, U., Banda, P., Chae, Y., Chaibub neto, E., Dorsey, E. R., Aydın, Z., Chen, A., Elo, L. L., Espino, C., Glaab, E., ... Omberg, L. (2021). Crowdsourcing digital health measures to predict Parkinson’s disease severity: the Parkinson’s Disease Digital Biomarker DREAM Challenge. npj Digital Medicine, 4(1), Article 53. https://doi.org/10.1038/s41746-021-00414-7

@article{f52a1def22564d02a3d8c4cbbc26cf03,

title = "Crowdsourcing digital health measures to predict Parkinson{\textquoteright}s disease severity: the Parkinson{\textquoteright}s Disease Digital Biomarker DREAM Challenge",

abstract = "Consumer wearables and sensors are a rich source of data about patients' daily disease and symptom burden, particularly in the case of movement disorders like Parkinson's disease (PD). However, interpreting these complex data into so-called digital biomarkers requires complicated analytical approaches, and validating these biomarkers requires sufficient data and unbiased evaluation methods. Here we describe the use of crowdsourcing to specifically evaluate and benchmark features derived from accelerometer and gyroscope data in two different datasets to predict the presence of PD and severity of three PD symptoms: tremor, dyskinesia, and bradykinesia. Forty teams from around the world submitted features, and achieved drastically improved predictive performance for PD status (best AUROC = 0.87), as well as tremor- (best AUPR = 0.75), dyskinesia- (best AUPR = 0.48) and bradykinesia-severity (best AUPR = 0.95).",

author = "Sieberts, \{Solveig K.\} and Jennifer Schaff and Marlena Duda and Pataki, \{B{\'a}lint {\'A}rmin\} and Ming Sun and Phil Snyder and Jean-francois Daneault and Federico Parisi and Gianluca Costante and Udi Rubin and Peter Banda and Yooree Chae and \{Chaibub neto\}, Elias and Dorsey, \{E. Ray\} and Zafer Aydın and Aipeng Chen and Elo, \{Laura L.\} and Carlos Espino and Enrico Glaab and Ethan Goan and Golabchi, \{Fatemeh Noushin\} and Yasin G{\"o}rmez and Jaakkola, \{Maria K.\} and Jitendra Jonnagaddala and Riku Kl{\'e}n and Dongmei Li and Christian Mcdaniel and Dimitri Perrin and Perumal, \{Thanneer M.\} and Rad, \{Nastaran Mohammadian\} and Erin Rainaldi and Stefano Sapienza and Patrick Schwab and Nikolai Shokhirev and Ven{\"a}l{\"a}inen, \{Mikko S.\} and Gloria Vergara-Diaz and Yuqian Zhang and Avner Abrami and Aditya Adhikary and Carla Agurto and Sherry Bhalla and Halil Bilgin and Vittorio Caggiano and Jun Cheng and Eden Deng and Qiwei Gan and Rajan Girsa and Zhi Han and Stephen Heisig and Kun Huang and Samad Jahandideh and Wolfgang Kopp and Kurz, \{Christoph F.\} and Gregor Lichtner and Raquel Norel and Raghava, \{G. P. S.\} and Tavpritesh Sethi and Nicholas Shawen and Vaibhav Tripathi and Matthew Tsai and Tongxin Wang and Yi Wu and Jie Zhang and Xinyu Zhang and Yuanjia Wang and Yuanfang Guan and Daniela Brunner and Paolo Bonato and Mangravite, \{Lara M.\} and Larsson Omberg",

note = "Publisher Copyright: {\textcopyright} 2021, The Author(s).",

year = "2021",

month = mar,

day = "19",

doi = "10.1038/s41746-021-00414-7",

language = "English",

volume = "4",

journal = "npj Digital Medicine",

issn = "2398-6352",

publisher = "Nature Publishing Group",

number = "1",

}

Sieberts, SK, Schaff, J, Duda, M, Pataki, BÁ, Sun, M, Snyder, P, Daneault, J, Parisi, F, Costante, G, Rubin, U, Banda, P, Chae, Y, Chaibub neto, E, Dorsey, ER, Aydın, Z, Chen, A, Elo, LL, Espino, C, Glaab, E, Goan, E, Golabchi, FN, Görmez, Y, Jaakkola, MK, Jonnagaddala, J, Klén, R, Li, D, Mcdaniel, C, Perrin, D, Perumal, TM, Rad, NM, Rainaldi, E, Sapienza, S, Schwab, P, Shokhirev, N, Venäläinen, MS, Vergara-Diaz, G, Zhang, Y, Abrami, A, Adhikary, A, Agurto, C, Bhalla, S, Bilgin, H, Caggiano, V, Cheng, J, Deng, E, Gan, Q, Girsa, R, Han, Z, Heisig, S, Huang, K, Jahandideh, S, Kopp, W, Kurz, CF, Lichtner, G, Norel, R, Raghava, GPS, Sethi, T, Shawen, N, Tripathi, V, Tsai, M, Wang, T, Wu, Y, Zhang, J, Zhang, X, Wang, Y, Guan, Y, Brunner, D, Bonato, P, Mangravite, LM & Omberg, L 2021, 'Crowdsourcing digital health measures to predict Parkinson’s disease severity: the Parkinson’s Disease Digital Biomarker DREAM Challenge', npj Digital Medicine, vol. 4, no. 1, 53. https://doi.org/10.1038/s41746-021-00414-7

TY - JOUR

T1 - Crowdsourcing digital health measures to predict Parkinson’s disease severity: the Parkinson’s Disease Digital Biomarker DREAM Challenge

AU - Sieberts, Solveig K.

AU - Schaff, Jennifer

AU - Duda, Marlena

AU - Pataki, Bálint Ármin

AU - Sun, Ming

AU - Snyder, Phil

AU - Daneault, Jean-francois

AU - Parisi, Federico

AU - Costante, Gianluca

AU - Rubin, Udi

AU - Banda, Peter

AU - Chae, Yooree

AU - Chaibub neto, Elias

AU - Dorsey, E. Ray

AU - Aydın, Zafer

AU - Chen, Aipeng

AU - Elo, Laura L.

AU - Espino, Carlos

AU - Glaab, Enrico

AU - Goan, Ethan

AU - Golabchi, Fatemeh Noushin

AU - Görmez, Yasin

AU - Jaakkola, Maria K.

AU - Jonnagaddala, Jitendra

AU - Klén, Riku

AU - Li, Dongmei

AU - Mcdaniel, Christian

AU - Perrin, Dimitri

AU - Perumal, Thanneer M.

AU - Rad, Nastaran Mohammadian

AU - Rainaldi, Erin

AU - Sapienza, Stefano

AU - Schwab, Patrick

AU - Shokhirev, Nikolai

AU - Venäläinen, Mikko S.

AU - Vergara-Diaz, Gloria

AU - Zhang, Yuqian

AU - Abrami, Avner

AU - Adhikary, Aditya

AU - Agurto, Carla

AU - Bhalla, Sherry

AU - Bilgin, Halil

AU - Caggiano, Vittorio

AU - Cheng, Jun

AU - Deng, Eden

AU - Gan, Qiwei

AU - Girsa, Rajan

AU - Han, Zhi

AU - Heisig, Stephen

AU - Huang, Kun

AU - Jahandideh, Samad

AU - Kopp, Wolfgang

AU - Kurz, Christoph F.

AU - Lichtner, Gregor

AU - Norel, Raquel

AU - Raghava, G. P. S.

AU - Sethi, Tavpritesh

AU - Shawen, Nicholas

AU - Tripathi, Vaibhav

AU - Tsai, Matthew

AU - Wang, Tongxin

AU - Wu, Yi

AU - Zhang, Jie

AU - Zhang, Xinyu

AU - Wang, Yuanjia

AU - Guan, Yuanfang

AU - Brunner, Daniela

AU - Bonato, Paolo

AU - Mangravite, Lara M.

AU - Omberg, Larsson

PY - 2021/3/19

Y1 - 2021/3/19

N2 - Consumer wearables and sensors are a rich source of data about patients' daily disease and symptom burden, particularly in the case of movement disorders like Parkinson's disease (PD). However, interpreting these complex data into so-called digital biomarkers requires complicated analytical approaches, and validating these biomarkers requires sufficient data and unbiased evaluation methods. Here we describe the use of crowdsourcing to specifically evaluate and benchmark features derived from accelerometer and gyroscope data in two different datasets to predict the presence of PD and severity of three PD symptoms: tremor, dyskinesia, and bradykinesia. Forty teams from around the world submitted features, and achieved drastically improved predictive performance for PD status (best AUROC = 0.87), as well as tremor- (best AUPR = 0.75), dyskinesia- (best AUPR = 0.48) and bradykinesia-severity (best AUPR = 0.95).

AB - Consumer wearables and sensors are a rich source of data about patients' daily disease and symptom burden, particularly in the case of movement disorders like Parkinson's disease (PD). However, interpreting these complex data into so-called digital biomarkers requires complicated analytical approaches, and validating these biomarkers requires sufficient data and unbiased evaluation methods. Here we describe the use of crowdsourcing to specifically evaluate and benchmark features derived from accelerometer and gyroscope data in two different datasets to predict the presence of PD and severity of three PD symptoms: tremor, dyskinesia, and bradykinesia. Forty teams from around the world submitted features, and achieved drastically improved predictive performance for PD status (best AUROC = 0.87), as well as tremor- (best AUPR = 0.75), dyskinesia- (best AUPR = 0.48) and bradykinesia-severity (best AUPR = 0.95).

U2 - 10.1038/s41746-021-00414-7

DO - 10.1038/s41746-021-00414-7

M3 - Article

C2 - 33742069

SN - 2398-6352

VL - 4

JO - npj Digital Medicine

JF - npj Digital Medicine

IS - 1

M1 - 53

ER -

Crowdsourcing digital health measures to predict Parkinson’s disease severity: the Parkinson’s Disease Digital Biomarker DREAM Challenge

Abstract

Access to Document

Fingerprint

Cite this