Abstract
| Original language | English |
|---|---|
| Article number | aac4716 |
| Journal | Science |
| Volume | 349 |
| Issue number | 6251 |
| DOIs | |
| Publication status | Published - 28 Aug 2015 |
| Externally published | Yes |
Access to Document
Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver
}
Estimating the reproducibility of psychological science. / Aarts, Alexander A.; Anderson, Joanna E.; Anderson, Christopher J.; Attridge, Peter R.; Attwood, Angela; Axt, Jordan; Babel, Molly; Bahnik, Stepan; Baranski, Erica; Barnett-Cowan, Michael; Bartmess, Elizabeth; Beer, Jennifer; Bell, Raoul; Bentley, Heather; Beyan, Leah; Binion, Grace; Borsboom, Denny; Bosch, Annick; Bosco, Frank A.; Bowman, Sara D.; Brandt, Mark J.; Braswell, Erin; Brohmer, Hilmar; Brown, Benjamin T.; Brown, Kristina; Bruening, Jovita; Calhoun-Sauls, Ann; Callahan, Shannon P.; Chagnon, Elizabeth; Chandler, Jesse; Chartier, Christopher R.; Cheung, Felix; Christopherson, Cody D.; Cillessen, Linda; Clay, Russ; Cleary, Hayley; Cloud, Mark D.; Cohn, Michael; Cohoon, Johanna; Columbus, Simon; Cordes, Andreas; Costantini, Giulio; Alvarez, Leslie D. Cramblet; Cremata, Ed; Crusius, Jan; DeCoster, Jamie; DeGaetano, Michelle A.; Della Penna, Nicolas; den Bezemer, Bobby; Deserno, Marie K.; Devitt, Olivia; Dewitte, Laura; Dobolyi, David G.; Dodson, Geneva T.; Donnellan, M. Brent; Donohue, Ryan; Dore, Rebecca A.; Dorrough, Angela; Dreber, Anna; Dugas, Michelle; Dunn, Elizabeth W.; Easey, Kayleigh; Eboigbe, Sylvia; Eggleston, Casey; Embley, Jo; Epskamp, Sacha; Errington, Timothy M.; Estel, Vivien; Farach, Frank J.; Feather, Jenelle; Fedor, Anna; Fernandez-Castilla, Belen; Fiedler, Susann; Field, James G.; Fitneva, Stanka A.; Flagan, Taru; Forest, Amanda L.; Forsell, Eskil; Foster, Joshua D.; Frank, Michael C.; Frazier, Rebecca S.; Fuchs, Heather; Gable, Philip; Galak, Jeff; Galliani, Elisa Maria; Gampa, Anup; Garcia, Sara; Gazarian, Douglas; Gilbert, Elizabeth; Giner-Sorolla, Roger; Gloeckner, Andreas; Goellner, Lars; Goh, Jin X.; Goldberg, Rebecca; Goodbourn, Patrick T.; Gordon-McKeon, Shauna; Gorges, Bryan; Gorges, Jessie; Goss, Justin; Graham, Jesse; Grange, James A.; Gray, Jeremy; Hartgerink, Chris; Hartshorne, Joshua K.; Hasselman, Fred; Hayes, Timothy; Heikensten, Emma; Henninger, Felix; Hodsoll, John; Holubar, Taylor; Hoogendoorn, Gea; Humphries, Denise J.; Hung, Cathy O. -Y.; Immelman, Nathali; Irsik, Vanessa C.; Jahn, Georg; Jaekel, Frank; Jekel, Marc; Johannesson, Magnus; Johnson, Larissa G.; Johnson, David J.; Johnson, Kate M.; Johnston, William J.; Jonas, Kai; Joy-Gaba, Jennifer A.; Kappes, Heather Barry; Kelso, Kim; Kidwell, Mallory C.; Kim, Seung Kyung; Kirkhart, Matthew; Kleinberg, Bennett; Knezevic, Goran; Kolorz, Franziska Maria; Kossakowski, Jolanda J.; Krause, Robert Wilhelm; Krijnen, Job; Kuhlmann, Tim; Kunkels, Yoram K.; Kyc, Megan M.; Lai, Calvin K.; Laique, Aamir; Lakens, Daniel; Lane, Kristin A.; Lassetter, Bethany; Lazarevic, Ljiljana B.; LeBel, Etienne P.; Lee, Key Jung; Lee, Minha; Lemm, Kristi; Levitan, Carmel A.; Lewis, Melissa; Lin, Lin; Lin, Stephanie; Lippold, Matthias; Loureiro, Darren; Luteijn, Ilse; Mackinnon, Sean; Mainard, Heather N.; Marigold, Denise C.; Martin, Daniel P.; Martinez, Tylar; Masicampo, E. J.; Matacotta, Josh; Mathur, Maya; May, Michael; Mechin, Nicole; Mehta, Pranjal; Meixner, Johannes M.; Melinger, Alissa; Miller, Jeremy K.; Miller, Mallorie; Moore, Katherine S.; Moeschl, Marcus; Motyl, Matt; Mueller, Stephanie M.; Munafo, Marcus R.; Neijenhuijs, Koen I.; Nervi, Taylor; Nicolas, Gandalf; Nilsonne, Gustav; Nosek, Brian A.; Nuijten, Michele B.; Olsson, Catherine; Osborne, Colleen; Ostkamp, Lutz; Pavel, Misha; Penton-Voak, Ian S.; Perna, Olivia; Pernet, Cyril; Perugini, Marco; Pipitone, R. Nathan; Pitts, Michael; Plessow, Franziska; Prenoveau, Jason M.; Rahal, Rima-Maria; Ratliff, Kate A.; Reinhard, David; Renkewitz, Frank; Ricker, Ashley A.; Rigney, Anastasia; Rivers, Andrew M.; Roebke, Mark; Rutchick, Abraham M.; Ryan, Robert S.; Sahin, Onur; Saide, Anondah; Sandstrom, Gillian M.; Santos, David; Saxe, Rebecca; Schlegelmilch, Rene; Schmidt, Kathleen; Scholz, Sabine; Seibel, Larissa; Selterman, Dylan Faulkner; Shaki, Samuel; Simpson, William B.; Sinclair, H. Colleen; Skorinko, Jeanine L. M.; Slowik, Agnieszka; Snyder, Joel S.; Soderberg, Courtney; Sonnleitner, Carina; Spencer, Nick; Spies, Jeffrey R.; Steegen, Sara; Stieger, Stefan; Strohminger, Nina; Sullivan, Gavin B.; Talhelm, Thomas; Tapia, Megan; te Dorsthorst, Anniek; Thomae, Manuela; Thomas, Sarah L.; Tio, Pia; Traets, Frits; Tsang, Steve; Tuerlinckx, Francis; Turchan, Paul; Valasek, Milan; van `t Veer, Anna E.; van Aert, Robbie C. M.; van Assen, Marcel A.; van Bork, Riet; van de Ven, Mathijs; van den Bergh, Don; van der Hulst, Marije; van Dooren, Roel; van Doorn, Johnny; van Renswoude, Daan R.; van Rijn, Hedderik; Vanpaemel, Wolf; Echeverria, Alejandro Vasquez; Vazquez, Melissa; Velez, Natalia; Vermue, Marieke; Verschoor, Mark; Vianello, Michelangelo; Voracek, Martin; Vuu, Gina; Wagenmakers, Eric-Jan; Weerdmeester, Joanneke; Welsh, Ashlee; Westgate, Erin C.; Wissink, Joeri; Wood, Michael; Woods, Andy; Wright, Emily; Wu, Sining; Zeelenberg, Marcel; Zuni, Kellylynn; Collaboration, Open Sci.
In: Science, Vol. 349, No. 6251, aac4716, 28.08.2015.Research output: Contribution to journal › Article › Academic › peer-review
TY - JOUR
T1 - Estimating the reproducibility of psychological science
AU - Aarts, Alexander A.
AU - Anderson, Joanna E.
AU - Anderson, Christopher J.
AU - Attridge, Peter R.
AU - Attwood, Angela
AU - Axt, Jordan
AU - Babel, Molly
AU - Bahnik, Stepan
AU - Baranski, Erica
AU - Barnett-Cowan, Michael
AU - Bartmess, Elizabeth
AU - Beer, Jennifer
AU - Bell, Raoul
AU - Bentley, Heather
AU - Beyan, Leah
AU - Binion, Grace
AU - Borsboom, Denny
AU - Bosch, Annick
AU - Bosco, Frank A.
AU - Bowman, Sara D.
AU - Brandt, Mark J.
AU - Braswell, Erin
AU - Brohmer, Hilmar
AU - Brown, Benjamin T.
AU - Brown, Kristina
AU - Bruening, Jovita
AU - Calhoun-Sauls, Ann
AU - Callahan, Shannon P.
AU - Chagnon, Elizabeth
AU - Chandler, Jesse
AU - Chartier, Christopher R.
AU - Cheung, Felix
AU - Christopherson, Cody D.
AU - Cillessen, Linda
AU - Clay, Russ
AU - Cleary, Hayley
AU - Cloud, Mark D.
AU - Cohn, Michael
AU - Cohoon, Johanna
AU - Columbus, Simon
AU - Cordes, Andreas
AU - Costantini, Giulio
AU - Alvarez, Leslie D. Cramblet
AU - Cremata, Ed
AU - Crusius, Jan
AU - DeCoster, Jamie
AU - DeGaetano, Michelle A.
AU - Della Penna, Nicolas
AU - den Bezemer, Bobby
AU - Deserno, Marie K.
AU - Devitt, Olivia
AU - Dewitte, Laura
AU - Dobolyi, David G.
AU - Dodson, Geneva T.
AU - Donnellan, M. Brent
AU - Donohue, Ryan
AU - Dore, Rebecca A.
AU - Dorrough, Angela
AU - Dreber, Anna
AU - Dugas, Michelle
AU - Dunn, Elizabeth W.
AU - Easey, Kayleigh
AU - Eboigbe, Sylvia
AU - Eggleston, Casey
AU - Embley, Jo
AU - Epskamp, Sacha
AU - Errington, Timothy M.
AU - Estel, Vivien
AU - Farach, Frank J.
AU - Feather, Jenelle
AU - Fedor, Anna
AU - Fernandez-Castilla, Belen
AU - Fiedler, Susann
AU - Field, James G.
AU - Fitneva, Stanka A.
AU - Flagan, Taru
AU - Forest, Amanda L.
AU - Forsell, Eskil
AU - Foster, Joshua D.
AU - Frank, Michael C.
AU - Frazier, Rebecca S.
AU - Fuchs, Heather
AU - Gable, Philip
AU - Galak, Jeff
AU - Galliani, Elisa Maria
AU - Gampa, Anup
AU - Garcia, Sara
AU - Gazarian, Douglas
AU - Gilbert, Elizabeth
AU - Giner-Sorolla, Roger
AU - Gloeckner, Andreas
AU - Goellner, Lars
AU - Goh, Jin X.
AU - Goldberg, Rebecca
AU - Goodbourn, Patrick T.
AU - Gordon-McKeon, Shauna
AU - Gorges, Bryan
AU - Gorges, Jessie
AU - Goss, Justin
AU - Graham, Jesse
AU - Grange, James A.
AU - Gray, Jeremy
AU - Hartgerink, Chris
AU - Hartshorne, Joshua K.
AU - Hasselman, Fred
AU - Hayes, Timothy
AU - Heikensten, Emma
AU - Henninger, Felix
AU - Hodsoll, John
AU - Holubar, Taylor
AU - Hoogendoorn, Gea
AU - Humphries, Denise J.
AU - Hung, Cathy O. -Y.
AU - Immelman, Nathali
AU - Irsik, Vanessa C.
AU - Jahn, Georg
AU - Jaekel, Frank
AU - Jekel, Marc
AU - Johannesson, Magnus
AU - Johnson, Larissa G.
AU - Johnson, David J.
AU - Johnson, Kate M.
AU - Johnston, William J.
AU - Jonas, Kai
AU - Joy-Gaba, Jennifer A.
AU - Kappes, Heather Barry
AU - Kelso, Kim
AU - Kidwell, Mallory C.
AU - Kim, Seung Kyung
AU - Kirkhart, Matthew
AU - Kleinberg, Bennett
AU - Knezevic, Goran
AU - Kolorz, Franziska Maria
AU - Kossakowski, Jolanda J.
AU - Krause, Robert Wilhelm
AU - Krijnen, Job
AU - Kuhlmann, Tim
AU - Kunkels, Yoram K.
AU - Kyc, Megan M.
AU - Lai, Calvin K.
AU - Laique, Aamir
AU - Lakens, Daniel
AU - Lane, Kristin A.
AU - Lassetter, Bethany
AU - Lazarevic, Ljiljana B.
AU - LeBel, Etienne P.
AU - Lee, Key Jung
AU - Lee, Minha
AU - Lemm, Kristi
AU - Levitan, Carmel A.
AU - Lewis, Melissa
AU - Lin, Lin
AU - Lin, Stephanie
AU - Lippold, Matthias
AU - Loureiro, Darren
AU - Luteijn, Ilse
AU - Mackinnon, Sean
AU - Mainard, Heather N.
AU - Marigold, Denise C.
AU - Martin, Daniel P.
AU - Martinez, Tylar
AU - Masicampo, E. J.
AU - Matacotta, Josh
AU - Mathur, Maya
AU - May, Michael
AU - Mechin, Nicole
AU - Mehta, Pranjal
AU - Meixner, Johannes M.
AU - Melinger, Alissa
AU - Miller, Jeremy K.
AU - Miller, Mallorie
AU - Moore, Katherine S.
AU - Moeschl, Marcus
AU - Motyl, Matt
AU - Mueller, Stephanie M.
AU - Munafo, Marcus R.
AU - Neijenhuijs, Koen I.
AU - Nervi, Taylor
AU - Nicolas, Gandalf
AU - Nilsonne, Gustav
AU - Nosek, Brian A.
AU - Nuijten, Michele B.
AU - Olsson, Catherine
AU - Osborne, Colleen
AU - Ostkamp, Lutz
AU - Pavel, Misha
AU - Penton-Voak, Ian S.
AU - Perna, Olivia
AU - Pernet, Cyril
AU - Perugini, Marco
AU - Pipitone, R. Nathan
AU - Pitts, Michael
AU - Plessow, Franziska
AU - Prenoveau, Jason M.
AU - Rahal, Rima-Maria
AU - Ratliff, Kate A.
AU - Reinhard, David
AU - Renkewitz, Frank
AU - Ricker, Ashley A.
AU - Rigney, Anastasia
AU - Rivers, Andrew M.
AU - Roebke, Mark
AU - Rutchick, Abraham M.
AU - Ryan, Robert S.
AU - Sahin, Onur
AU - Saide, Anondah
AU - Sandstrom, Gillian M.
AU - Santos, David
AU - Saxe, Rebecca
AU - Schlegelmilch, Rene
AU - Schmidt, Kathleen
AU - Scholz, Sabine
AU - Seibel, Larissa
AU - Selterman, Dylan Faulkner
AU - Shaki, Samuel
AU - Simpson, William B.
AU - Sinclair, H. Colleen
AU - Skorinko, Jeanine L. M.
AU - Slowik, Agnieszka
AU - Snyder, Joel S.
AU - Soderberg, Courtney
AU - Sonnleitner, Carina
AU - Spencer, Nick
AU - Spies, Jeffrey R.
AU - Steegen, Sara
AU - Stieger, Stefan
AU - Strohminger, Nina
AU - Sullivan, Gavin B.
AU - Talhelm, Thomas
AU - Tapia, Megan
AU - te Dorsthorst, Anniek
AU - Thomae, Manuela
AU - Thomas, Sarah L.
AU - Tio, Pia
AU - Traets, Frits
AU - Tsang, Steve
AU - Tuerlinckx, Francis
AU - Turchan, Paul
AU - Valasek, Milan
AU - van `t Veer, Anna E.
AU - van Aert, Robbie C. M.
AU - van Assen, Marcel A.
AU - van Bork, Riet
AU - van de Ven, Mathijs
AU - van den Bergh, Don
AU - van der Hulst, Marije
AU - van Dooren, Roel
AU - van Doorn, Johnny
AU - van Renswoude, Daan R.
AU - van Rijn, Hedderik
AU - Vanpaemel, Wolf
AU - Echeverria, Alejandro Vasquez
AU - Vazquez, Melissa
AU - Velez, Natalia
AU - Vermue, Marieke
AU - Verschoor, Mark
AU - Vianello, Michelangelo
AU - Voracek, Martin
AU - Vuu, Gina
AU - Wagenmakers, Eric-Jan
AU - Weerdmeester, Joanneke
AU - Welsh, Ashlee
AU - Westgate, Erin C.
AU - Wissink, Joeri
AU - Wood, Michael
AU - Woods, Andy
AU - Wright, Emily
AU - Wu, Sining
AU - Zeelenberg, Marcel
AU - Zuni, Kellylynn
AU - Collaboration, Open Sci
PY - 2015/8/28
Y1 - 2015/8/28
N2 - Reproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. We conducted replications of 100 experimental and correlational studies published in three psychology journals using high-powered designs and original materials when available. Replication effects were half the magnitude of original effects, representing a substantial decline. Ninety-seven percent of original studies had statistically significant results. Thirty-six percent of replications had statistically significant results; 47% of original effect sizes were in the 95% confidence interval of the replication effect size; 39% of effects were subjectively rated to have replicated the original result; and if no bias in original results is assumed, combining original and replication results left 68% with statistically significant effects. Correlational tests suggest that replication success was better predicted by the strength of original evidence than by characteristics of the original and replication teams.
AB - Reproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. We conducted replications of 100 experimental and correlational studies published in three psychology journals using high-powered designs and original materials when available. Replication effects were half the magnitude of original effects, representing a substantial decline. Ninety-seven percent of original studies had statistically significant results. Thirty-six percent of replications had statistically significant results; 47% of original effect sizes were in the 95% confidence interval of the replication effect size; 39% of effects were subjectively rated to have replicated the original result; and if no bias in original results is assumed, combining original and replication results left 68% with statistically significant effects. Correlational tests suggest that replication success was better predicted by the strength of original evidence than by characteristics of the original and replication teams.
U2 - 10.1126/science.aac4716
DO - 10.1126/science.aac4716
M3 - Article
C2 - 26315443
VL - 349
JO - Science
JF - Science
SN - 0036-8075
IS - 6251
M1 - aac4716
ER -