Published multi-lab studies as of 2023-02-23

All published human multi-lab studies I could find as of 2023-02-23.

Anton Olsson-Collentine true
2023-02-23

Back in 2018 we wanted examine meta-analytic heterogeneity in research in social and cognitive psychology when there were only minor differences between primary studies, so we went looking for multi-lab replication projects. At the time, we found 10 multi-lab projects and during the years it took to publish the study (https://doi.org/10.1037/bul0000294) we added 3 more for a total of 13 projects with 68 primary effect sizes.

Recently I was looking for an updated list of multi-lab projects and couldn’t find one, so I collated my own. Below are all the published (human) multi-lab projects that I have been able to find at the time of writing (2023-02-23). There are now an impressive 32 human multi-lab projects published.

title author year link
In COVID-19 Health Messaging, Loss Framing Increases Anxiety with Little-to-No Concomitant Benefits: Experimental Evidence from 84 Countries Dorison et al.  2022 doi
Situational Factors Shape Moral Judgements in the Trolley Dilemma in Eastern, Southern and Western Countries in a Culturally Diverse Sample Bago et al.  2022 doi
A Global Experiment on Motivating Social Distancing during the COVID-19 Pandemic Psychological Science Accelerator Self-Determination Theory Collaboration 2022 doi
Many Labs 4: Failure to Replicate Mortality Salience Effect With and Without Original Author Involvement Klein et al.  2022 doi
#EEGManyLabs: Investigating the Replicability of Influential EEG Experiments Pavlov et al.  2021 doi
To Which World Regions Does the Valence- Dominance Model of Social Perception Apply? Jones et al.  2021 doi
A Multi-Country Test of Brief Reappraisal Interventions on Emotions during the COVID-19 Pandemic Wang et al.  2021 doi
Incidental Attitude Formation via the Surveillance Task: A Preregistered Replication of the Olson and Fazio (2001) Study Moran et al.  2021 doi
A Multilab Replication of the Ego Depletion Effect Dang et al.  2021 doi
A Multilab Study of Bilingual Infants: Exploring the Preference for Infant-Directed Speech Byers-Heinlein et al.  2021 doi
Multilab Direct Replication of Flavell, Beach, and Chinsky (1966): Spontaneous Verbal Rehearsal in a Memory Task as a Function of Age Elliott et al.  2021 doi
ManyClasses 1: Assessing the Generalizable Effect of Immediate Feedback Versus Delayed Feedback Across Many College Classes Fyfe et al.  2021 doi
A Multi-Site Collaborative Study of the Hostile Priming Effect McCarthy et al.  2021 doi
Reexamining the Effect of Gustatory Disgust on Moral Judgment: A Multilab Direct Replication of Eskine, Kacinik, and Prinz (2011) Ghelfi et al.  2020 doi
Registered Replication Report on Fischer, Castel, Dodd, and Pratt (2003) Colling et al.  2020 doi
Many Labs 5: Testing Pre-Data-Collection Peer Review as an Intervention to Increase Replicability Ebersole et al.  2020 doi
A Demonstration of the Collaborative Replication and Education Project: Replication Attempts of the Red-Romance Effect Wagge et al.  2019 doi
Registered Replication Report: Dijksterhuis and van Knippenberg (1998): O’Donnell et al.  2018 doi
Registered Replication Report on Srull and Wyer (1979) McCarthy et al.  2018 doi
Registered Replication Report on Mazar, Amir, and Ariely (2008) Verschuere et al.  2018 doi
Many Labs 2: Investigating Variation in Replicability Across Samples and Settings Klein et al.  2018 doi
Self-Esteem, Self-Disclosure, Self-Expression, and Connection on Facebook: A Collaborative Replication Meta-Analysis Leighton et al.  2018 doi
An Investigation of Abstract Construal on Impression Formation: A Multi-Lab Replication of McCarthy and Skowronski (2011) McCarthy et al.  2018 doi
Registered Replication Report: Rand, Greene, and Nowak (2012) Bouwmeester et al.  2017 doi
The Pipeline Project: Pre-publication Independent Replications of a Single Laboratory’s Research Pipeline Schweinsberg et al.  2016 doi
Many Labs 3: Evaluating Participant Pool Quality across the Academic Semester via Replication Ebersole et al.  2016 doi
Registered Replication Report: Hart & Albarrac’in (2011) Eerland et al.  2016 doi
A Multilab Preregistered Replication of the Ego-Depletion Effect Hagger et al.  2016 doi
Registered Replication Report: Study 1 From Finkel, Rusbult, Kumashiro, & Hannon (2002) Cheung et al.  2016 doi
Registered Replication Report: Strack, Martin, & Stepper (1988) Wagenmakers et al.  2016 doi
Investigating Variation in Replicability: A ‘Many Labs’ Replication Project Klein et al.  2014 doi
Registered Replication Report: Schooler and Engstler-Schooler (1990) Alogna et al.  2014 doi
Alogna, V. K., M. K. Attaya, P. Aucoin, Š Bahník, S. Birch, A. R. Birt, B. H. Bornstein, et al. 2014. “Registered Replication Report: Schooler and Engstler-Schooler (1990).” Perspectives on Psychological Science 9 (September): 556–78. https://doi.org/10.1177/1745691614545653.
Bago, Bence, Marton Kovacs, John Protzko, Tamas Nagy, Zoltan Kekecs, Bence Palfi, Matus Adamkovic, et al. 2022. “Situational Factors Shape Moral Judgements in the Trolley Dilemma in Eastern, Southern and Western Countries in a Culturally Diverse Sample.” Nature Human Behaviour 6 (6): 880–95. https://doi.org/10.1038/s41562-022-01319-5.
Bouwmeester, S., P. P. J. L. Verkoeijen, B. Aczel, F. Barbosa, L. Bègue, P. Brañas-Garza, T. G. H. Chmura, et al. 2017. “Registered Replication Report: Rand, Greene, and Nowak (2012).” Perspectives on Psychological Science 12 (3): 527–42. https://doi.org/10.1177/1745691617693624.
Byers-Heinlein, Krista, Angeline Sin Mei Tsui, Christina Bergmann, Alexis K. Black, Anna Brown, Maria Julia Carbajal, Samantha Durrant, et al. 2021. “A Multilab Study of Bilingual Infants: Exploring the Preference for Infant-Directed Speech.” Advances in Methods and Practices in Psychological Science 4 (1): 2515245920974622. https://doi.org/10.1177/2515245920974622.
Cheung, I., L. Campbell, E. P. LeBel, R. A. Ackerman, B. Aykutoğlu, Š. Bahník, J. D. Bowen, et al. 2016. “Registered Replication Report: Study 1 From Finkel, Rusbult, Kumashiro, & Hannon (2002).” Perspectives on Psychological Science 11 (5): 750–64. https://doi.org/10.1177/1745691616664694.
Colling, Lincoln J., Dénes Szűcs, Damiano De Marco, Krzysztof Cipora, Rolf Ulrich, Hans-Christoph Nuerk, Mojtaba Soltanlou, et al. 2020. “Registered Replication Report on Fischer, Castel, Dodd, and Pratt (2003).” Advances in Methods and Practices in Psychological Science 3 (2): 143–62. https://doi.org/10.1177/2515245920903079.
Dang, Junhua, Paul Barker, Anna Baumert, Margriet Bentvelzen, Elliot Berkman, Nita Buchholz, Jacek Buczny, et al. 2021. “A Multilab Replication of the Ego Depletion Effect.” Social Psychological and Personality Science 12 (1): 14–24. https://doi.org/10.1177/1948550619887702.
Dorison, Charles A., Jennifer S. Lerner, Blake H. Heller, Alexander J. Rothman, Ichiro I. Kawachi, Ke Wang, Vaughan W. Rees, et al. 2022. “In COVID-19 Health Messaging, Loss Framing Increases Anxiety with Little-to-No Concomitant Benefits: Experimental Evidence from 84 Countries.” Affective Science 3 (3): 577–602. https://doi.org/10.1007/s42761-022-00128-3.
Ebersole, Charles R., Olivia E. Atherton, Aimee L. Belanger, Hayley M. Skulborstad, Jill M. Allen, Jonathan B. Banks, Erica Baranski, et al. 2016. “Many Labs 3: Evaluating Participant Pool Quality Across the Academic Semester via Replication.” Journal of Experimental Social Psychology, Special issue: Confirmatory, 67 (November): 68–82. https://doi.org/10.1016/j.jesp.2015.10.012.
Ebersole, Charles R., Maya B. Mathur, Erica Baranski, Diane-Jo Bart-Plange, Nicholas R. Buttrick, Christopher R. Chartier, Katherine S. Corker, et al. 2020. “Many Labs 5: Testing Pre-Data-Collection Peer Review as an Intervention to Increase Replicability.” Advances in Methods and Practices in Psychological Science 3 (3): 309–31. https://doi.org/10.1177/2515245920958687.
Eerland, A., A. M. Sherrill, J. P. Magliano, R. A. Zwaan, J. D. Arnal, P. Aucoin, S. A. Berger, et al. 2016. “Registered Replication Report: Hart & Albarracín (2011).” Perspectives on Psychological Science 11 (1): 158–71. https://doi.org/10.1177/1745691615605826.
Elliott, Emily M., Candice C. Morey, Angela M. AuBuchon, Nelson Cowan, Chris Jarrold, Eryn J. Adams, Meg Attwood, et al. 2021. “Multilab Direct Replication of Flavell, Beach, and Chinsky (1966): Spontaneous Verbal Rehearsal in a Memory Task as a Function of Age.” Advances in Methods and Practices in Psychological Science 4 (2): 25152459211018187. https://doi.org/10.1177/25152459211018187.
Fyfe, Emily R., Joshua R. de Leeuw, Paulo F. Carvalho, Robert L. Goldstone, Janelle Sherman, David Admiraal, Laura K. Alford, et al. 2021. “ManyClasses 1: Assessing the Generalizable Effect of Immediate Feedback Versus Delayed Feedback Across Many College Classes.” Advances in Methods and Practices in Psychological Science 4 (3): 25152459211027575. https://doi.org/10.1177/25152459211027575.
Ghelfi, Eric, Cody D. Christopherson, Heather L. Urry, Richie L. Lenne, Nicole Legate, Mary Ann Fischer, Fieke M. A. Wagemans, et al. 2020. “Reexamining the Effect of Gustatory Disgust on Moral Judgment: A Multilab Direct Replication of Eskine, Kacinik, and Prinz (2011).” Advances in Methods and Practices in Psychological Science 3 (1): 3–23. https://doi.org/10.1177/2515245919881152.
Hagger, M. S., N. L. D. Chatzisarantis, H. Alberts, C. O. Anggono, C. Batailler, A. R. Birt, R. Brand, et al. 2016. “A Multilab Preregistered Replication of the Ego-Depletion Effect.” Perspectives on Psychological Science 11 (4): 546–73. https://doi.org/10.1177/1745691616652873.
Jones, Benedict C., Lisa M. DeBruine, Jessica K. Flake, Marco Tullio Liuzza, Jan Antfolk, Nwadiogo C. Arinze, Izuchukwu L. G. Ndukaihe, et al. 2021. “To Which World Regions Does the Valencedominance Model of Social Perception Apply?” Nature Human Behaviour 5 (1): 159–69. https://doi.org/10.1038/s41562-020-01007-2.
Klein, Richard A., Kate A. Ratliff, Michelangelo Vianello, Reginald B. Adams, Štěpán Bahník, Michael J. Bernstein, Konrad Bocian, et al. 2014. “Investigating Variation in Replicability: A ‘Many Labs’ Replication Project.” Social Psychology 45 (3): 142–52. https://doi.org/10.1027/1864-9335/a000178.
Klein, Richard A., Michelangelo Vianello, Fred Hasselman, Byron G. Adams, Reginald B. Adams, Sinan Alper, Mark Aveyard, et al. 2018. “Many Labs 2: Investigating Variation in Replicability Across Samples and Settings.” Advances in Methods and Practices in Psychological Science 1 (4): 443–90. https://doi.org/10.1177/2515245918810225.
Klein, Richard A, Corey L. Cook, Charles R. Ebersole, Christine Vitiello, Brian A. Nosek, Joseph Hilgard, Paul Hangsan Ahn, et al. 2022. “Many Labs 4: Failure to Replicate Mortality Salience Effect With and Without Original Author Involvement.” Collabra: Psychology 8 (1): 35271. https://doi.org/10.1525/collabra.35271.
Leighton, Dana C., Nicole Legate, Sara LePine, Samantha F. Anderson, and Jon Grahe. 2018. “Self-Esteem, Self-Disclosure, Self-Expression, and Connection on Facebook: A Collaborative Replication Meta-Analysis.” Psi Chi Journal of Psychological Research 23: 98–109. https://doi.org/10.24839/2325-7342.JN23.2.98.
McCarthy, Randy J., Jessica L. Hartnett, Jeremy D. Heider, Cory R. Scherer, Sarah E. Wood, Austin Lee Nichols, John E. Edlund, and W. Richard Walker. 2018. “An Investigation of Abstract Construal on Impression Formation: A Multi-Lab Replication of McCarthy and Skowronski (2011).” International Review of Social Psychology. https://doi.org/10.5334/irsp.133.
McCarthy, Randy J., John J. Skowronski, Bruno Verschuere, Ewout H. Meijer, Ariane Jim, Katherine Hoogesteyn, Robin Orthey, et al. 2018. “Registered Replication Report on Srull and Wyer (1979).” Advances in Methods and Practices in Psychological Science 1 (3): 321–36. https://doi.org/10.1177/2515245918777487.
McCarthy, Randy, Will Gervais, Balazs Aczel, Rosemary L. Al-Kire, Mark Aveyard, Silvia Marcella Baraldo, Lemi Baruh, et al. 2021. “A Multi-Site Collaborative Study of the Hostile Priming Effect.” Collabra: Psychology 7 (1): 18738. https://doi.org/10.1525/collabra.18738.
Moran, Tal, Sean Hughes, Ian Hussey, Miguel A. Vadillo, Michael A. Olson, Frederik Aust, Karoline Bading, et al. 2021. “Incidental Attitude Formation via the Surveillance Task: A Preregistered Replication of the Olson and Fazio (2001) Study.” Psychological Science 32 (1): 120–31. https://doi.org/10.1177/0956797620968526.
O’Donnell, Michael, Leif D. Nelson, Evi Ackermann, Balazs Aczel, Athfah Akhtar, Silvio Aldrovandi, Nasseem Alshaif, et al. 2018. “Registered Replication Report: Dijksterhuis and van Knippenberg (1998):” Perspectives on Psychological Science 13 (2): 268–94. https://doi.org/10.1177/1745691618755704.
Pavlov, Yuri G., Nika Adamian, Stefan Appelhoff, Mahnaz Arvaneh, Christopher S. Y. Benwell, Christian Beste, Amy R. Bland, et al. 2021. “#EEGManyLabs: Investigating the Replicability of Influential EEG Experiments.” Cortex 144 (November): 213–29. https://doi.org/10.1016/j.cortex.2021.03.013.
Psychological Science Accelerator Self-Determination Theory Collaboration. 2022. “A Global Experiment on Motivating Social Distancing During the COVID-19 Pandemic.” Proceedings of the National Academy of Sciences 119 (22): e2111091119. https://doi.org/10.1073/pnas.2111091119.
Schweinsberg, Martin, Nikhil Madan, Michelangelo Vianello, S. Amy Sommer, Jennifer Jordan, Warren Tierney, Eli Awtrey, et al. 2016. “The Pipeline Project: Pre-publication Independent Replications of a Single Laboratory’s Research Pipeline.” Journal of Experimental Social Psychology, Rigorous and Replicable Methods in Social Psychology, 66 (September): 55–67. https://doi.org/10.1016/j.jesp.2015.10.001.
Verschuere, Bruno, Ewout H. Meijer, Ariane Jim, Katherine Hoogesteyn, Robin Orthey, Randy J. McCarthy, John J. Skowronski, et al. 2018. “Registered Replication Report on Mazar, Amir, and Ariely (2008).” Advances in Methods and Practices in Psychological Science 1 (3): 299–317. https://doi.org/10.1177/2515245918781032.
Wagenmakers, E.-J., T. Beek, L. Dijkhoff, Q. F. Gronau, A. Acosta, R. B. Adams, D. N. Albohn, et al. 2016. “Registered Replication Report: Strack, Martin, & Stepper (1988).” Perspectives on Psychological Science 11 (6): 917–28. https://doi.org/10.1177/1745691616674458.
Wagge, Jordan R., Cristina Baciu, Kasia Banas, Joel T. Nadler, Sascha Schwarz, Yanna Weisberg, Hans IJzerman, Nicole Legate, and Jon Grahe. 2019. “A Demonstration of the Collaborative Replication and Education Project: Replication Attempts of the Red-Romance Effect.” Edited by Simine Vazire and Randy McCarthy. Collabra: Psychology 5 (1): 5. https://doi.org/10.1525/collabra.177.
Wang, Ke, Amit Goldenberg, Charles A. Dorison, Jeremy K. Miller, Andero Uusberg, Jennifer S. Lerner, James J. Gross, et al. 2021. “A Multi-Country Test of Brief Reappraisal Interventions on Emotions During the COVID-19 Pandemic.” Nature Human Behaviour 5 (8): 1089–1110. https://doi.org/10.1038/s41562-021-01173-x.

References

Citation

For attribution, please cite this work as

Olsson-Collentine (2023, Feb. 23). Anton Olsson-Collentine: Published multi-lab studies as of 2023-02-23. Retrieved from olssoncollentine.com/posts/multi-lab-studies-as-of-2023-02-23/

BibTeX citation

@misc{olsson-collentine2023published,
  author = {Olsson-Collentine, Anton},
  title = {Anton Olsson-Collentine: Published multi-lab studies as of 2023-02-23},
  url = {olssoncollentine.com/posts/multi-lab-studies-as-of-2023-02-23/},
  year = {2023}
}