Analysis of Verified Replication Packages from the American Journal for Political Science

Project Description

Since 2015, the American Journal for Political Science (AJPS) has adopted a Replication and Verification Policy ( that requires researchers to deposit replication materials (including data and code) required to reproduce analytic results reported in published articles. In cooperation with the Odum Institute for Research in Social Science, AJPS has implemented a process for third-party verification. Replication packages deposited since the adoption of this policy should have a higher success rate for computational reproducibility than packages that were not verified.

This internship will explore the rates of reproducibility for packages submitted before and after the policy adoption and implementation of the Odum workflow. Results should be comparable to those reported by Kingi et al for the American Economic Journal. The intern will download a subset of packages from the AJPS Dataverse and attempt to reproduce the reported results (with reasonable effort) using the Whole Tale platform, when possible.

Necessary Prerequisites:

Desirable Skills / Qualifications:

Expected Outcomes:

Primary Mentor: Craig Willis, National Center for Supercomputing Application, University of Illinois at Urbana-Champaign

Secondary Mentor(s): TBD