Can ACGME Milestones predict surgical specialty board passage: an example in Obstetrics and Gynecology

Abstract

Can ACGME Milestones predict surgical specialty board passage: an example in Obstetrics and Gynecology Sarah Ottum1 , Conrad Chao2,§ , Sejal Tamakuwala3 , Joshua Dean4 , Adib Shafi5 , Katherine Jennifer Kramer6 , Satinder Kaur7,§ , Maurice-Andre Recanati8, * 1Department of Obstetrics and Gynecology, University of Cincinnati, Cincinnati, OH 45267, USA 2Department of Maternal-Fetal Medicine, University of New Mexico School of Medicine, Albuquerque, NM 87131, USA 3Department of Obstetrics and Gynecology, Emory University, Atlanta, GA 30322, USA 4Department of Obstetrics and Gynecology, Beaumont, Royal Oak, MI 48073, USA 5Department of Computer Sciences, Wayne State University, Detroit, MI 48201, USA 6Department of Obstetrics and Gynecology, St. Vincent’s Catholic Medical Centers, New York, NY 10011, USA 7Department of Obstetrics and Gynecology, Wayne State University, Detroit, MI 48201, USA 8Department of Obstetrics and Gynecology and NIH-Women’s Reproductive Health Research Scholar, Wayne State University, Detroit, MI 48201, USA *Correspondence: marecanati@wayne.edu (Maurice-Andre Recanati) § Current or former program director or chair. DOI:10.31083/j.ceog4805168 This is an open access article under the CC BY 4.0 license (https://creativecommons.org/licenses/by/4.0/). Submitted: 23 May 2021 Revised: 13 July 2021 Accepted: 20 July 2021 Published: 15 October 2021 Background: Multiple tools including Accreditation Council for Graduate Medical Education (ACGME) standardized milestones can be utilized to assess trainee and residency program performance. However, little is known regarding the objective validation of these tools in predicting written board passage. Methods: In this retrospective study, data was gathered on n = 45 Wayne State University Obstetrics and Gynecology program graduates over the five-year period ending July 2018. United States Medical Licensing Examination (USMLE) scores, Council on Resident Education in Obstetrics and Gynecology (CREOG) in-training scores and ACGME milestones were used to predict American Board of Obstetrics and Gynecology (ABOG) board passage success on first attempt. Significance was set at p < 0.05. Results: Written board passage was associated with average CREOGs (p = 0.01) and milestones (p = 0.008) while USMLE1 was not significantly associated (p = 0.055). USMLE1 <217 (Positive predictive value (PPV) = 96%). CREOGs <197 (PPV = 100%) and milestones <3.25 (PPV = 100%), particularly practice-based learning and systems-based practice milestones were most strongly correlated with board failure. Using a combination of these two milestones, it is possible to correctly predict board passage using our model (PPV = 86%). Discussion: This study is the first validating the utility of milestones in a surgical specialty by demonstrating their ability to predict board passage. Residents with CREOGs or milestones below thresholds are at risk for board failure and may warrant early intervention. Keywords CREOG; ABOG; Board passage; Milestones; USMLE