Predictors of Performance on USMLE Step 1

Background: Ninety-four percent of program directors cited the USMLE Step 1 score as the most important factor in determining applicant competitiveness for residency. Thus, medical students are motivated to attain the highest possible scores. Since Step 1 recently switched to a pass/fail standard, it is important to analyze factors that predict meeting this goal. Objective: To investigate the factors that can influence or predict performance on USMLE Step 1. Methods: We conducted a systematic literature search on PubMed, Web of Science, Scopus and ERIC in 2019.


IntroductIon
The license to practice medicine in the United States requires passing the United States Medical Licensing Examination (USMLE). USMLE Step 1 is the first of the three licensure examinations taken by medical students to assess the examinee's understanding of, and ability to apply, basic science concepts to medical practice. 1 Since grading systems are inconsistent among schools, USMLE Step 1 provides a more objective method of assessment. In fact, 94% of residency program directors put an applicant's USMLE Step 1 score as the most important factor in determining their competitiveness for admission. 2 In February 2020, the Federation of State Medical Boards (FSMB) and the National Board of Medical Examiners ® (NBME ® ), co-sponsors of the United States Medical Licensing Examination ® (USMLE ® ), announced changing the Step 1 score reporting from a three-digit numeric score to reporting only pass/fail with a passing score of 194. 1 This decision is aimed at protecting medical students' wellbeing by decreasing the stress they usually experience while struggling to achieve the highest possible score. In one to two years, when the decision goes into effect, the students' focus will shift toward passing the test rather than achieving a high score. As is the case with the USMLE step 2 Clinical Skills (CS) exam, passing the Step 1 test on the first attempt will carry significant weight in assessing residency applicants' competitiveness.
Due to the significance of Step 1 for both medical students and residency programs, multiple studies have examined factors that may predict Step 1 performance. The goal of this study is to conduct a review of prior research and identify which factors are significantly associated with passing Step 1. The predictors will be categorized as either unmodifiable or modifiable factors. Unmodifiable factors are concrete student characteristics associated with increased performance on USMLE Step 1. Modifiable factors are actions that students can take to increase their performance on USMLE Step 1. Particular emphasis will be placed on modifiable factors.

Methods
A systematic literature search was conducted using PubMed, Web of Science, Scopus and ERIC. A systematic review was chosen to give a clear and inclusive overview of the relevant studies on USMLE Step 1 performance. Key words were a combination of the following: USMLE, Step-1, score, success, predictors. Our criteria specified that articles must be published within the last 15 years (2005-2019), with the most recent publication on May 22, 2019. The initial literature search yielded 275 articles, which were then narrowed to 45 articles. Additional criteria specified that selected articles must focus on the USMLE Step 1 outcome and allopathic medical schools located in the United States. All articles that did not meet these criteria were excluded, resulting in 27 articles that were reviewed and agreed upon by all authors of this study. An additional two studies were identified manually after the initial search process and included after agreement among study investigators about the new data. A total of 29 studies were included in this review. Figure 1 details the process of our systematic literature search based on the addressed criteria.

results
Articles were selected based on the inclusion criteria and subject number with particular attention to those focused on passing/failing USMLE Step 1. While our focus is on passing/failing, the current pool of research focuses on scoring well since the implementation of a pass/fail Step 1 is recent. Of the 29 articles, 3 specifically looked at whether the outcome was pass/fail. Many of the studies examined similar variables and had overlapping findings, such as a positive correlation between MCAT score, multiple choice questions completed, unique Anki cards, review book usage and Step 1. In contrast, some factors did not follow those themes. At one school, students who participated in a peer-led Step 1 review course had higher average Step 1 scores than non-participants. 3,4 Andriole and Jeffe examined race and premedical debt and found a higher likelihood of not passing Step 1 on the first attempt for underrepresented minorities and those who had a premedical debt of at least $50,000. 5 Test anxiety was negatively correlated with Step 1 performance. 6,7 Interestingly, students who had higher perceptions of adequacy had better performance. 8 The perception of a meaningful learning environment, emotional climate and student-student interaction were positively associated with Step 1 performance. 9

dIscussIon unModIfIAble fActors
Unmodifiable factors are concrete student characteristics associated with performance on USMLE Step 1. For the purpose of this article, grades in years 1 and 2 as well as NBME/CBSE performance are considered unmodifiable.

MCAT SCore
The MCAT score was a strong predictor of Step 1 performance. 10 Individuals with MCAT scores ≤29 (pre-2015) were more likely to not pass Step 1 on the first try. 5 In addition, as the number of times an individual took the MCAT increased, the Step1 score decreased. 11 The correlation between MCAT performance and Step 1 achievement is not consistent across all schools. Ghaffari-Rafi et al. found students at a university with lower MCAT scores outperformed their expected average on USMLE Step 1. 12 Therefore, MCAT scores cannot be said to be the sole predictor of performance as this effect may be mediated by curriculaspecific to individual medical schools (e.g., integrating early clinical exposure into years 1 and 2).

UndergrAdUATe gPA
There is a positive relationship between the undergraduate GPA and unimpeded progress toward medical school graduation, such as passing USMLE Step 1; however, this effect levels off as the GPA increases above 3.5. 13 There was also a statistically significant correlation between the average Step 1 score and the median GPA. 12

Pre-CliniCAl grAdeS
Other researchers found that performance in pre-clinical classes was a much stronger predictor of USMLE step 1 performance than pre-admission variables such as MCAT scores. 14 In particular, Giordano et al. found a significant correlation with the USMLE Step 1 score and straight As during the first year of medical school. 15 Achievement in individual courses was also strongly associated with Step 1 performance. Anatomy is a strong overall predictor, with both class rank in medical gross anatomy and score on a comprehensive gross anatomy final significantly correlating with passing Step 1. 16 This effect is also seen among students who opted to take a gross anatomy course immediately prior to medical school matriculation. 17 In addition, a specialized focus in Anatomical Sciences positively influences USMLE Step 1 performance. Fredieu and Pre-admission data, NBME scores The prediction model (P < .0001) was accounted for by MCAT (21%) and NBME scores from the first two years (40-50%), predicting 63% of at-risk students. MCAT score, NBME scores The following NBME scores were significantly and positively associated with Step Men averaged 1.5 points higher than women on Step 1. Undergraduate science GPA was more associated with Step 1 for women than for men (7.58 and 6.60 point increase, respectively, in Step 1 score for every one-point increase in GPA). There is a positive correlation between median matriculant MCAT score and average USMLE step 1 score performance (corr = 0.72, P < 0.0001). Giordano  Performance on Step 1 was significantly correlated with performance on both NBME and CBSE exams (r = 0.730, P < 0.001). With the same average MCAT score, MCAT repeaters achieve similar Step 1 scores as non-repeaters. For a given MCAT score (not using the average score approach), MCAT repeaters are expected to achieve lower Step 1 scores than non-repeaters (p < 0.01).

Variables Findings
Snyder found that USMLE step 1 scores were significantly higher for students enrolled in their MD/MS program than for the MD program. 18 This enhanced performance may be due to serving as anatomy TAs for the medical school by providing the opportunity to further increase their knowledge.
Performance in pre-clinical courses during year 2 also had a strong predictive ability. Performance in these courses accurately predicted 97.2% of those who passed and 80% of those who failed Step 1. 19 Two courses with the highest failure rate, Organ Systems and Human Structure, were significantly correlated with USMLE Step 1 performance. 20 Pathology coursework was also found to be a predictor of USMLE step 1 scores. 21

CBSe/nBMe
Standardized exams created by the National Board of Medical Examiners were also found to be predictors of USMLE Step 1 performance. The Comprehensive Basic Sciences Exam (CBSE) allows students to gauge readiness for USMLE Step 1. Performance on USMLE Step 1 is significantly correlated with the performance on the CBSE taken immediately preceding the dedicated study period. 15 In addition, CBSE results with scores less than 66 were significantly associated with failure of USMLE Step 1. 22 deMogrAPhiCS Gender, student debt, underrepresented minority status, age, and clinical experience were predictors of USMLE Step 1 performance. 23 After controlling for MCAT scores and undergraduate GPAs, men scored 1.5 points higher on Step 1 than women. In addition to female gender, underrepresented minorities in medicines, older matriculation age, and premedical debt of at least $50,000 were associated with increased risk of poor performance on Step 1. 5 However, Giordano et al. found that financial need while in medical school is a statistically significant positive predictor of USMLE Step 1 score. 15 Students with formal clinical experience prior to medical school perform better on USMLE Steps 1 and 2 than their counterparts without clinical experience. 24 Formal clinical experience is defined as employment in the healthcare field for any amount of time or scribing for at least 6 months prior to matriculation. Shadowing was not considered clinical experience. In conjunction, these studies suggest that if a student is going to take time off between college and medical school, working in the healthcare field may be beneficial.
The demographics of the medical school have also been shown to influence the USMLE 1score. National Institutes of Health research grants provided to either the medical school itself or to individual faculty members are associated with better student performance on Step 1. 12 In addition, students attending a private medical school scored, on average, 5 points higher than students attending public medical schools.
In contrast to many other studies, Giordano et al. did not find a relationship between the USMLE Step 1 Score and MCAT score, gender, prior medical training, or age. 15

ModIfIAble fActors
Modifiable factors are actions that students can take to increase their performance on USMLE Step 1. These include studying prior to the designated study period, utilizing a 700-page review book, multiple choice questions, and Anki cards, participating in peer led tutoring, timing of when Step 1 is taken, and psychological factors.

STUdy hABiTS
Students who began studying prior to the dedicated study period and increased their study time by averaging 11.0 hours per day over a period of 35.3 days performed 4.2 points better on USMLE Step 1. 25 Each additional cover to cover reading of the review book was associated with a 2.3 point increase. These effects remained even after controlling for MCAT scores, pre-clinical exam performance, and self-identified score goal. Each additional 445 boards-style questions or 1700 unique Anki flashcards were associated with a one-point increase on Step 1. 6 The percentage correct of USMLE World (UWorld) Question Bank was also associated with increased performance on Step 1. 15 Commercial preparatory courses were not associated with higher USMLE Step 1 score. 26

Peer led TUToring And groUP STUdy
Participation in a peer-led review course was associated with higher Step 1 scores. 3 Although many participants felt that the course did not decrease their anxiety or increase confidence in their performance, the presence of anxiety may have been a powerful motivating factor as demonstrated by the higher scores of participants than nonparticipants.
Although half of students surveyed reported group studying was helpful to them, it was not significantly correlated with exam performance. 27

TiMing of USMle STeP 1
The timing of when Step 1 was taken was not significantly associated with differences in scores as long as the exam was taken within two months of finishing second year courses. After two months, scores declined as time between finishing pre-clinical material and taking Step1 increased. 26

PSyChologiCAl fACTorS
Psychological factors, such as burnout, feelings of depression, and test anxiety, were examined by Alcamo et al. 3 Test anxiety, depression, and burnout were all significant negative predictors of exam performance. 6 Another study by Green et al. supports this relationship by demonstrating a modest inversely correlation between test anxiety and USMLE step 1 scores. 7 Anxiety of students who participated in a test-taking course decreased after completing Step 1 while anxiety among the control group increased. However, the control group still outperformed the intervention group on Step 1.

liMiTATionS
One of the major limitations to the present study was the relatively few studies that focused on modifiable factors rather than unmodifiable factors (i.e., MCAT score, gender, pre-clinical grades, etc.). Studies that did focus on modifiable factors predominantly examined board preparatory resources, which were correlated with improved scores across the board for all students; however, it may be of greater benefit to instead categorize resources based on student learning styles. Categorizing board preparatory resources using these parameters may allow students to gain maximum benefit from their study efforts. For instance, Kim et al. found that students with reading/writing learning style preferences performed better on Step 1 than those with multimodal preferences. 28 This conclusion is logical considering that Step 1 requires students to read text-heavy questions and extract pertinent information in a time efficient manner. It follows that students with a predisposed affinity for reading/writing tend to score higher than students who do not have this learning style preference. However, Lynch et al. found a significant positive correlation between Step 1 scores and abstract thinking. 29 Considering that both of these studies show improved scores based on students' innate mental qualities, future studies that correlate learning preferences with appropriately oriented preparatory resources could have important implications for students.
Furthermore, it is plausible that categorizing factors associated with modifiable factors is an unachievable task as many factors contribute to success on USMLE Step 1. These factors could include differing motivations for achievement, differing student instruction among medical schools, and emphasis on Step 1 success by different specialties.

conclusIon
Many studies focused on unmodifiable factors, such as MCAT score, gender and pre-clinical grades; however, there are also several modifiable factors that are strongly associated with a higher score on a student's USMLE Step 1 score. Specifically, medical students can increase the number of multiple choice questions completed, unique Anki cards seen, and complete passes of First Aid for the USMLE Step 1 to increase their scores. However, the work effort needed to add one point to students ' Step scores can be inefficient, discouraging, and unrealistic. This finding may