The Value of Rosh Review Question Bank Performance Analytics to Emergency Medicine Residency Program Directors
|
If you are the presenter of this abstract (or if you cite this abstract in a talk or on a poster), please show the QR code in your slide or poster (QR code contains this URL). |
Abstract
Background: Prior work in cognitive and educational psychology have demonstrated the effectiveness of practice testing as a study technique for physician board examination. Rosh Review is an online educational platform centered around a question bank of board-style multiple choice questions that is used by many emergency medicine resident physicians as part of their preparation for initial board certification. Rosh Review recently introduced a novel package of performance analytics called “PD Dash” for use by residency program directors to gain greater insight into practice question performance by their residents and to help identify knowledge gaps, trends, and make performance predictions.
Objective: This study aimed to identify the degree to which residency program directors used the novel PD Dash functionality and characterize their assessment of its utility and value.
Methods: We conducted a survey of the forty-eight emergency medicine residency program directors who had access to the PD Dash during its inaugural academic year to assess whether they used it and ascertain their perceptions of the value of the analytics it provided.
Results: Twenty-five program directors completed the survey (52%), collectively responsible for approximately 870 resident physicians. Eighty-four percent of respondents had used the PD Dash to analyze the individual performance of their residents on the question bank. Seventy-two percent had compared their residency program’s overall performance to other programs across the country. Of program directors who used the PD Dash, 90% found it to be overall “very useful” or “extremely useful” (4 or 5 on a 5-point Likert scale) and rated the added value of the PD Dash, beyond simple access to the question bank alone, an average 4.1 on a 5-point Likert scale. Eighty-six percent said they were “very likely” or “extremely likely” to renew their access to the PD Dash for the next academic year.
Conclusions: While somewhat limited by the survey’s response rate, a large majority of responding program directors both used the PD Dash and found the data it provides to be useful and valuable. The fact that a large proportion of respondents plan to renew their subscriptions indicates that the PD Dash is a worthwhile addition to the Rosh Review question bank. Future analyses should focus on the specifics of how performance data are used by program directors, the importance of PD Dash data in their educational decision-making, and outcomes-driven research to assess whether the use of Rosh Review and the PD Dash ultimately improve board examination performance.
Objective: This study aimed to identify the degree to which residency program directors used the novel PD Dash functionality and characterize their assessment of its utility and value.
Methods: We conducted a survey of the forty-eight emergency medicine residency program directors who had access to the PD Dash during its inaugural academic year to assess whether they used it and ascertain their perceptions of the value of the analytics it provided.
Results: Twenty-five program directors completed the survey (52%), collectively responsible for approximately 870 resident physicians. Eighty-four percent of respondents had used the PD Dash to analyze the individual performance of their residents on the question bank. Seventy-two percent had compared their residency program’s overall performance to other programs across the country. Of program directors who used the PD Dash, 90% found it to be overall “very useful” or “extremely useful” (4 or 5 on a 5-point Likert scale) and rated the added value of the PD Dash, beyond simple access to the question bank alone, an average 4.1 on a 5-point Likert scale. Eighty-six percent said they were “very likely” or “extremely likely” to renew their access to the PD Dash for the next academic year.
Conclusions: While somewhat limited by the survey’s response rate, a large majority of responding program directors both used the PD Dash and found the data it provides to be useful and valuable. The fact that a large proportion of respondents plan to renew their subscriptions indicates that the PD Dash is a worthwhile addition to the Rosh Review question bank. Future analyses should focus on the specifics of how performance data are used by program directors, the importance of PD Dash data in their educational decision-making, and outcomes-driven research to assess whether the use of Rosh Review and the PD Dash ultimately improve board examination performance.
Medicine 2.0® is happy to support and promote other conferences and workshops in this area. Contact us to produce, disseminate and promote your conference or workshop under this label and in this event series. In addition, we are always looking for hosts of future World Congresses. Medicine 2.0® is a registered trademark of JMIR Publications Inc., the leading academic ehealth publisher.
This work is licensed under a Creative Commons Attribution 3.0 License.