Background and Aims: Fellowship courses have been designed to improve the knowledge, skills, and performance of medical clinicians. The admission exam consisted of four-choice questions, but developing these questions is not easy, so continuous question monitoring is vital to maintain and improve the quality of questions. In this study, the questions of the admission fellowship exam of the cardiology department of Tehran University of Medical Sciences in 2021 and 2022 were analyzed.
Materials and Methods: The structure of 130 four-choice questions was evaluated by using a checklist, and then the taxonomies of each of the questions were determined. Difficulty coefficient, discrimination coefficient, efficiency of distractors, and reliability of the exam were determined and the relationship between them was calculated.
Results: 61% of questions had bugs in the stem and 62% had bugs in the choices part. 66% were in Taxonomy I. 67% of the questions had appropriate difficulty, 58% had appropriate discrimination coefficient and questions with three functioning distractors made up 47%. The reliability of these exams was 0.788 and 0.826. There was a significant correlation between some variables mentioned above.
Conclusion: The reliability of these exams was good, but questions with appropriate difficulty, with appropriate discrimination, or questions with high-performance distractors accounted for approximately only one-half of the questions and there were structural problems in many questions. In addition, most of the questions were in category I.