Subjective quality of multiple choice questions used in undergraduate courses in orthopedics and other specialties

Background and Objective: Multiple Choice Questions (MCQs) can sample broad domains of knowledge efficiently and reliably. The MCQs of lower order C1(Cognitive Level1=Recall of knowledge) do not fulfill this purpose and those of higher cognitive order C2 (Cognitive Level2=Interpret) &C3(Cognitive Level3=Analyze) are better at assessing the problem solving capabilities of the student. Every good educational activity must be supported by quality examination to complete the objectives of a curriculum. The objective of the study was to evaluate MCQs presently being used in internal examinations of medical colleges in Lahore.

Methods: Papers consisting of MCQs from Orthopedics other specialties were collected in June 2019 from different medical colleges of Lahore and reviewed by a senior medical teacher without blinding and without his knowing the scores the students had been awarded before. Question statement, clinical scenarios, options and other mistakes were assessed in each item on predetermined criteria. Cognitive level of the item was determined if it was asking for a recall/identify/ analyze response. The results were tabulated and compared in two groups i.e. Miscellaneous and Orthopedics.

Results: Most of the items(total=589) in both groups were of C1 cognitive level though Orthopedics (229) were slightly better (χ2 = 49.882 P-Value = 0.000 (Statistically Significant). Miscellaneous group (360) was better in quality in making clinical scenarios (χ2 = 29.952 P-Value = 0.000 (Statistically Significant) and writing a question statement without confusion. Options were better written in both groups. A good percentage of items needed to be corrected for mistakes in spellings, grammar and segregation into under graduate level.

Conclusions: The cognitive level of assessment tool s MCQs is quiet low in both groups especially clinical scenario construction can be improved. Mistakes in spellings, grammar and conceptual mediocrity is common in both groups.