Application of ChatGPT-generated MCQs in department rotation exit exams for radiology residents
Ma Xilun
Yu Yanxi
Xiao Yubin
Sun Danmiao
Ke Dongping
Mo Yongkang
Mei Wei
Huang Ruibin
Abstract：Objective This study aimed to investigate the potential application of ChatGPT in assessing the department exit exam of radiology residents.Methods We employed the platform designed by OpenAI,which was trained through GPT Builder to generate case-based multiple-choice questions(MCQs),to output personalized case-based MCQs tailored to the different clinical specialties of resident physicians.A total of 33 resident physicians from the Department of Radiology and Internal Medicine at the First Affiliated Hospital of Shantou University Medical College participated in the exam.Feedback evaluations on the examination papers were collected through questionnaires completed by resident physicians and 10 teachers.Subjective and objective evaluations were conducted on the examination papers.Results The difficulty coefficient of the examination paper was rated as"easy"according to both subjective and objective assessments,meeting expected standards.Objective assessment revealed that the level of difficulty for the internal medicine group examination was 0.89,while the difficulty level for the radiology group examination was 0.87.However,the discrimination and the degree of confidence exhibited relatively low values,with a discrimination coefficient of 0.25 for the internal medicine group and 0.28 for the radiology group,as well as a reliability coefficient of 0.39 for the internal medicine group and 0.22 for the radiology group.Both resident physicians and teachers provided high multidimensional evaluation scores for the examination papers,primarily falling within"neutral"and"satisfied"levels.Conclusion Utilizing ChatGPT to generate personalized multiple-choice questions for the department exit exams of radiology residents is feasible and holds broad prospects as well as significant practical value.
Keywords：ChatGPTcase-based multiple-choice questionsdepartment of radiologyexit exam
Publication Date：2025-10-20
Online Publishing Date：2025-10-24(First online date of this platform, not the publication date of the document)
Pages：6( 746-750,760 )