To review and refine a repository of case simulations developed by the Faculty of Medicine and to enable their use in faculty-wide summative assessments of undergraduate medical students

Principal Supervisor

Professor S.M. Kumta, Department of Orthopaedics and Traumatology
Duration

2 years
Approved Budget

HK $496,341
 
  • Project Objectives
  • Description
  • Evaluation of outcomes
  • Dissemination of results & deliverables

Project Objectives

This project’s aim was to review and refine 210 web-based cases with reference to

  • instructional Design and Educational objectives
  • content Validity and accuracy
  • quality of Feedback
  • scoring and marking adjustments

Description of process and deliverables

We used a modified Bloom’s taxonomy to classify cases on the basis of the type of knowledge and the level of cognitive process involved in answering questions embedded in the cases. Of the original 210 cases, only 145 were reviewed, of which 90 were selected for subsequent use in summative testing. A check-list based scoring scheme has been introduced within the FACS engine. This check-list driven FACS has enabled us to construct tests wherein it is possible to penetrate deeper into the student’s interpretation of radiographic, pathologic and other investigations such as Electocardiographs, targeting the higher cognitive domains of “analysis and evaluation”.

Summative FACS, linked to key learning outcomes, have been developed in the following areas:

Discipline Key Learning Outcome Targeted
Cardiology (medicine) Ability to interpret Electrocardiograms & Identify life threatening changes.
Radiology Ability to interpret Skeletal Radiographs and make judgments.
Orthopaedics Ability to Identify bony lesions and abnormalities on radiographs.
Anaesthesia Performing Pre-operative assessment and identifying potential risks for anaesthesia
Plastic Surgery Ability to Identify and distinguish between cutaneous lesions, particularly between malignant and benign ones.
Upper GI surgery Ability to prioritize, the investigations and management of patients with Upper GI bleeding
Colorectal Surgery Ability to prioritize, the investigations and management of patients with Bleeding PR

Evaluation of outcomes

We have monitored the student’s use of FACS and their progress in terms of scores attained with successive attempts.

We have also collected data on student performance and preferences for FACS based on the type of knowledge and cognitive depth required to answer embedded questions.
We monitored 4480 Logins of 302 Year-5 Students on a set of Electrocardiography related FACS. Students showed significant progress (pre test 52%, post Test 73%).
FACS with higher cognitive demands were repeatedly attempted as compared to FACS with questions based on simple recall or synthesis.

The percentage of the 212 students we tracked and who returned to FACS cases varied with their cognitive demand and was as follows:
Recall Cases 12% (n=25), Analysis/Application 64% (n=135), Evaluation 80% (n=170), Metacognitive knowledge 54% (n=114).

Dissemination of results & deliverables

Preliminary results have been shared at the following key international meetings

  • AMEE Medical Education Conference Malaga 2009
  • HK-China-Taiwan Conference on medical Education 2008
  • Workshop on medical Education – HK May 2009
  • E-Learning Expo CUHK 2008
  • Published in : Anaesth Intensive Care 1009;37: Web-based Formative

Assessment Case Studies: role in a final year medicine two-week anaesthesia course, L.A.H. Critchley, S.M. Kumta, J. Ware, J. Wong.