To evaluate the assessment model, both quantitative and qualitative type methods were used to collect feedbacks from students and teachers. A questionnaire had been developed to collect students’ feedbacks. Teachers of The Chinese University of Hong Kong who wish to use an on-line version of the questionnaire please contact us. The remaining of this section presents evaluation results based on the analysis of the cases completed in the academic year 2005/2006. Evaluation Methodology Evaluation focused on the learning impact and practicality of the new assessment strategies. For the first several pilot case studies, most of the evaluation data was qualitative data obtained through interviewing the various parties involved in the courses – the teachers, the students, and the teaching assistant who progressively became more involved in the project (e.g. in running the voting process for the assessment criteria, and in calculating the various teacher-, self- and peer-marks). The guiding questions for the teacher and student interviews were: Can you describe the rationale behind self- and peer-assessment? Do you/students agree with this rationale? Do you/students think there really is a learning benefit? How did students derive assessment criteria? What process did students use in deciding on self-assessment marks? What process did students use in deciding on peer-assessment marks? Do you/students think it is fair to use self-assessment marks? Do you/students think it is fair to use peer-assessment marks? Should a system to elicit within-group contributions be added to differentiate individual scores within groups? If yes, how might this be done? (obligatory/ optional? collaborative/ individual moderation?) How generally applicable do you think this method for group assignments in other subjects (e.g. business subjects)? What do students think of the assessment criteria after actually using them in grading? Any additions or deletions? Any other suggestions about the marking scheme? back to the top Perceived benefits The reactions of teachers, students and teaching assistants involved in the case studies were very positive about the new assessment strategies. There were suggestions for changes and improvements, for example in cutting the workload, fine-tuning the logistics of arriving at the marking criteria, or adjusting the best ways to calculate the scores. However, these constructive comments were made on the basis that they supported the new strategies, realized the learning benefits that could result, favoured the continuation of the practice in the course, and agreed that similar tactics could be used to improve other courses. Learning Potential Students who participated in the focus-group meetings agreed with the rationale behind the practice. The teachers had explained the meaning of the practice at the beginning of the courses and the students could articulate their understanding well. Students, in general, appreciated that the new method had assisted learning in the following ways: becoming aware of the requirements of good performance in learning activities such as presentations and research reports; providing them a framework to improve their work; enabling them to plan how to study; assisting them to judge the quality of work in the field; clarifying the course objectives; motivating them to treat the course assignments seriously; becoming more conscious about how others view their own performance; and being conscious about how their own work is compared with the others. Students remarked that they appreciated the requirements of good presentations and reports through selecting the criteria themselves. They agreed that students benefited through being involved in the criteria-making process. They would have gained less if the teachers had provided the final marking scheme and no student participation were involved. The criteria acted as a guideline and framework to direct the flow and the components of the presentations and reports. For example, one student in Course B noted that the level of performance in the course presentations was generally better than that in the other courses she studied. One student in Course C stated that she had planned her course study more effectively because she knew the main points of the course at the beginning. Students agreed that the self- and peer-assessment aspects assisted them to correctly judge the nature of work in this field. They also considered that being able to make judgments on work is very important to learning. Because of the marking activities, they paid more attention to the performance of others. They noted whether other groups had met all the criteria or not. In addition, they reflected on what criteria they had met and compared their performance with other groups. This provided students with a very good opportunity for them to self-reflect on their performance, and they recognized and appreciated this. They also became more objective in coming up with the self-assessment marks and felt this would assist them in making improvements in the future. back to the top The teaching assistant interviewed in Course C also thought that the system was meaningful because it gave students a chance to think about what criteria were important to the evaluation of work in their field. Knowing what aspects were more important at the beginning of the course gave students a clear objective in the course. She thought that the idea of self- and peer-assessment was fair. Students knew how the marks were allocated and awarded, which meant that the process was more transparent. Since students were allowed to participate in the marking process, they did the assignments seriously. Overall, she thought that students learnt more in the course because of the new assessment strategies. Teachers also observed students’ high level of engagement in the assessment tasks. One of the main goals of the new practice was to encourage students to seriously judge the quality of work. This goal was clearly achieved. Students also remarked that they had become very serious in assessing themselves and others. When asked how they could come up with self-assessment marks, all the focus-group participants said that they had good discussions with all the other group members. They were also more conscious about how teachers and peers viewed their performance. Some students remarked that the ‘Q and A’ session after their own presentation was a good opportunity for them to collect feedback from the teachers and the students about their performance. After receiving the opinions from others, they could make use of the comments to adjust their self-assessment marks. Lastly, they were more conscious about how their work was compared with the others. One student reflected on the strategy used and said he compared the performance of several group presentations before giving self- and peer-scores. Other students used the performance of their group as a reference for marking other groups. back to the top Practicality The assessment strategies could be run reasonably easily in courses. The following points could be generalized from the feedback concerning the various aspects of the practicality of the method. The process is not time-consuming. The workload on the teacher and teaching assistant is manageable. The method can be implemented in a variety of courses. The added self- and peer-contributions enhance students’ perceptions of the fairness of assessm There were concerns about excessive workload from students in the early case but the issue was resolved as the logistics improved. In Course B, for example, students remarked that the whole process was not really time-consuming. They really liked the marking system. The teaching assistant in Course C also thought that the existing mechanism was fine. Students made proposals for the calculations and the whole class discussed the system. Everything was discussed, clarified and decided with the involvement of all course members. She thought that the resultant workload on the teaching assistant was manageable. Students generally agreed these assessment strategies system could be implemented in other courses and students would benefit. The approach might be more applicable to courses having projects or presentations as assignments. It helped students to reflect upon their own performance as well as eliminating elements of unfair marking by avoiding sole dependence on the teacher’s subjective impression. Students noted that the strategies might not work so well in large classes though, or might need adaptation. The workload would be too high if students had to rate the performance of all groups, though students recognized there could be a number of ways of clustering or rostering groups so that each group had a small number of other groups to review. The method might not be appropriate for courses with a strong focus on teaching key concepts only. The method is potentially of most benefit in courses that focus on other levels of learning: for example, courses that have a focus on students building good learning habits, solving problems and applying concepts in authentic situations. Also, repetition would be a problem if more courses use a similar set of assessment strategies. One student of Course C noted that he felt he had adequately learnt how to make self-assessments from the two assessment activities in the course. He thought that repeating the method would not yield much larger learning outcomes and could become repetitive. back to the top