Adapting and Surviving the Move to Online SESs & TEQs
Duquesne is piloting a change in how it administers the Student Evaluation Surveys (SESs) and Law TEQs. Most students will complete the surveys online from November 14 at 9:00 a.m. until December 6 at 5:00 p.m. on Blackboard's "MY Surveys / SES / TEQ" tab or through a free mobile app for smartphones. While students' identities will remain anonymous, faculty will be able to monitor the submission response rates for their courses. The online surveys will afford a quicker processing time, and faculty will receive the results about a month after grades are submitted.
Why are schools replacing paper surveys with an online format?
According to Johnson (2003), paper surveys are "a costly, time consuming process that is inconvenient for faculty and often restricts the thoughtfulness and depth of student responses." The benefits of online student ratings include "ease of administration; more complete data collection; longer, more thoughtful student responses; reduced processing time and costs; more accurate data collection and reporting; and more detailed, user friendly reports" (Johnson, 2003).
What are faculty concerns about switching to an online format?
Dommeyer, et al (2004), says, "Faculty fear that the online method will produce a lower response rate and a less accurate response than the traditional method. Moreover, they fear not only that the online method may attract responses from students who rarely attend class but also that some students will be influenced by their peers during online evaluation."
Do online surveys attract negatively biased student ratings?
A recent study by Kherfi (2011) at the American University of Sharjah examined 18,000 surveys from 4,000 students in 848 classes. Kerfi concludes:
"Respondents are different from nonrespondents in various ways. On the one hand, regression results showed that first-semester students have a higher likelihood of participation, and students respond more if the course is a major requirement. On the other hand, men, students with below-average course load, and students with low CGPA or low course grade are less likely to participate. A matched pairs test that completely controls for class- and instructor-invariant student characteristics confirms the finding that students who do better in a course are more likely to participate in SET. Furthermore, I report evidence indicating that students who were more likely to have strong opinions about a course (i.e., those who responded earlier than others) had on average positive views. Thus, contradicting a common belief, SET does not attract disproportionally more dissatisfied students. The findings suggest that student ratings of faculty might be biased upward." In the study conducted by Dommeyer (2004) and colleagues, they conclude that "online evaluations do not produce significantly different mean evaluation scores than traditional in-class evaluations."
Do online teaching surveys produce low response rates?
The research suggests that faculty behaviors impact response rates. In a study at Brigham Young University, Johnson (2003) found that the way faculty communicate with students about the online survey influences the response rate:
|Type of Faculty Communication||Average Response Rate|
|Assigned students to complete online rating forms but did not give them points||77%|
|Encouraged students to complete the online forms but did not make it a formal assignment||32%|
|Did not mention the online student-rating forms to students||20%|
Further, Johnson (2003) says, "It appears that when completion of online rating forms is assigned or encouraged in more than one course (as was the case in most of the pilot courses), the likelihood of student respondents completing questionnaires for all their courses improves considerably."
Ballantyne (2003) found that student response rates were impacted by faculty telling students how they use the survey results to change their teaching. "Research at Murdoch and at other institutions has shown that responding to students about changes made as a result of their feedback has positive effects. This communication to students not only makes them more likely to complete a feedback questionnaire, it also helps them feel that they are heard and that their concerns are considered."
How can you encourage your students to participate in the online surveys?
- Inform your students about the new online survey procedures. For more information on the online course evaluations, read http://times.duq.edu/2011/11/online-forms-make-course-faculty-evaluations-easier.
- Tell students how you use survey responses to improve your teaching and adjust the course.
- Make the completion of the online ratings a course assignment (e.g. "Tonight, as part of your homework, please complete the online course evaluation on blackboard or your smartphone.").
- Start your next semester by discussing what you learned from the surveys and how you are adjusting your teaching or something about the course as a result.
Disclaimer -- CTE does not formally evaluate teaching or create policy in how faculty evaluation is conducted at Duquesne. This tip is based on literature relevant to online course evaluations and is not intended as a statement for or against online student evaluations of teaching.
Ballantyne, C. (2003). "Online Evaluations of Teaching: An Examination of Current Practice and Considerations for the Future." New Directions for Teaching and Learning 96, 103-112.
Dommeyer, C., Baum, P., Hanna, R., & Chapman, K. (2004). "Gathering faculty teaching evaluations by in-class and online surveys: their effects on response rates and evaluations." Assessment & Evaluation in Higher Education 29 (1), 611-623.
Johnson, T.D. (2003). "Online Student Ratings: Will Students Respond?" New Directions for Teaching and Learning 96, 49-59.
Kherfi, S. (2011). "Whose Opinion is it anyway? Determinants of Participation in Student Evaluation of Teaching." Journal of Economic Education 42 (1), 19-30.