Automated Course Evaluation (ACE) Frequently Asked Questions for Faculty
Course evaluations for Fall 2022 that start after Thanksgiving will be administered through Watermark Course Evaluations and Surveys (CES). As we progress in the implementation of the new system, this FAQ will be updated. The following provides insight into the new system based upon UNL and UNK's documentation and experience.
2. What actions will be taken to ensure adequate response rates?
3. What should I tell my students about the automated course evaluation system?
4. Will I be able to monitor response rates while the course evaluation is open for students?
5. How will I know the summary reports for my courses are complete?
6. How do I save a copy of my course evaluation results?
7. Who will have access to the results of my course evaluations?
8. Will students have access to the evaluation results for courses?
9. What should I do if I am having problems with my course evaluations or Canvas?
10. What kinds of instructor tools are available in CES?
11. What is the student experience using CES?
1. How do I access my Course Evaluation Management dashboard to view current response rates and/or set custom dates?
A response rate monitor will be available in Canvas while evaluations are running. To access the dashboard, click Instructor Course Evaluations on the left navigation menu of Canvas.
The new CES system does not support custom dating. Evaluations will be distributed according to the schedule below. Syllabi will need to be adjusted accordingly.
Distribution Schedule:
|
Class Duration (Non-Inclusive of Course End Date) |
Evaluation Begin |
Evaluation Close (Days from Evaluation Begin) |
Student Reminder Email (Days from Evaluation Begin) |
Student Reminder Email 2 (Days from Evaluation Begin) |
|
80+ Days (or Regular Session) |
Class End Date minus (–) 21 days |
14 |
7 |
12 |
|
41-79 Days |
Class End Date -14 days |
13 |
4 |
11 |
|
15-40 Days |
Class End Date -7 days |
6 |
4 |
- |
|
<= 14 Days |
Class End Date -2 days |
2 |
- |
- |
2. What actions will be taken to ensure adequate response rates?
In addition to regular e-mail reminders, the CES system offers a deep integration with the Cavas LMS. This will give us the opportunity to create pop-up message reminders to students within Canvas. Best practices indicate that when students login, they will see a pop-up message like the below:
Eventually, the "Do It Later" option will dissapear railroading the student to the evaluation. Options also exist to lock the student out of Canvas and/or final grades until the evaluations have been submitted.
As with other systems, however, student engagement by the instructor is the best driver of response rates. Other campuses that have switched to online evaluations report that their faculty receive the response rates they request of their students. Faculty who monitor response rates and actively encourage students receive higher response rates than those who do not. If faculty show they value course evaluations, educate their students on how evaluations are used, and emphasize that student input is taken seriously, response rates increase significantly. (Gaillard et. al., 2006)
Students have said regarding course evaluations that unless they are in class, they are not thinking about class. As a result, best practices also indicate that setting aside time in class to fill out evaluations greatly increases response rates. Please note that this in-class time will be in conjunction with, not instead of, the time window mentioned in the table above. You may want to prepare your students ahead of time by saying something like: "Course evaluations are coming up [next week]. You will see some notifications and reminders in Canvas or by e-mail wanting you do complete them. We are taking the evaluation [next Thursday during the last 15 minutes of class]. Please bring your laptops or mobile devices that day."
3. What should I tell my students about the automated course evaluation system?
To ensure a good response rate and receive thoughtful feedback from students, emphasize the importance of course evaluations to the success of UNO’s academic programs and to you personally.
Stress the various ways of accessing evaluations (Canvas, email link, and mobile devices).
Remind them that their responses remain confidential and aggregated results will not be seen until after final grades for the term have been submitted.
4. Will I be able to monitor response rates while the course evaluation is open for students?
A response rate monitor will be available in Canvas while evalutions are running. Faculty will not be able to identify the status for any particular student or whether they completed the evaluation.
Be sure to follow-up with students to remind them to complete the course evaluation. A paper presented at the annual meeting of the American Educational Research Association showed that in following-up with non-responding students, 50% of non-responders were not aware the evaluation was available to be taken, and another 16% forgot (Johnson, 2002).
5. How will I know the summary reports for my courses are complete?
One week after final grade submission deadline, email notifications that results dashboards have been updated are sent to faculty. Faculty result dashboards for various summer sessions will be updated at the end of the full summer term. Faculty will be able to access the results dashboard through Canvas.
6. How do I save a copy of my course evaluation results?
Reports can be exported as a PDF file.
7. Who will have access to the results of my course evaluations?
Generally, in addition to the faculty member, each unit chair/director/dean will also have access to individual instructor as well as aggregated results.
8. Will students have access to the evaluation results for courses?
Students will not have access to summary data collected for a specific course or instructor. As the new process is further implemented, UNO administration along with Faculty Senate and Student Government will consider ways to provide students with more robust information concerning courses and evaluations, while respecting rights of individual faculty.
9. What should I do if I am having problems with my course evaluations or Canvas?
Please contact the Systems Intelligence and Data Analytics office at unoevalhelp@unomaha.edu or the UNO Helpdesk at 402.554.4357.
10. What instructor tools are available in CES?
Dashboard: When entering CES, all your data and tools are arranged in a dashboard panel. Here is a sample:
Manage Courses: The Manage Courses tab allows you to view your list of courses, deployment dates, and surveys attached to your course.
- Click on the Manage Courses link in the purple menu bar at the top.
- In the next screen, click on View (the magnifying glass icon) on the right.
- In that screen, you will see each course listed with its deployment start and end dates.
- Additionally, you can see the survey(s) that is/are attached to your course(s) by clicking the icon under Preview.
Response Rate Tracker: The Response Rate Tracker is at the top right. During deployment, you can view, in real time, how many students have taken, and how many have left to take, the course evaluation. The rate showing is cumulative for all your courses--click the link and it will display course-level response rates.
Use this tool to gauge the level of participation you are receiving, then to encourage students to take the evaluation, whether it be during a class, on a Canvas course announcement, or by Canvas messenger. Your involvement in the process can help foster higher response rates.
Reports: One week following the deadline for full-term final grades, you will have access to your report(s) in your CES dashboard. To access your reports, click on the links under Project Results, or the “Project Results” link under the Results drop-down menu, found in the purple menu bar at the top of your dashboard.
11. What is the student experience using CES?
Other than encouraging your students to take the evaluations, you do not have to do anything — the system is automated behind the scenes.
The students receive:
- A dashboard with a list of the course evaluations they have available to complete
- A welcome e-mail on the survey start date with instructions (contains a dashboard link)
- Canvas page notifications (“pop-ups”) reminding them to take the evals at spaced intervals during deployment (contains a dashboard link)
- A Student Course Evaluations link in their Canvas course navigation menu to their dashboard (like the instructor one in your Canvas course page)
- Help buttons at the bottom of the survey itself that disappear near the survey end date
- A “non-responder” e-mail sent 1 day before the survey end date, for students who have not completed the eval(s).
*NOTE: CES does NOT appear in the student view of a Canvas instructor account.
References
Gaillard, F., Mitchell, S, & Kavota, V. (2006). Students, Faculty, And Administrators’ Perception Of Students’ Evaluations Of Faculty In Higher Education Business Schools. Journal of College Teaching & Learning, 3(8): 77-90.
Johnson, T. (2002). Online student ratings: Will students respond? Paper presented at the annual meeting of the American Educational Research Association, New Orleans, 2002. Retrieved from http://www.armstrong.edu/images/institutional_research/onlinesurvey_will_students_respond.pdf, last accessed 10/28/2014.
Suggested Reading
Ballantyne, C.S. (2003). Online evaluations of teaching: An examination of current practice and considerations for the future. In D. L. Sorenson & T. D. Johnson (Eds.), New Directions for Teaching and Learning #96: Online students ratings of instruction (pp. 103-112). San Francisco, CA: Jossey-Bass.
Norris, J., & Conn, C. (2005). Investigating Strategies for Increasing Student Response Rates to Online-Delivered Course Evaluations. Quarterly Review of Distance Education, 6: 13-29.