Rethinking Assessment in Online Classes

“Will this be on the test?”

If you have spent any time in a classroom you have likely heard this question. The implication being that if it isn’t important enough to be on the test then it isn’t important enough for the student to remember. As teachers, however, we take issue with that notion. Everything is important, but it cannot always be boiled down to easily graded test questions. In fact, it is often the deeper, and usually more muddy, concepts that are the most important but also the least conducive to testing. The result is that when assessing students we tend to gloss over these concepts in favor of more easily measurable indicators of learning. This is particularly true in online classes, which are mediated by Learning Management Systems (LMS) that often have limited tools for assessment. For example, all major LMS support a variety of quiz and test tools ranging from true/false, to multiple choice, to short answer and essay questions, but there are few alternatives beyond these testing approaches. Online teachers may be better off foregoing these built in tools and rethinking how assessment in an online class can and should look.

There are a number of reasons why online tests are a bad idea. First, unless proctored, we cannot really have any faith in the results. Students have a world of resources available to them and even when the LMS can lock out the browser students still have access to their smartphone, tablet or other computer. Proctored or open book tests are not really the answer either. The problem here isn’t really the potential for cheating. The problem is that tests are simply a poor measure of learning. At best, they measure students’ ability to apply learning to contrived situations and at worst simply test recall. In conversations with students, they frequently admit to cramming for tests and then immediately forgetting the information once the test is over. The result is a mismatch between the high level of learning, critical thinking and problem solving we want to see in our students and the low level way in which we measure it.

One solution is to rethink assessment and develop practical and meaningful alternatives to testing. To demonstrate, below are three different approaches to online assessment (which can be used in traditional classes as well). When presenting such alternatives, the first complaint I usually hear from instructors is that they are time-consuming to develop and grade. True, a high level of student thinking and engagement is going to require a higher level of instructor involvement. However, this does not have to mean more work for overworked teachers. There are alternative approaches to grading, and there is also a shift in the work load that often accompanies a shift in pedagogy. Test-based classes tend to require a lot of upfront work on the part of the instructor such as preparing tests and delivering lectures. When moving from a teacher-centered pedagogy to a student-centered pedagogy much of the work is also shifted from front-end to back-end. In other words, much of the teacher-centered work is shifted to students freeing up time for more meaningful assessments.

These three alternative approaches to online assessment took place in various settings:

• an undergraduate course on technology and ethics at a major university,
• an undergraduate course in energy technology at a community college, and
• a graduate course in teacher education at a major university.

I currently teach a course on technology and ethics that also fulfills a university requirement for a writing intensive class. As such, the course does involve writing a couple papers. The first paper students write is an initial investigation into a technology and its potential ethical implications (e.g. digital piracy, cloning, artificial intelligence, etc.). For this paper, students are given a choice between a traditional opinion paper, a multi-media project incorporating a mix of text, video, audio and graphics, or an analysis of a popular movie (e.g. Wall-E, I Robot, The Island). Students also have choice on their topic and, of course, the resources used in their analysis (although they are required to use at least one class reading as well). While it would be possible to develop a test over the various ethical principles explored, it isn’t their recall of the ethics that is important but their ability to apply ethical reasoning to a chosen technology. Their papers/projects also go through a peer edit process that allows for a variety of feedback. Peer editors are encouraged to ask questions if something doesn’t make sense, offer suggestions to improve clarity or flow, and provide positive comments when something is particularly clear, interesting or well-reasoned. This also provides students an opportunity to see how other students write, formulate arguments and cite evidence. I grade the final draft and provide additional formative feedback that will help them as they think about and write the second paper. In this way, I not only assess their current level of understanding, but also help them develop their critical thinking.

While working as an instructional designer at a community college, I worked with faculty in a clean energy technology program to develop online and hybrid courses as part of a federal grant. Some faculty had already begun to use online assessments in their face-to-face class. These tended to involve multiple-choice tests over concepts covered in class or course readings and as a result really only tested student recall of the concepts. Taking such a test online meant it had to be proctored as answers could easily be looked up. In a particular engineering safety course, test questions involved, for example, identifying the type of fire extinguisher needed for different fires. The alternative assessment we developed was a site survey. Students selected a publicly accessible building (library, school, office, restaurant, etc.) and after obtaining permission performed a survey of the various safety aspects including exits, fire suppression, and evacuation plans and wrote up their analysis and suggestions. Rather than simply focusing on recall of concepts, students had to apply concepts and evaluate the effectiveness of the implementation. This type of assessment is particularly well suited to an online class as students have complete flexibility regarding when and where to do the project and the write up could be submitted as text through email or the LMS dropbox or through more creative means such as a video or via any of the various web tools out there. This approach to assessment is also in line with adult learning theory in that it is meaningful and relevant to the student. They see the value in applying their learning to real world situations that mirror the actual work they may do after graduation. The assessment can also be collaborative. Students can conduct the site assessment in small groups discussing their findings as they go, and then each student submit their own analysis. This project also allowed for an alternative approach to grading. While the instructor did ultimately assign a grade, students initially evaluated each other and once again had to apply their learning to identify misunderstanding or gaps in the analysis.

While working as an adjunct technology integration instructor for a post-baccalaureate K12 initial licensure program, I developed an entirely project-based course. Students explored state standards, wrote objectives and developed lessons that integrated technology in meaningful and transforming ways. Each project was a practical lesson that could be used in their future classrooms and provided an opportunity to assess a mix of student learning including: aligning objectives and standards, aligning assessments and objectives, instructional strategies, design of student-centered learning environments, and meaningful technology integration. Projects were often shared with peers providing these soon-to-be teachers with a variety of practical ideas.

Project-based learning certainly isn’t new, but it also isn’t overly common – especially in online classes which are often reluctantly taught by instructors who would rather teach in a classroom. As a result, these online classes tend to mirror their face-to-face approach, which may not transfer well to an online environment. These classes often need to be redesigned for effective online delivery, and one aspect of such a redesign should be a rethinking of online assessment.