Using Testing in All Aspects of Course Development

Posted October 3, 2018

By Adam Persky

I have become a big fan of the bottom line up front concept (BLUF) so I will start with my recommendations and approaches to testing.

First, having students retrieve information from their memory improves the retention of information and skills. We have plenty of data (and meta-analyses) to show that. Typically, we ask students to retrieve information in the form of a test or quiz (hence it being called the testing effect). But we can functionalize this effect in a variety of ways within a course.

Three sheets of paper labeled "quiz, test, exam" with multiple-choice answer design — Several ways to retrieve stored information - quiz, test, exam

Pre-Course. Before the course even starts I give a multiple choice test. This serves 2 purposes. The first, broadly it assess what students know prior to starting my course. Second, it reactivates marginal knowledge – knowledge that may be stored within memory but has become inaccessible with time due to disuse. The information is in there but doesn’t come out easily.

Pre-Class. Before most classes I typically give an in-class, multiple choice quiz. This gives me some sense of what students have retained from pre-class preparation, and again it reactivates marginal knowledge. Also, I need to hold students accountable for their preparation, so a pre-class quiz is necessary. These pre-class quizzes are effective since students use points to assign importance. No points = they will skip preparing for class and if students are not prepared for class, it is challenging to teach them at higher levels of Bloom’s Taxonomy or get them to more deeply understand the material.

During class. I use several cases that have multiple choice questions – at least 3 or 4 per case. If I use clickers, I ask multiple choice questions regarding information covered during previous class periods. Why? To take advantage of the testing effect.

Post-Class. Most classes have extended learning opportunities – that is, homework. If there is homework, it usually consists of more open-ended questions so students have to generate responses. If they can generate the response during practice, it is easier to answer a multiple choice question on an exam. Harder if I reversed the order. These assignments also may take different perspectives from the class discussion to help students generalize the rules to problem solving.

Examinations. I give one maybe two high stakes exams, but they are cumulative and usually multiple choice. Cumulative is important because knowledge is cumulative, and because testing reinforces memory. If we spent time talking about it during class, I hope students retain the information over the semester. Why multiple choice? They’re just easier to grade, even though it can be challenging to write higher order multiple choice questions.

Well, now you know how the story ends. The rest is how I got to this point and more of the reasoning behind why I do things this way.

Teaching was easy when I knew nothing about learning or memory. As I learned about teaching, I started to learn more about learning and why what we did in the classroom should work. So for the past 10+ years, I’ve been teaching myself cognitive psychology. However, it was not until recently that I began to look at assessment as a learning tool. Yes, we typically think of formative assessment as a learning tool, but less so for summative. Typically we think of assessment to give feedback, however, assessment is a tool that can do so much more.

A quote that reads: "Teaching was easy when I knew nothing about learning or memory"

Here is the thing. To the student, the quiz or test is the most important thing. The assessment indicates to them what you find valuable for them to learn. This is probably why assessment is the second step in instructional design – develop learning objectives, figure how you will measure progress and if students meet those goals (i.e., assessment), and then design activities to help them achieve those goals.

Now, I work in a professional program. Students take prerequisite coursework and we hope they have remembered something. So as an instructor, I need to assess their prior knowledge, as it can make or break learning in my course. I also want students to apply their knowledge. However, in order for them to apply knowledge, they first need baseline knowledge. To do this, they need to prepare on their own and I need to assess where they are and what they know to hold them accountable. I also have to keep in mind that we forget very quickly and I need to go back and remind students of, again, prior knowledge. The good news, once information is learned, and then forgotten, it generally returns much faster than the time it took to originally learn that material. Ultimately, I need to make sure students have met the standards I set out for them, and testing can help me with this.

What is marginal knowledge? Think of a tip-of-the-tongue state. You know a word, what it may start with, about how long it is, but you can’t spit it out. Marginal knowledge is information that is stored well in your memory but you have not used it in a long time, so accessing it becomes problematic. Thus, you get the feeling of knowing the word but can’t quite access it. Beth Marsh and her graduate student Allison Cantor at Duke University demonstrated that we can reactivate marginal knowledge with a multiple-choice question if the answer choice is one of the choices. This is great, as it is much easier and more efficient to write a multiple-choice question than to re-teach information. Therefore, before the course or a class period, I use multiple choice questions to reactivate marginal knowledge and to assess student knowledge. Two birds – one stone.

An images of a brain with arms flexing weights and the traditional four answer options for tests - a, b, c, d, located at each corner of the image.

Now, during class, we know that the testing effect is a pretty powerful learning tool, so how can we get students to retrieve information? Well, I use a variety of methods to do so, including quizzes, cases, questioning techniques and to some degree, clickers. Oddly, most research shows no learning benefit of clickers (good for keeping attention – not so good for learning). I believe this is because we use clickers incorrectly in terms of strengthening memory. Typically, an instructor will teach and throw up a clicker question. While this is very helpful for students to act on information as soon as they learn it, they are probably going to be able to answer the question correctly without too much effort. However, if you ask the same question during the next class period, some forgetting has happened, and the task of retrieval is more difficult, but also much more impactful in strengthening memory (and more helpful in diagnosing learning). Thus, I use clickers mostly at the beginning of class over material from the last class – or material even further back. This practice is not only retrieval, but also spaced retrieval with feedback, and is supported by the three components that help memory: testing, feedback, and spacing.

Then there are the exams. These are multiple choice simply because I have 150+ students and I can now write questions pretty easily at the middle or higher levels of Bloom’s Taxonomy. While this is mostly to assess where students are, I try to provide feedback on performance so students can continue to learn from the test. This feedback is delayed – right after the exam or the next day – because delaying feedback improves memory. Why? Maybe because it also acts as a retrieval attempt. Students have to think about the question and what they were thinking when they answered it.

Bottom line, assessments are important and everything we do in the classroom needs to be efficient and powerful. We are missing opportunities to help student learning if we don’t do more with testing.

Bio

Adam Persky is a clinical professor at the UNC Eshelman School of Pharmacy. Within the pharmacy school, he teaches physiology and pharmacokinetics and has received several of the School’s teaching awards, including Best Overall Instructor. He was named a Distinguished Teaching Scholar by the American Associations of College of Pharmacy. He is the associate editor for the American Journal of Pharmaceutical Education and on the editorial board for College Teaching. He has given over 100 workshops across the country on a variety of topics related to teaching and learning and has numerous publications within the scholarship of teaching and learning. He also equates teaching to baking. Prepare the ingredients (mise en place) and follow the instructions until you get comfortable with the recipe. Then ad lib once you know how things work but make small changes at first!