The course syllabus is a tool that can be used to increase student engagement. Research indicates that engaged students achieve higher-level learning. When redesigning a course, I came to appreciate the significance of the course syllabus as a tool to promote student engagement.
To redesign the course, I followed course design principles. I used data collected from previous sections of the course and reflected on student outcomes and needs. I considered appropriate assignments aligned with course objectives. I mapped out engaging content to meet student learning outcomes. I followed best practices to create the “perfect” course. I placed relevant information in the course syllabus. However, looking at the course syllabus as a reflection of the changes I made, I realized that the first impression my syllabus made was not a good one.
To start, the syllabus for my dynamic, engaging, redesigned course was very old-fashioned (Figure 1). This syllabus could easily be created on a typewriter long before computers and Microsoft Word were ubiquitous tools for syllabus creation. In addition to a lack of visual interest and design, the syllabus contained policies and information copied from syllabi to syllabi over the years. These policies and procedures were not written to reflect current perspectives. My syllabus was as exciting to read as a terms of service agreement. This realization sparked a complete redesign of my syllabus in design and content (Figure 2).
I’d like to share with you what I learned during my discovery process about transforming syllabi to increase student engagement. To begin, a syllabus bloated with content, lacking design, and/or constructed without considering the power of the syllabus to serve as an important student learning and motivational tool is not typically the kind of syllabus that makes a good first impression for a course. Looking at my original syllabus, I realized I was missing an important opportunity to use the syllabus as a powerful tool. I decided to transform my syllabus to promote student engagement and enhance the learning experience.
My syllabus needed to accurately reflect the course redesign, explain course content, demonstrate pedagogical style, motivate students, and promote student learning and engagement. I wanted the syllabus to make an accurate first and lasting impression to encourage student interest and enthusiasm for the course. For more specific information about a syllabus makeover process, refer to Dr. Tona Hangen who had a similar realization about the power of the course syllabus.
The process of creating an engaging syllabus is not limited to adding pictures and columns. Developing a motivational and engaging syllabus requires content, organization, tone, and a learner-centered perspective of the syllabus. My process of syllabus redesign began by exploring basic assumptions about the syllabus. I asked myself Who? What? When? Where? How? and Why? questions about syllabus construction and design. I’m happy to share the resources and information I found to guide your efforts at syllabus transformation.
Who is the audience for the syllabus?
Answering this question helps articulate the basic importance of a syllabus. A well-designed syllabus provides an important organizational tool for students and instructors. The syllabus assists students and instructors to efficiently plan and prepare for the course. The usefulness of a syllabus is not limited to students and instructors.
A syllabus is evidence of course learning and provides departmental offices, other faculty, and supervisors with pertinent information about the course and teaching methods. A course syllabus is frequently the starting document provided for peer and course evaluation.
A syllabus is sometimes used to offer proof of mastery of specific content when transferring the course credits to another institution or meeting the requirements of a pre-requisite course. As we develop our syllabus it is important to recognize the syllabus may be used for a variety of purposes. For additional information, Harvard University’s Derek Bok Center for Teaching and Learning provides a brief overview of the functions of a syllabus.
What content belongs in a syllabus?
Recognizing the importance of a syllabus, we’ll consider the content that belongs in the syllabus. Kevin Gannon writing for The Chronicle of Higher Education offers practical advice regarding syllabus content. The article provides a good review of recommended content and discusses additional issues encountered when developing syllabi. In addition to required syllabus content that might be mandated by our employers, my ideas for presenting the content to promote student engagement are provided below. Consider your syllabi using the list below. What item(s) is/are beneficial to improve in your course syllabi?
Provide an overview, rationale, and schedule for the course to motivate students. Make sure the schedule is easy for students to understand and follow.
Clearly define student and instructor responsibilities allowing all to know what’s expected.
Develop course level objectives and student learning outcomes that assist students with identifying what they will learn and do in the course.
Describe clear assessment/evaluation methods explaining what students must do to be successful in the course. Be explicit and provide exemplary examples, rubrics, and other tools to support the best student work possible. Examples and rubrics do not need to be part of the syllabus, but should be easily available.
Foster a sense of community that helps students belong. Use inclusive language in the syllabus.
Explain course and University policies so students know exactly what is required. Consider brief explanations with hyperlinks to complete policies.
Include difficult to obtain materials. Don’t make students waste time finding and gathering required course materials.
Offer learner support and resources. Consider what resources help students learn and succeed in the course. In an online course, this means making sure there is adequate technology support and instructions.
A syllabus that promotes student engagement is easy to read and constructed in a manner that makes perfect sense to the students. The syllabus is explicit and easily understood. Developing syllabus content takes time and should be something we assess anew and improve each time we teach our course.
When do we create a syllabus?
The course design process begins with selecting materials and course content. In the earliest phases of the course design process, we develop general course goals. With course materials and general goals, we develop a syllabus reflecting sound course design principles. At this point, the syllabus serves as a planning tool and is ready before developing lectures. The syllabus undergoes frequent revision until we achieve a syllabus that is designed to increase student engagement. Backward design may be a good approach to syllabus development, however, we must be certain the syllabus is complete and ready at least a few weeks prior to the start of the course. The Teaching Center of Washington University of St. Louis provides a useful course planning timeline.
The course syllabus is important. Despite debate over syllabus bloat and required content, we recognize that transforming a syllabus to be learner-centered means that the syllabus is not developed at the last minute. Syllabus development is carefully designed and planned to create a syllabus that is clear, easy to read, and explicit.
Where do students access our syllabus?
A paper syllabus may be required and provided to students on the first day of class. In online courses, the syllabus may be emailed to students with instructions on how to access the course. The important response to this question is to make sure that students have convenient access to the syllabus. Even if we’re teaching a face-to-face class, posting the syllabus on our learning management system (Canvas, Blackboard, Moodle) or other hosting site makes the syllabus available on-demand for students. Unlimited access to a syllabus promotes syllabus use and engagement with course materials and content. Although the sample covers of the syllabi reviewed previously (Figures 1 and 2) are Word and PDF documents, some instructors use other syllabi formats including infographics, Prezi presentations, flip books, google templates and a variety of different formats. The best syllabus format is determined by presenting the course content in a manner that promotes student learning and interest.
With creative syllabus design, we must be mindful of accessibility issues, copyright, and other pragmatic issues. If you’re not ready to fully transform the format of your syllabus, look at Billie Hara’s course map that graphically illustrates how assignments and learning outcomes are connected in her course. Adding a graphic element in the syllabus may make course expectations more explicit.
How do we express ideas in a syllabus and set a tone for the course?
Our syllabus makes a first and lasting impression. Students and colleagues judge our teaching, communication, course content, grading rigor, organization skills, and more based on our syllabi. We want to express our course design choices in a manner that clearly expresses ideas and expectations relevant to our course. We have an opportunity to take a warm or cool tone in relaying syllabus information. For example, a cool tone is set by this statement, “Students will attend every class. If a student misses three classes, the student is dropped from the course”. The same information can be expressed in a warmer tone with the statement. “You need to attend every session of our class to be fully engaged in the learning process. If illness or emergency circumstances will cause you to miss class, please notify the instructor by email. Please be aware that as per our University attendance policy, if you miss more than three classes you should drop the class”. To review additional examples and learn more about how the tone of a syllabus relates to student perceptions of a course and the instructor, refer to an experiment conducted by Harnish & Bridges.
explain student responsibility for self-directed learning
provide a course schedule
include assignment guidelines and rubrics (or directs students to the documents)
We need the syllabus. It is usually required. However, we can do better with our syllabi. We can appreciate the value of the syllabus as a tool that teaches, makes expectations explicit, promotes student success, and introduces the course as a dynamic and engaging learning experience.
I compared data between two course sections where the only change made to the experimental course was redesigning the syllabus to promote student engagement. When comparing student rating scores (IDEA instrument) between the courses, I found scores were higher (statistically significant) in the course with the redesigned syllabus compared to the course with a traditional syllabus. Information on the IDEA instrument is available here: https://www.ideaedu.org/Services/Services-to-Improve-Teaching-and-Learning/Student-Ratings-of-Instruction. This data reinforced additional data and student feedback from assessment measures throughout the courses. A redesigned syllabus increased engagement and interest in the course content. The students appreciated the planning and relevance of the syllabus to their unique learning needs. For me, I verified the legitimacy of expending effort, thought, and design into the syllabus construction process. I hope this blog assists and inspires you when transforming your syllabi to increase student engagement and learning.
Kathleen (Kathy) Klein is Interim Director of the Center for Learning Design and Associate Professor of Occupational Therapy at Stockton University in Galloway, New Jersey. Dr. Klein’s role in the Center for Learning Design supports faculty design and delivery of effective courses that promote student learning. She teaches clinical neuroscience, research methods, motor performance, activity analysis, and professional issues. She has varied research interests including the scholarship of teaching and learning and clinical pediatric issues related to health/wellbeing, self-regulation, and executive function. Dr. Klein presents at a variety of conferences and consults for organizations related to best practices in education, professional development, and corporate training. Dr. Klein received her post-professional doctorate degree in occupational therapy at the University of Kansas Medical Center. She completed her BS and post-professional MS in occupational therapy at Thomas Jefferson University in Philadelphia, PA. Dr. Klein can be reached by email at [email protected]
Can you recall the last time that you sat in the audience of a PowerPoint Presentation? What was that experience like? If the presentation was a typical, it probably included:
The presenter reading directly from the screen
Lots of bullet points
Distracting animations and transitions
I have been conducting a Pecha Kucha Workshop for college faculty titled Pecha Kucha Create Presentations with Pop and style for three years. I always open our session by asking faculty to think about the last time they had the pleasure (or pain) of sitting through a PowerPoint presentation. I want them to think about what worked and what could have been improved. Invariably, the conversation brings up many-- if not all-- of the points mentioned above. The misuse of presentation technologies can distract from the content and lead to boredom and disengagement.
Why Pecha Kucha?
The Pecha Kucha format can help to remedy many of these common PowerPoint woes. Pecha Kucha was developed by the managers of a Japanese Architectural firm (Klein Dytham Architecture), that was suffering from many of the same presentation issues highlighted above. The Pecha Kucha method consists of 20 slides, and the speaker speaks for 20 seconds about each slide. If you are doing the math, the total length of your presentation time is 6 minutes and 40 seconds. Creating a presentation less than seven minutes long, with only twenty seconds per slide cuts down on the rambling that is often characteristic of PowerPoint delivery. A successful Pecha Kucha is tightly edited and arranged in bite-sized pieces. It requires careful consideration to arrange the slides in a sensible way. It also incentivizes presenters to rehearse so as not to waste time. The result, typically, is a polished presentation.
Another feature that sets the Pecha Kucha apart from a traditional presentation style is the design of the slides. As the audience only has twenty seconds to absorb each slide, you want to select an impactful image that is worth looking at for twenty seconds. The slide will only be visible to the viewer briefly, so it should not be an image that the audience has to labor over. Pecha Kucha benefits from tight structure while also giving creative latitude to the presenter, turning the act of presenting into storytelling.
As an instructor, you can do a lot with this presentation format to make presentations livelier and more engaging to your students.
Let’s look at some ways you can use Pecha Kucha with your students.
Structure your lessons around two or three Pecha Kuchas that are about seven minutes in length. Each of these mini lectures can cover a major theme for your lesson. Between each mini-lecture, provide students with an activity that reinforces the main concept of your lecture.
2. Introduce a topic to your students
Since the Pecha Kucha is so short, it is an ideal format to introduce a new topic to your students. An introduction to a topic via Pecha Kucha can be used either face to face or online. After using a Pecha Kucha to introduce a concept, you can allow time for questions and answers, or an online discussion board inquiry.
3. Image Rich Review of a Topic:
At the conclusion of your lecture, or online material, use the Pecha Kucha method to provide students with an image-rich review of the concepts that you presented.
4. Final Exam Review
When reviewing for your final exam, create a few Pecha Kuchas with a topical overview of your content. After you present each Pecha Kucha, let your students interview you with any additional questions they might have about preparing for the final exam.
5. Tell a Story
Do you have a relevant story that fits in with the material that you are presenting in your traditional or online classroom? Tell that story in Pecha Kucha Format, using compelling images, in 6 minutes and 40 seconds.
6. Visual Case Scenarios:
Are you already using case studies with your students? If you are not already using case studies, are you interested in using them with your students? Either way the Pecha Kucha format gives you a great way to present a case study with your students and get them practicing skills! After you present your case scenario to your students, consider letting them work through the case in groups.
7. Project Overview:
If you are assigning a project to your students, “pitch” the project to them by using the Pecha Kucha format to outline the project’s requirements. After you present the project to the students using the Pecha Kucha format, students can interview you for clarification. If you are using the Pecha Kucha format to pitch the project online, a bonus is that the students will have a succinct presentation that they can review as needed to gather project requirements.
8. Online Presentations:
When creating presentations for students in an online format, brevity is key, otherwise students will not watch the presentation. The Pecha Kucha method allows you to employ chunking when you post a video, and to deliver the material in bite-sized pieces.
9. Big Picture Concepts:
Most instructors have big picture concepts that are important to their field. You can use Pecha Kucha to deliver these big picture topics to students. Highlight the topic, why it is important to your field, and how it contributes to the history of your field. This gives students a foundation in the thinking that characterizes your discipline.
10. Introduce yourself to your students
Whether you are teaching an online, blended, or face to face course, as a first step, you introduce yourself to your students. Use the Pecha Kucha format to introduce yourself to your students, talk about why you chose your field, your teaching philosophy and other items you feel comfortable to share in a unique, picture format.
11. Vocabulary Pecha Kucha
If students need to recall vocabulary or other concepts, give each student a vocabulary word from your list, and have each student create one twenty second Pecha Kucha slide on the definition of their word. Each student takes turns delivering their word to the class.
Help students make sense of the Pecha Kucha
Since the Pecha Kucha format is a brisk paced format, I recommend faculty develop some exercises to help students “unpack” the Pecha Kucha that was just presented.
One Minute Paper
One tactic you can use is the One Minute Paper. After you deliver the Pecha Kucha, you assign a one-minute paper, where you ask the students about the main point of your lecture. In a face-to-face class you can then call on a few students to read their papers. That will check comprehension and spark discussion.
Muddiest Point paper
If you know that your Pecha Kucha covered a dense topic consider having students complete a muddiest point paper, where you ask them what they found most unclear about the Pecha Kucha.
How students can use Pecha Kucha
The Pecha Kucha format is not just for faculty. Pecha Kucha provides students with a good template for presentation development which moves the focus from reading bullet points to effective presentation, followed by discussion.
1. Group Projects
Pecha Kucha gives students a structure to complete group projects. Since the format is only seven minutes long, this gives you lots of opportunities to assign students a specific portion of a topic. This saves you from needing to listen to several versions of the same presentation. Your students will also be more engaged when they listen to their peers as they will be presenting on a different topic.
2. Personal Reflection Presentation
Student ePortfolios are popular in Higher Education right now because they encourage student self-reflection and introspection on the relationship of their current learning to the world around them and their careers. Pecha Kucha gives students a great structure to reflect on your course content and how it has affected their lives.
3. Student Introductions
At the beginning of a semester, you have students introduce themselves to each other because it builds community. Get your students to introduce themselves using Pecha Kucha and give them a structure to organize their thoughts and tell a compelling story.
4. Teach a topic
The best way to learn something is to cultivate the ability to teach it to someone else. Give your students a chance to instruct their peers using the Pecha Kucha format. Select a lesson and assign different groups to teach each theme, Pecha Kucha style. After the lesson each group can entertain questions.
General Tips for Pecha Kucha Development
Since the Pecha Kucha is about storytelling, when designing a Pecha Kucha, you should start with thinking about the story you are going to tell in seven minutes or less.
Start by developing a storyboard, rather than developing directly in PowerPoint. This will help you focus on the story you are trying to tell instead of designing a PowerPoint and trying to make your presentation “fit” the PowerPoint you develop. Low Tech storyboarding tools include:
Printing the Notes Pages of a PowerPoint Slide
Time each slide to make sure that it is 20 seconds. Remember to add time for pauses and transitioning to the next slide
Next, intentionally plan the images that will go on your slide. Plan thoughtful images that highlight the theme you are trying to convey. Try to think of the images that will advance your message before you even start looking for images.
Tips for Designing the actual Pecha Kucha in PowerPoint
Avoid most PowerPoint templates as they are cluttered and difficult to digest when working with twenty seconds per slide.
Text should be kept to a minimum. If you need text, use brief keywords or phrases, as you only have 20 seconds per slide. When using text, use a Sans-Serif font, such as Helvetica or Arial.
Avoid using Sound clips or videos.
Set the slides to automatically advance after 20 seconds.
Don’t use slide transitions. The slides advance so rapidly that the viewer will find transitions distracting. You don't need sounds or video clips. Your voice will carry the presentation.
Rehearse your Delivery of the Pecha Kucha
Practice what you are saying. This rehearsal is critical to being able to be polished while delivering the Pecha Kucha. Listen to your delivery. Do you have too much to say? Remember to breathe as you present. If the timing is tight, edit your presentation. You don't want to sound rushed while delivering the Pecha Kucha.
Baker, T. J. (2014). Pecha Kucha & English Language Teaching: Changing The Classroom. Amazon Digital Services LLC.
Shank, Patti (Ed.). (2011). The Online Learning Idea Book, Volume 2: Proven Ways to Enhance Technology-Based and Blended Learning. San Francisco: Pfeiffer.
Michele Knight is an Instructional Designer for the Department of ELearning Innovation and Teaching Excellence, (ELITE) at Montgomery College. Her primary role is to help faculty integrate technology into their teaching. Prior to working for Montgomery College, Michele was employed as a Multimedia Specialist for Chesapeake College. Michele’s professional interests include teaching with mobile technologies, techniques to build public speaking skills, and the effective integration of technology into instruction. She holds a Bachelor of Science degree in Computer Information Systems and a Master of Arts degree in Instructional Systems Design, from the University of Maryland Baltimore County. To get in touch with Michele, please visit www.linkedin.com/in/michele-knight
“Be the guide on the side, not the sage on the stage. I am the guide who creates the learning experience and then steps back to let learners take over”. – Sharon Bowman
Do your students ever nod off, watch the clock, or retreat to their phones during lecture? In this fast-paced digital age it can be challenging to keep student attention. Harder still is engaging interest and helping students make their thinking visible. A little innovation can go a long way toward increasing student engagement in three areas: doing (behavioral), thinking (cognitive), and feeling (emotional, interest). If you are ready to plunge in and try active engagement in your lessons, check out the spicy strategies. If these strategies are a bit too much for your instructional palette, check out the milder alternatives.
Dealing with Student and Faculty Resistance to Active Learning
Students and instructors alike sometimes turn up their noses at these types of learning activities. Admittedly, they are artificial and so can sometimes seem “hokey” or even frivolous. In her book called Learner-Centered Teaching, Maryellen Weimer details dealing with the “nuts and bolts” of implementation and addresses both faculty and student resistance. Weimer recognizes that these types of activities involve additional planning and ask more of instructors than do traditional lectures.
Her advice, when students scoff at active approaches, is “don’t take it personally.” My approach is to be upfront with students at the start of the semester. During the first day of class, I inform students, model, and demonstrate my teaching philosophy which includes active learning and an inclusive environment. Be open with students about the purpose of active strategies and tell students ahead of time when and how they will be implemented.
By contrast, Weimer’s advice for dealing with faculty resistance is “do not seek to convert the masses.” Your colleagues exist in their own, unique web of tenure, research, personal style, and so forth. These activities are not for every teaching style and so it is best that you limit your discussion of them to like-minded colleagues. Perhaps most importantly, Weimer emphasizes the importance of documenting the impact of your approaches.
So what will it be, spicy or mild?
My area of scholarship is educational psychology. I have been in the field of education for more than 20 years and hold an advanced professional certification in teaching. My experience has involved working with toddlers, managing literacy programming, inclusion classroom co-teaching, mentoring teachers, working with adolescents’ social and emotional skills and teaching in middle schools and college.
Ambrose, S. A., Bridges, M. W., Lovett, M. C., DiPietro, M., & Norman, M. K. (2010). How Learning Works. San Francisco: Jossey-Bass.
Bowman, S. (2014). Presenting with Pizzazz. Glenbrook: Bowperson Publishing.
I’ve been teaching criminal justice courses both in residence and online for six years. With so many stories about police events in the news, my students and I have plenty of real-world examples to discuss. Unfortunately, from time to time, these conversations can seem superficial.
This is especially the case when we cover the ethical dilemma involving police accepting gratuities – such as free meals or a free cup of coffee. Some students agree that police should be able to accept gratuities because the public is showing appreciation for their service and it would be impolite to decline such a generous offer. Other students argue that police should not accept gratuities because these gifts can be a slippery slope to corruption.
To deepen the discussion, we must move beyond these cursory arguments. For example, how might other officers react when one turns down a free coffee that all other officers have accepted? Might that lead other officers to distrust the refuser? Might that have implications in other areas of the workplace? Maybe so, maybe not; but, this is a conversation worth having. And too often in my class we don’t have it.
I turned to my instructional designer, Louisa, for advice. We discussed ways to increase student engagement. Ultimately, we thought a decision tree activity might lead to deeper conversations on this topic.
Display Logic: Useful Feature of Survey Software
I had several conversations with students in my office about what might happen if a police officer accepted a free cup of coffee. Across many such discussions, I developed a decision tree with various paths based on the decisions made. I mapped it out in Word; in retrospect, Excel might have been an easier tool. Excel has the ability to hide long text in one small cell while also displaying it in full in the formula box above when clicking on the individual cell.
Penn State, where I teach, offers employees licenses to use Qualtrics. Other survey software has similar attributes. However, having used Qualtrics on past research, I was familiar with some of its features. One, in particular, is “display logic" (see Figure 1). Display logic only shows an item on some condition set by the programmer.
This feature is especially useful for developing decision trees, where we want the student to go to a scenario based on a previous response. For example, the opening scenario asks whether the student, as a rookie officer, would accept a free cup of coffee when offered:
You are a rookie officer on the afternoon shift. You stop for coffee at the Sheetz on your beat, where you meet up with Joe, another officer who’s been on the job for about 12 years. You and Joe walk up to the register to pay for your coffee and the cashier refuses to accept your money: “Your money is no good here! We appreciate your service. Coffee’s on us”. Joe thanks the cashier and tells you, “It’s nice to be appreciated once in a while. That’s why all the guys stop here for coffee”. What do you do?
If a student does not accept a free cup of coffee, s/he is redirected to a similar scenario with more peer pressure to accept; if s/he continues to decline the free cup of coffee, other officers progressively withdraw trust, leading to dangerous situations.
If a student accepts the free cup of coffee, the next scenario that will display asks how s/he would respond to two incidents:
You’ve been regularly going to Sheetz for the free coffee for some time now. You know all of the Sheetz employees and they know you. Your Sheetz visit is the one thing you look forward to on your shift – it’s a nice break from dealing with the worst of society. About an hour after your Sheetz visit on tonight’s shift, two calls come over the radio: A robbery is in progress at the Sheetz AND there’s a man with a gun threatening the cashier at the Turkey Hill a block away from the Sheetz. You’re exactly between the two, but you can only respond to one call. Which call do you take first?
If the student responds to Sheetz, the next scenario increases the gratuities officers are offered to both lunch and coffee. However, the Turkey Hill was robbed and the cashier was injured, prompting an Internal Affairs investigation because of the slow response time. If the student responds to Turkey Hill, the Sheetz employees rescind the free coffee offer for all officers and now other officers are upset with the student rookie.
Once the decision tree was entered into Qualtrics, I asked police officer friends to review the scenarios for a reality check. While some scenarios are a bit over the top with the intention to drive the point home to students, they agreed the scenarios are possible.
As one can see, the progressive nature of the decision trees leads students to deeper understanding of the ethical decision-making process. The tool accomplishes many objectives: it allows students to consider angles they might naturally overlook, it emphasizes ethical decision making as a process rather than as blindly following rules, and it provides insight into real world policing issues.
For class participation, students are asked to complete the decision tree multiple times, making different decisions each time. Then, students post their responses to a prompt on the online discussion board. (For the classes held in-residence, students are required to post to weekly discussion prompts online.)
Adding the decision tree activity was a nice break from the typical discussion prompt and students seemed more engaged with the topic. Students had deeper conversations about police gratuities and, broadly, about corruption. We were able to link some of their thoughts to other course topics, such as police subculture, when they asked, “That wouldn’t really happen, would it?!”
Overall, this was a welcome addition to the class discussions and a fruitful way to use survey software.
Dr. Jennifer Gibbs is an Assistant Professor of Criminal Justice in the School of Public Affairs at Penn State Harrisburg, where she studies policing topics, including public attitudes toward police, violence against police and police recruitment and hiring. Her work on social distance and attitudes toward police, co-authored with Dr. Jonathan Lee, was recognized in the 2016 Emerald Literati Network Awards for Excellence.
Louisa Nkrumah is a Learning Experience Designer at the University of Maryland, College Park in their Teaching and Learning Transformation Center (TLTC). In her role, she provides instructional consulting, support, and training, learning technologies support and implementation, and instructional design support. Prior to joining the TLTC in June 2018, Louisa was an instructional designer at Penn State Harrisburg, where she collaborated with faculty to design, develop, and revise courses for the online Criminal Justice and Homeland Security programs via Penn State World Campus. Louisa holds a B.S. in Applied Behavioral Sciences and an M.Ed. in Training and Development, both from Penn State Harrisburg.
Steve has to write a research paper for his psychology course. Given wide latitude by his instructor, he elected to take up a project assessing the relative merits of cognitive therapy as an alternative to medication for the treatment of depression. The trouble is, Steve already strongly believes that cognitive therapy is a superior alternative. That prior judgment is likely to inform his research and the paper he’ll eventually write.
Steve begins by searching for articles on this matter. He uses key phrases such as “advantages of talk therapy”, “why cognitive therapy is better than medication”, and “serious risks of SSRIs.” He does not bother to complete searches with phrases such “when is medication a better alternative to cognitive therapy”, or “comparison of outcomes for different therapeutic approaches to depression.” That is, he is searching in a manner that will tend to lead him to articles that support his current position. He will find confirmation for the view he already holds. He is also avoiding searches that would tend to lead him to material that could disconfirm his position.
Nevertheless, Steve finds and reads some articles whose conclusions run counter to his view. He finds them to be lacking, though, citing weak methodology, lack of author expertise, and so on. In short, he finds sources that support his view to be more authoritative than those that don’t.
These are typical manifestations of confirmation bias. As a teacher of psychology, you may already explore it as a concept with your students, but it’s clear that many students are also affected by it in practice. I’m not a psychology professor, but I teach critical thinking in an interdisciplinary general education program that also emphasizes research and writing. Over the years, I’ve seen many research projects culminating in argumentative essays gone awry, thanks to confirmation bias. This post explores how it can arise in coursework and discusses what we might do to mitigate it.
Definitions of confirmation bias are many and varied, but I’ll proceed with this understanding.
“Confirmation bias refers to a type of selective thinking whereby one tends to notice and to look for what confirms one's beliefs, and to ignore, not look for, or undervalue the relevance of what contradicts one's beliefs.” (Carroll, 2016)
We can see both aspects of this present in Steve’s work. He went about looking for relevant evidence in a way that heavily favored confirming data over disconfirming data, and he heavily discounted the latter when he found it.
Many teachers who assign argumentative essays require students to acknowledge and respond to counter arguments. Do such requirements succeed in getting students to treat multiple perspectives fairly and comprehensively? They often fail to help because students tend to misrepresent counterarguments or address them dismissively. Not only do they fail, but they may make matters even worse!
A well known 1979 study conducted at Stanford University supports this outlook (Lord, Ross & Lepper, 1979). Of the undergraduate participants in the study, half were supporters of capital punishment, and half opposed the practice. Each subject was presented with the results of two studies. One study presented evidence confirming the efficacy of capital punishment as a deterrent, and the other presented disconfirming evidence. Students tended to evaluate the methodology and reasoning of the study that happened to support their side more strongly. The tendencies to overrate the strength of favorable sources and underrate the strength of unfavorable ones had the effect of moving people on both sides to more extreme positions on the issue. The effect of exposing students to a balanced set of opposing views was not moderating. It was polarizing.
In my experience, students are more likely to handle opposing sources and opposing views poorly when they are directly put to the task of taking a side and arguing for it. For the typical student in that situation, everything is about finding and organizing evidence that advances their single goal - winning an argument. They have little reason to avoid the worst tendencies of confirmation bias.
Part of the problem is that students dismiss opposing arguments because they are induced to present arguments with confidence. So, we ought to try to present different stakes for them. This could mean shifting research and writing goals away from argument, or it could mean bringing in intermediate steps. A different kind of assignment can focus on research as an end in itself, where students must think about the merits of sources, that is, their authority and their content strengths. This can culminate in products such as critical annotated bibliographies, where students can be evaluated on (and perhaps more importantly, rewarded for) fair and thorough evaluations of sources.
How could this help to lessen the impact of confirmation bias on students? When they are made to take opposing sources and arguments seriously and consider them more carefully (which they are more likely to do when they know they’ll be graded on precisely that), those same sources and arguments will not be so easily dismissed or downplayed further down the road. Once a student has acknowledged the merits of a study that challenges their prior judgments, it cannot then be dealt with as trivial in an argumentative research paper.
Another strategy that holds promise involves disabusing students of a certain notion. Many students approach arguments as contests that they don’t just have to win; they often feel a need to dominate! For the typical student, is it enough that the balance of evidence tips in their favor, or is it important to that every single issue that bears on their argument go their way? I’ve had students admit to having thought dichotomously about the positions they take in debates, thinking along the lines of “Either I’m right about everything on this issue, or I’m on the wrong side of it, altogether.” Students can get better at avoiding such false dichotomies.
Conversations with students like that one have led me to suspect that another kind of fallacy is at play, one that only fuels confirmation bias. I have dubbed it (tentatively) “The fallacy of the overwhelming case.” It is a flawed approach to reasoning wherein one assumes that any point that would, if true, support a conclusion, should be taken as true. Working under such an assumption, students may feel obliged to argue for points where the evidence is not on their side. This increases the psychological pressure to be dismissive of compelling evidence to the contrary.
Think about those students who argue for or against capital punishment. Four crucial issues that bear on the morality of capital punishment come to mind.
1. Does the threat of capital punishment deter heinous crimes?
2. Are there misdeeds so terrible that they warrant death?
3. Does the possibility of mistaken convictions make an irrevocable punishment unjust?
4. Can people who have committed evil find redemption?
We may disagree about their relative weight, but a student who covers the standard bases in the capital punishment argument would probably wish to address all those issues, and probably more. To these questions, the pro-capital punishment researcher would answer the first two questions above in the affirmative and the last two in the negative, and marshal evidence and reasons to support those answers. An opponent of capital punishment would more likely defend the opposite answer to each question.
Were I to ever approve a student’s proposal for this topic (and that seems unlikely!) for a major argumentative paper, I would urge that person to consider that the best answer to at least one of those questions might not be the one they like. However, a supporter of capital punishment could, for example, concede Question 1 or 3 to the opposition. Likewise, an opponent could concede Question 1 (agreeing for the sake of argument that the threat of capital punishment can be a deterrent) but defend an abolitionist view on other grounds.
You might consider requiring this approach of students. As they develop arguments to support a position, they could also have to prepare to concede at least one point to an opposing side. In so doing, they may gain a better appreciation for moderation and qualification in their arguments.
I wish I could guarantee that these methods would practically eliminate the effects of confirmation bias. In truth, the outlook among many cognitive psychologists on this matter is grim. There are success stories to be told, though. And if we can’t make confirmation bias go away, we can try to deflect it a little. By incorporating new steps into your research projects, you may still see confirmation bias reinforcing many of their prior views, but you may steer them toward more reasonable versions of those views.
Andrew Marx is a faculty member of University College at Virginia Commonwealth University. There, he teaches interdisciplinary courses such as Focused Inquiry, Inquiry and the Craft of Argument, and Pseudoscience. He earned his Ph.D. in philosophy from City of New York, but continues to draw upon his undergraduate major in psychology from University of Delaware to infuse his courses on critical thinking and ethics with insights from that discipline.
Lord, C. G., Ross, L., & Lepper, M. R. (1979). Biased assimilation and attitude polarization: The effects of prior theories on subsequently considered evidence. Journal of Personality and Social Psychology, 37(11), 2098-2109.
As a professor at a teaching-focused institution, I am frequently encouraged to offer courses in service areas of the curriculum, such as our first-year program. My general approach when creating these courses is to start with a useful (and hopefully engaging) topic that can be presented through the lens of popular culture. I previously taught courses on the science of a zombie apocalypse (since converted to a summer science camp) and pranking as a form of communication (co-taught with Dr. Paul Muhlhauser–read about it here)(1). For the first-year program, I wanted to focus on metacognitive strategies that students would be able to use to become better learners. That was a topic that I, a cognitive psychologist, felt comfortable covering with a group of incoming students. I also needed to cover some of the basics on how to succeed as a college student and how to acclimate to what many find to be a very novel situation (i.e., being away at college). The goal was to come up with an overarching theme that would allow for the integration of these various topics. At the very least, I needed a numbered list of items (for the demonstration of simple mnemonics) and a story that focused on change. Enter, Doctor Who!
The first episode of the show, An Unearthly Child (1963), quickly establishes its premise and ends with the Doctor and his companions leaving Earth to start their first adventure. I showed this episode for the first meeting of the class during the orientation weekend. I used it to establish the program, but more importantly to establish what the course was going to be–a new adventure in which we would use pop culture to enhance academics(2). I also used a cheesy line about how they would all get to regenerate into whatever they wanted to be over the next four years. I wanted to make sure they knew what type of humor they were in for. It could only get better from there.
Who is the Doctor? Why Doctor Who?
Perhaps it would be helpful to clarify some aspects of the show. The Doctor is a time-travelling alien who changes, or regenerates, into new forms and must frequently adjust to new worlds, new friends, and new bodies. On the surface, the show seemed to match the basic characteristics I wanted–the show focuses largely on change, and the various incarnations of the Doctor could serve as an ordered list. As a fan of the show, I decided to make the ultimate sacrifice (3) and dig even deeper into the rich history of the show (55 years and counting) to see what additional connections I could mine for lecture and discussion topics, assignments, and other class activities.
One such connection involved the various incarnations (4) of the Doctor and perspective taking, which might be used to enhance learning. I hoped that trying to take the perspective of various versions of the same character (in addition to a number of other strategies used in the course) would increase the likelihood of transferring the skills learned in this course to other situations (5). Each of the incarnations of the Doctor displays a different personality. I wanted to cover each of the incarnations during the semester so that students were exposed to all of the different versions of the character. At the time of the course, there were 13 canonical incarnations (6). The purpose of covering each was to expose the students to the differences in the personality of the character over time. A number of assignments asked them to take the perspectives of characters from the show, and I wanted to make sure they had viewed at least one story for each Doctor. Additionally, they were required to read the section about each incarnation from the book, The Doctors Are In: The Essential and Unofficial Guide to Doctor Who's Greatest Time Lord. The authors of that text do an excellent job explaining the characteristics of each version of the Doctor and highlighting a few key episodes.
What Learning Objectives Were Considered?
In addition to the general goals I had for the course, it also needed to meet a number of student learning outcomes, some of which were required by McDaniel College. Many of these learning outcomes are similar to what one would find at other institutions and programs for first-year students.
How Did I Address the Learning Objectives?
The course needed to reach a number of goals, and the students and I would need to cover a lot of information over the course of the semester. To facilitate this process, I included a number of assignments that varied in scope and nature. The table below lists some brief information about each one.
For the sake of brevity, I will not go into detail on the logic behind each assignment (7). I do, however, think it might be worthwhile to address the final blog project in more detail.
I wanted the course to be a little different from other first-year courses, especially the main writing assignment. The individuals who oversee the first-year program at McDaniel College strongly encourage staged writing assignments that include a lot of feedback to the students. It was also the case that the students would arrive with a variety of backgrounds (academic and otherwise). I wanted to present a staged writing assignment that would address the learning outcomes mentioned above and allow for the variability in their abilities and interests. I also wanted something that was not tied to a specific system of formatting that my students may not have previously used (8). I had heard of other instructors using blogs as part of writing assignments, and I thought that the variability in the media (word, images, sounds, etc.) students could use as part of the assignment would allow them to approach the assignment in ways that played to their strengths and experiences. This would also force the students to work on their writing and communication skills but also allow for a simplified formatting style that we could create together. I found examples of blog assignments online and spoke with a colleague from the English department who had recently taught a blogging course.
Additionally, this assignment was intended to address a number of learning outcomes (see Table 1) for the course. Specifically, the students would (hopefully) improve as writers and communicators, as evaluators and users of scientific research (improved information literacy), and in their ability to apply metacognitive strategies in various situations.
For the assignment, students were tasked with communicating scientific information to their reader in an approachable way. They were to do this by writing a blog post about a metacognitive strategy of their choice. Students needed to describe the strategy in detail and justify its use with empirical research. Finally, the students needed to show how to integrate the strategy through the use of pop culture. They were permitted to use any pop culture topic, not only Doctor Who (9).
The students responded well to the course assignments and seemed to enjoy them overall. A few questions remain, however: what connection were the students able to make and how effective were the approaches used in the course?
The students were able to make a variety of connection between works from pop culture and academic topics. They conveyed these connections through various assignments, exam responses, and in-class discussions. Many of the academic topics came from our class, but some came from other classes the students were taking. As you might predict based on the courses first-year students are likely to take, many of those other topics came from introductory courses in psychology, sociology, history, and language. Here are some of the most interesting examples.
A number of students discussed examples of characters from the show suffering from confirmation bias.
A student used her blog to discuss a long-term approach to learning the canon of Doctor Who. She integrated metacognitive strategies such as prereading and priming in the context of schema building.
In a similar manner, a student used the complex canon and collection of characters from the show Supernatural. She suggested the use of semantic networks and concept mapping as the approaches to learning the information.
A student used the show The Walking Dead to explain illusion of knowledge and the Dunning-Kruger effect by discussing the various strategies used by characters in the show with leadership roles.
A student described detailed connections she noticed between Remembrance of the Daleks and a lecture about rising racial tensions given by visiting speaker and NAACP president Cornell Brooks. Remembrance used a story about the Daleks to comment on racial tensions in the 1960s, so connections to Brooks’ talk were well founded.
One of the groups used the episodes The Empty Child and The Doctor Dances as a starting point to teach the class about the Blitz as part of their group presentation. Both episodes take place during that part of World War II.
In her blog, a student used Doctor Who to explain structural formulas (chemistry) and connected these concepts to the metacognitive strategy of desirable difficulties. She also discussed false memory as a potential side effect.
I was very impressed by the variety and scope of the connections students displayed over the course of the semester. My hope is that students will continue to make these types of connections and transfer their learning in the future.
Were the Approaches Effective? (and What I Hope to Evaluate Next Time)
As this was my first time teaching the course, my main concern was to create a course that functioned smoothly. Given the time-consuming nature of these types of courses, I wanted to make sure all of the material was in place for the students to have a productive semester. With that said, any course like this should be evaluated for effectiveness. The future iterations will involve more formal measurements. For now, here is what I conclude based on admittedly weak evidence.
1. Some learning occurred.
Students performed reasonably well on the assignments and exams. Not all of the students, but most of them showed retention and application. Many of the students grasped the material at the same level or better than students in my upper-level cognitive psychology course (which is a challenging course).
2. Some information was transferred to other courses.
As the instructor of the course, I also served as the academic advisor for each of the students. This gave me an opportunity to talk with the students about their progression in other classes. They frequently mentioned use of the metacognitive strategies (especially, testing, interleaving, and spaced practice) when prepping for other classes. They also frequently admitted to cramming in some cases, despite knowing it was not ideal. At the very least, they seemed to be able to detect what they were doing inefficiently.
3. The students engaged in the college community.
The students overall gave positive reports about the two assignments that required them to attend campus events. They did not universally enjoy the events they attended, but they reported benefits from the assignment. A number of the students collaborated to start a Doctor Who club. They registered as an official campus organization so they could continue meeting and discussing the show.
Overall, the course appears to have been successful in its first iteration. For the next version (regeneration?) I hope to put into place more formal measures of learning, including some empirically validated assessments. I enjoyed teaching this course, and I hope to make it as effective as possible in the future.
Materials Available on the Open Science Framework (OSF)
Various course materials from this class can be found on the OSF. Please use them as you see fit. If you have questions or suggestions to make them better, please let me know!
The TL;DR Version
Jack Arnal is a cognitive psychologist at McDaniel College. He received his Ph.D. in experimental psychology from the University of Arkansas. His research interests include prospective memory, false memory, and eyewitness testimony. Find him on Twitter @DrArnal or visit his website for more information.
1. I sometimes wonder how far I can push these topics before they won’t be approved. It is an empirical question, I guess.
2. This matches with the original remit for the BBC and the show: to educate and inform. Many of the older stories focused on historical events as a way of educating the audience.
3. The academic life can be difficult sometimes, but we persist.
4. Let’s skip the dissociative identity disorder jokes.
5. The likelihood of transfer relies on a number of factors, including appropriate retrieval and application at the correct time. Additional instructions and discussion in class was also part of the strategy. For a good review, see Barnett, S. M., & Ceci, S. J. (2002).
6. Of course, 13 is an oft-debated number. You see, the 9th incarnation did not call himself Doctor, and the 11th incarnation (referred to as the 10th Doctor) actually sent his regeneration energy into…I can see why people throw up their hands. Maybe I can turn this discussion into an activity about operational definitions and scientific communication?
8. My experience has been that students come to college with limited experience with APA-style. I have not used MLA or any other type of formatting since the turn of the century. I had little interest in trying to learn a new formatting style for the purposes of one assignment.
9. Sherlock, Adventure Time, Supernatural, and sports were a few of the pop culture items used.
In the busyness of undergraduate studies, students sometimes lose sight of the big picture. Understandably, many students do not take the time to step back and make the connection between what they are doing in their studies and how this relates to who they want to be, and where they want to go. This might be a tough sell, but I’m going to make the argument that a grainy, lengthy, twenty-year old video of a lecture by a cantankerous mathematician is essential viewing for students. Not only that, but I believe this video can provide a springboard for instructors to facilitate meaningful class discussion regarding careers, motivation, and work habits. Enter Richard Hamming and his lecture, “You and Your Research” (https://youtu.be/a1zDuOPkMSw). This lecture provides instructors with a tool to help students focus on long-term goals while potentially providing a greater sense of purpose in the short term.
Richard Hamming was a mathematician who was widely recognized for his contributions to computer science. In 1995, Hamming recorded a lecture focused on how to conduct research. This was not new material for Hamming, as a transcript of an earlier version of this presentation was published in 1986 (Hamming & Kaiser, 1986). An instructor could assign the transcript for students to read and not use class time for the video, but I believe this would be a mistake. The later version of this lecture provides greater insight from someone who reached the top of their field. While research is in the title of the talk, the content is really about how to excel in nearly any career. It is a rare opportunity for students to hear what it takes to excel at the highest level from someone who has done it, and is now looking back on their career.
Hamming discusses how he would set time aside every Friday for what he called, “Great Thoughts”. During this time, he focused on the important problems in the field, and more generally, he would consider the issues that really matter. The first time I showed the video to one of my classes, I asked them to set aside time for great thoughts. I wanted them to consider why they are interested in the field, and what contributions they wanted to make. The students were enthusiastic, and seemed to think this was a fine idea. At the end of the semester, I asked how many students regularly engaged in great thoughts. The answer was one. One single student. The rest of the students said they wanted to, but couldn’t find the time, and also reported that they didn’t know what to focus on. Although frustrated, I wasn’t ready to give up yet.
Hamming dedicated 10% of his time to challenging himself with questions about the field. I decided that I would dedicate 10% of my 400-level senior seminar class to challenge students with thought-provoking questions, which I based around Hamming’s video. Students in this course watch the Hamming video early in the term, and are then given one “Great Thought” question a week with ten minutes of class time dedicated to discussion. The discussions are conducted in groups of three to four students, and following the small group discussion, we briefly discuss the question as a class. The feedback I’ve received has been overwhelmingly positive, and the discussions have been eye-opening both for me and for the students themselves.
Here's a sample of Hamming’s advice that I have found most inspiring for students, and examples of the questions that have been drawn from his talk.
Leading a significant life
Hamming said, “. . . you need to live the life you want to live, and you do this by doing something significant.” I ask students what is your definition of significant? The discussion does not have to focus on what is significant only in the field of psychology, but generally, what does it mean to the student to make a significant contribution. The most common response to this question, at least initially, is a blank look followed by a sombre, “I…don’t…know.”. This is where the group discussion is invaluable.
Luck favours the prepared
In “You and Your Research”, Hamming discusses the somewhat tired cliché of how luck favours the prepared. I’ve always found the notion that luck favours the prepared, while accurate, to be too vague to be of any use for most students. To challenge them to really consider what this means, I pose the following question, “It’s your lucky day, you’re at a conference and a professor you desperately want to work with at graduate school is there and you have an opportunity to talk to them. What type of preparation would be needed to ensure this bout of luck turns into something meaningful?”. The discussion can tie back into Hamming’s point that the way you lead your life from day to day is the way you prepare yourself for success or failure.
You need a vision of what you want to do and who you are going to be
Hamming uses the clumsy metaphor of a pretty girl and a drunken sailor, but his point is solid. Without a clear vision of where you want to go, it can be difficult to focus on what is important, or to even know what is important. The question I have students discuss is, “Assuming that everything you want to do works out according to plan, where are you in ten years? Be specific, where are you working, what research are you doing, and what contributions have you made?”. It’s amazing how many students haven’t really thought through exactly what they want to achieve. For example, some want to go to graduate school, but don’t have a clear idea of what they hope to accomplish by doing this.
Study success closely and determine which elements of successful people you can adapt to your situation
Hamming tells a great story about his admiration of John Tukey. Hamming could not understand how someone Tukey’s age could know so much and was told that “you would be surprised how much you would know if you worked as hard as he did for that many years.” It’s valuable for students to look at people they admire, and then explore what it took for that person to succeed.
I ask students to discuss who inspires them and why. As a follow-up, if they could meet one psychologist, living or deceased, who would it be? The idea here is to get them to focus on people they admire, and then start to think about what aspects of these people they could incorporate into how they approach their career.
Communication is the key
One student concern that is reliably brought to my attention is a fear of speaking in public. This is not particularly surprising as it is one of the most common phobias. Hamming has an interesting perspective. He points out that to be successful, people need to be able to effectively communicate orally, in written work, and casually. By orally, he is referring to giving a presentation, and by casually, he is referring to discussing research in an informal setting. Hamming states that when watching a presentation of any kind, be it a lecture or formal academic talk, one should be listening both for content and for style. The listener should look for what they like, what they don’t, and what aspects they can adapt to their own style. To facilitate this type of discussion, I ask students to name the best presentation they have seen, which does not have to be restricted to academics. Some students list motivational speakers, comedians, or even politicians. The goal here is to get students thinking about which aspects of successful speakers they can incorporate into their own style.
Great (Final) Thoughts
The goal of these discussion questions is to encourage students to think about some of the big picture issues. This can be difficult to do, but using Hamming’s video as a springboard has been successful in my courses. While Hamming provides a lot of solid advice, his approach is not the only way. I like to have students debate aspects of Hamming’s view on work-life balance, and contrast this with a great article by Radhika Nagpal, who has a different slant and is also highly successful (Nagpal, 2013). For other perspectives on Hamming, I highly recommend Erren et al.’s (2015) work on Hamming’s rules for lifelong learning and for doing your best research (Erren et al., 2007).
I encourage instructors to try adding at least a few “Great Thought” questions into their lectures, and I would love to hear what types of questions you use and the responses that you receive.
Rodney Schmaltz is an Associate Professor of Psychology at MacEwan University. His research focuses on pseudoscientific thinking, with an emphasis on strategies to promote and teach scientific skepticism.
Erren, T. C., Slanger, T. E., Groß, J. V., Bourne, P. E., & Cullen, P. (2015). Ten Simple Rules for Lifelong Learning, According to Hamming. PLoS Computational Biology, 11(2), e1004020–6. http://doi.org/10.1371/journal.pcbi.1004020
Erren, T. C., Cullen, P., Erren, M., & Bourne, P. E. (2007). Ten Simple Rules for Doing Your Best Research, According to Hamming. PLoS Computational Biology, 3(10), e213–2. http://doi.org/10.1371/journal.pcbi.0030213
Hamming, R., & Kaiser, J. F. (1986). You and your research. Transcription of the Bell Communications Research Colloquium Seminar. University of Virginia, 7.
Nagpal, R. (2013). The awesomest 7-year postdoc or: How I learned to stop worrying and love the tenure-track faculty life. Scientific American, Guest Blog.
I have become a big fan of the bottom line up front concept (BLUF) so I will start with my recommendations and approaches to testing.
First, having students retrieve information from their memory improves the retention of information and skills. We have plenty of data (and meta-analyses) to show that. Typically, we ask students to retrieve information in the form of a test or quiz (hence it being called the testing effect). But we can functionalize this effect in a variety of ways within a course.
Pre-Course. Before the course even starts I give a multiple choice test. This serves 2 purposes. The first, broadly it assess what students know prior to starting my course. Second, it reactivates marginal knowledge – knowledge that may be stored within memory but has become inaccessible with time due to disuse. The information is in there but doesn’t come out easily.
Pre-Class. Before most classes I typically give an in-class, multiple choice quiz. This gives me some sense of what students have retained from pre-class preparation, and again it reactivates marginal knowledge. Also, I need to hold students accountable for their preparation, so a pre-class quiz is necessary. These pre-class quizzes are effective since students use points to assign importance. No points = they will skip preparing for class and if students are not prepared for class, it is challenging to teach them at higher levels of Bloom’s Taxonomy or get them to more deeply understand the material.
During class. I use several cases that have multiple choice questions – at least 3 or 4 per case. If I use clickers, I ask multiple choice questions regarding information covered during previous class periods. Why? To take advantage of the testing effect.
Post-Class. Most classes have extended learning opportunities – that is, homework. If there is homework, it usually consists of more open-ended questions so students have to generate responses. If they can generate the response during practice, it is easier to answer a multiple choice question on an exam. Harder if I reversed the order. These assignments also may take different perspectives from the class discussion to help students generalize the rules to problem solving.
Examinations. I give one maybe two high stakes exams, but they are cumulative and usually multiple choice. Cumulative is important because knowledge is cumulative, and because testing reinforces memory. If we spent time talking about it during class, I hope students retain the information over the semester. Why multiple choice? They’re just easier to grade, even though it can be challenging to write higher order multiple choice questions.
Well, now you know how the story ends. The rest is how I got to this point and more of the reasoning behind why I do things this way.
Teaching was easy when I knew nothing about learning or memory. As I learned about teaching, I started to learn more about learning and why what we did in the classroom should work. So for the past 10+ years, I’ve been teaching myself cognitive psychology. However, it was not until recently that I began to look at assessment as a learning tool. Yes, we typically think of formative assessment as a learning tool, but less so for summative. Typically we think of assessment to give feedback, however, assessment is a tool that can do so much more.
Here is the thing. To the student, the quiz or test is the most important thing. The assessment indicates to them what you find valuable for them to learn. This is probably why assessment is the second step in instructional design – develop learning objectives, figure how you will measure progress and if students meet those goals (i.e., assessment), and then design activities to help them achieve those goals.
Now, I work in a professional program. Students take prerequisite coursework and we hope they have remembered something. So as an instructor, I need to assess their prior knowledge, as it can make or break learning in my course. I also want students to apply their knowledge. However, in order for them to apply knowledge, they first need baseline knowledge. To do this, they need to prepare on their own and I need to assess where they are and what they know to hold them accountable. I also have to keep in mind that we forget very quickly and I need to go back and remind students of, again, prior knowledge. The good news, once information is learned, and then forgotten, it generally returns much faster than the time it took to originally learn that material. Ultimately, I need to make sure students have met the standards I set out for them, and testing can help me with this.
What is marginal knowledge? Think of a tip-of-the-tongue state. You know a word, what it may start with, about how long it is, but you can’t spit it out. Marginal knowledge is information that is stored well in your memory but you have not used it in a long time, so accessing it becomes problematic. Thus, you get the feeling of knowing the word but can’t quite access it. Beth Marsh and her graduate student Allison Cantor at Duke University demonstrated that we can reactivate marginal knowledge with a multiple-choice question if the answer choice is one of the choices. This is great, as it is much easier and more efficient to write a multiple-choice question than to re-teach information. Therefore, before the course or a class period, I use multiple choice questions to reactivate marginal knowledge and to assess student knowledge. Two birds – one stone.
Now, during class, we know that the testing effect is a pretty powerful learning tool, so how can we get students to retrieve information? Well, I use a variety of methods to do so, including quizzes, cases, questioning techniques and to some degree, clickers. Oddly, most research shows no learning benefit of clickers (good for keeping attention – not so good for learning). I believe this is because we use clickers incorrectly in terms of strengthening memory. Typically, an instructor will teach and throw up a clicker question. While this is very helpful for students to act on information as soon as they learn it, they are probably going to be able to answer the question correctly without too much effort. However, if you ask the same question during the next class period, some forgetting has happened, and the task of retrieval is more difficult, but also much more impactful in strengthening memory (and more helpful in diagnosing learning). Thus, I use clickers mostly at the beginning of class over material from the last class – or material even further back. This practice is not only retrieval, but also spaced retrieval with feedback, and is supported by the three components that help memory: testing, feedback, and spacing.
Then there are the exams. These are multiple choice simply because I have 150+ students and I can now write questions pretty easily at the middle or higher levels of Bloom’s Taxonomy. While this is mostly to assess where students are, I try to provide feedback on performance so students can continue to learn from the test. This feedback is delayed – right after the exam or the next day – because delaying feedback improves memory. Why? Maybe because it also acts as a retrieval attempt. Students have to think about the question and what they were thinking when they answered it.
Bottom line, assessments are important and everything we do in the classroom needs to be efficient and powerful. We are missing opportunities to help student learning if we don’t do more with testing.
Adam Persky is a clinical professor at the UNC Eshelman School of Pharmacy. Within the pharmacy school, he teaches physiology and pharmacokinetics and has received several of the School’s teaching awards, including Best Overall Instructor. He was named a Distinguished Teaching Scholar by the American Associations of College of Pharmacy. He is the associate editor for the American Journal of Pharmaceutical Education and on the editorial board for College Teaching. He has given over 100 workshops across the country on a variety of topics related to teaching and learning and has numerous publications within the scholarship of teaching and learning. He also equates teaching to baking. Prepare the ingredients (mise en place) and follow the instructions until you get comfortable with the recipe. Then ad lib once you know how things work but make small changes at first!
Like many conception stories filled with eager promises, Northwest Vista College’s Psi Beta chapter was conceived in a moment of spontaneity - in front of a hiring committee - when I was asked, “What strengths will you be bringing to the psychology department? “ I fervently responded with, “I would love to start a Psi Beta chapter on this campus.” That reply was a defining moment for me. Once hired, a colleague and I began the exciting, yet arduous task of following through on my passionate promise by assembling our own chapter of the national honor society for psych majors at two-year colleges.
Four years have passed – they grow up so fast – and since then the number of eligible Psi Beta candidates has more than doubled despite our declared psych major numbers remaining relatively stable. It’s clear too that our burgeoning chapter has significantly impacted our department. Students enrolled in psychology courses learn about Psi Beta during the first few days of class and our dynamic presence on campus is evident. Subsequently, psych majors are motivated to achieve academic excellence early on in their college career so that they too can be a part of something extraordinary. While raising a Psi Beta chapter isn’t child’s play, neither my colleague nor I anticipated the enormous benefits that our chapter would bestow upon us, our students, and our department.
If you would like to bring your own Psi Beta bundle of joy into the world, here are a few applicable life lessons we learned along the way:
What to Expect When You’re Expecting a Psi Beta Chapter
Lesson #1 – Choosing the Perfect Mate/Co-Advisor(s)
Select the right person to descend the rabbit hole with you. Thankfully, this proved to be an easy task for me. Dr. Cynthia Jacox, now retired, was a remarkably supportive co-advisor. No task felt too difficult with her by my side, and the encouragement she provided was invaluable. When establishing your chapter’s framework, it is extremely beneficial to have at least two different perspectives when considering the ways you’ll award academic excellence and support members’ interests. A joint effort by two well-intentioned advisors can make all the difference.
Lesson #2 – It Takes a Village
Everything you’ll need to provide sustenance for a successful chapter is located on the Psi Beta website. In fact, there is an entire section for potential Faculty Advisors that explains the application for the charter process and highlights the benefits of becoming an advisor. A bylaws template, chapter handbook, and sample letter seeking support from your college administration is also provided. The process is easy to navigate, and if questions arise, I can attest to the support that Dr. Jerry Rudmann, Executive Director of Psi Beta, and his staff as well as the Psi Beta Council will happily provide.
Birth of your Chapter and Associated Labor Pains
Lesson #3 – It’s a …… Psi Beta Chapter!
Once the charter has been approved by the Psi Beta Council, and you’ve followed your college’s Student Life protocol for new organizations, you are now ready for delivery. Your college’s Institutional Research Team should be able to assist in identifying potential Psi Beta candidates that meet your designated criteria as established in your bylaws. In my experience, notifying the potential candidates of their acceptance is one of the most exciting, but laborious aspects of an advisor’s duties. The reason for this is simple and something most college instructors already encounter on a weekly basis: Students aren’t the best at checking their college-issued email. In an effort to circumvent this minor obstacle, some chapters will deliver hand-written invitations to each eligible student.
Offering orientation sessions for Psi Beta candidates early in the semester is an excellent way to bring the new cohort together and provide introductory information, such as the chapter mission and goals, benefits, active membership requirements/bylaws, officer duties/vacancies and upcoming events.
Lesson # 4 – Celebrate Your New Additions
Commemorating new members’ entry into your Psi Beta chapter is a notable event. The induction ceremony honors those that have achieved academic excellence and are eager to become future contributors in the field of psychology. The Psi Beta website offers explicit guides on how to conduct various ceremonies, including the induction ceremony. Induction ceremonies can range from extravagant to low key depending on factors such as budget and time constraints. Our first few ceremonies were fairly elaborate. Families were invited, a classical guitarist played in the background, dinner was served, candles were lit, and our charismatic keynote speaker wore a bowtie and gave each new member a $20 bill. Since then, we have settled on a more informal, yet intimate ceremony that takes places during a regularly scheduled meeting.
The Early Years through Adolescence
Lesson #5 – Navigating the Playground
Now that your chapter has been established, it’s time to focus on exploration, growth and sustainability. As part of the development of our initial framework, we asked ourselves how our chapter would promote student development within the field of psychology. We identified four critical components that align with the mission of Psi Beta: Community Service, Educational Opportunities, Research, and Collaboration. Prior to each semester, advisors, officers, and interested members meet to brainstorm how we’ll best fulfill each component. The critical element in the design of your semester activities is that they should be student-driven. Let your officers/members dictate which guest speakers will present, which tours you’ll take, which service opportunities the chapter will engage in, and whether to participate in the annual Psi Beta research project, etc. Better yet, ask them to organize these events. Our job as advisors is to monitor and facilitate, analogous to the parental role on a playground. We have to fight the urge to swoop in and grip their legs as they traverse the monkey bars. Avoid over-advising when possible. Comparable to the adverse effects of over-parenting, over-advising can limit members’ sense of autonomy and defuse initiative.
Lesson #6 – The Graduation Speech
We have recently started a new tradition within our chapter. Graduating members are encouraged to stand during our last meeting in May and share their final thoughts/words of wisdom with the group. I am never prepared for what comes next. Words such as life-changing, transformative, gratitude, understood, supportive, and motivated emerge as a common thread in the tapestry woven before us. Tears are shed, people embrace, and the strong friendships forged are never more evident than at that moment in time. One student shared how she felt adrift at the beginning of her college career. It wasn’t until her discovery of psychology and her subsequent involvement in Psi Beta that she finally felt at home. This seems to be a mutual sentiment expressed by many within the chapter. Offering a Psi Beta chapter affords members an opportunity to prop each other up as they face obstacles and progress toward their goals. And as an advisor, hearing how the chapter has impacted your members’ lives is incredibly rewarding and motivating.
Emerging Adulthood and Beyond
Lesson #7 – Facilitating Collaboration
At Northwest Vista College, our psychology club and Psi Beta chapter are heavily intertwined. Our psychology club was well-established before the inception of our Psi Beta chapter. Both the club and the chapter are similar in purpose; however, the chapter encourages academic excellence, provides more research opportunities, and offers national recognition through various awards, scholarships, and grants. We found that layering the honor society on top of the existing psychology club was beneficial for all. The direct relationship between the two created a pathway so that lateral movement and participation is easy. Psychology club members are inspired to meet the criteria necessary for Psi Beta membership, and Psi Beta members placed on probation are encouraged to participate in psychology club pursuits. Either direction fosters continued growth as both the club and the chapter expose members to a myriad of developmental opportunities.
Lesson #8 – A Psi Beta Chapter (Life) Review
Over the past four years, I have been privy to a secret, something only known and whispered about by other Psi Beta and psychology club advisors. While some of our colleagues are dedicating their time to the latest college and state directed initiatives, we are joyfully working alongside students outside of the classroom setting. We are grooming our freshman and sophomore members for more than just the next course or transfer to a four-year university. Our ultimate goal is to prepare our members to become competitive applicants for graduate schools of their choice. Achieving this goal entails exposure to various aspects of the field and acquiring hands-on experience, such as conducting research and presenting at conferences, volunteering/interning at psychology-related organizations, and discerning which subfield of psychology bestfits their interests/skills. Here is the real secret, from the advisory perch we have a truly special view of something within our members that tends to be elusive – intrapersonal growth. We witness members evolve from academic drifters with an interest in psychology to self-assured colleagues who may eventually birth a Psi Beta chapter of their own.
“Psi Beta was beyond a worthwhile experience. It became the place for guidance; career-wise, as well as personally, there were opportunities and knowledge available. My fellow members and advisors changed my life.” – Diane Goguen, Former President of the Northwest Vista College Chapter
Jennifer Fox is an Assistant Professor of Psychology at Northwest Vista College in San Antonio, Texas. Prior to her role as co-advisor of Psi Beta, she served as a co-advisor for the psychology club at South Texas College for 8 years. She is passionate about promoting inter and intrapersonal growth among her students as they prepare to become future contributors to the field.
“Our students live in cacophony. Clamour, chatter and din fill their ears, and may even injure them…. We can educate ourselves to be models of intellectuals who trust and value silence, who practice what we have always known; when no one is speaking, someone is learning. We can create oases of silence where cool springs of insight trickle and flow.”(1)
I slipped into an auditorium where nearly every seat was taken—150 medical and doctoral students in the basic sciences. The class on the topic of oral pathology was to be taught by a relatively new faculty member on our campus.
It didn’t take me long to see why this professor was considered to be one of the best. She had spent hours developing detailed slides to illustrate her topic. She used good examples. She spoke clearly and at a good tempo. But, she was the only one who spoke. She didn’t ask a single question of the students and the students didn’t ask a single question from her. There were no pauses for students to consult with each other, to apply the ideas presented to a case, or to write an answer to a thought question. Students simply sat and listened. When the time was up, everyone quickly left.
A couple of days later, I shared my notes with the professor. I showed her a number of positive strategies she used in her teaching and complimented her on her excellent learning materials. I shared my observations of the attention levels of the students, especially my awareness of the learner fatigue that seemed to be present toward the end of the hour, as students note-taking diminished, their eyes began to glaze over, and shoulders slumped. Some even took out their phones. The instructor took this feedback but didn’t ask me for any suggestions. As I was leaving, I mentioned that I would be happy to look at her next lecture plan to see if I might have any ideas to help with student engagement, if she would like. She looked quite surprised at this and asked, “What would you have done for this lecture?”
When I showed her that no one spoke but her and that we know that the person doing the talking is the one doing the learning, she responded, “Oh, I know. I love giving this lecture because it nails it down in my head.”
A few pauses during the hour-long session could have made a big difference—the difference between a well-delivered presentation and a powerful learning experience for students, but when I tried to suggest this, her reply was “Oh, no—I don’t have time for any of that—I must end on time and I have carefully planned for each minute so I am sure to cover all of the important points.” It was all about her. It was all about teaching, not about learning.
Myths That Keep Us from Pausing
How many times have we been in similar situations? Too much content to cover and not enough time—so we fill all of our allotted time with teacher talk? It is so hard for us to give up the myths that shape our classroom behavior:
If I don’t say it, they won’t learn it
If I DO say it, they WILL learn it
I must cover the content
Cognitive Science Research Supports Pausing
But we cannot ignore important studies, starting as early as the 70’s and 80’s and replicated within the last few years, which all report similar outcomes. These studies looked at pausing briefly during the lecture. Instructors in these studies taught two groups—one had lecture the whole time, the other took an occasional break, a 1-3-minute pause. The teachers who spent less time talking covered less content, but paused to allow students to briefly share their notes with each other, to discuss together, or to instruct each other. In every case, the group who had less lecture--less material covered--had the following results:
More content recalled from the presentation
Better test results, short term and long term
Higher evaluation scores for the presentation from participants (2-8)
Talking Equals Learning: Listening Rarely Does
These are outcomes we all seek as instructors, but yet we find it hard to believe that if we talk less our students will learn more. We have a difficult time giving up the podium. C. Bland Tomkinson says it well, “Cover less, discover more.”(9) In other words, it is wise to talk less and plan for pauses—opportunities for students to do some of the talking, because only when they share in the talking are they truly learning.
Herman has it figured out.
Pauses Are Important—Particularly at the Start, Middle, and Close of the Lesson
One small change in our teaching can lead to big changes in learning. A great way to try a small change is to insert an occasional pause into our lectures. In Part 2 of this blog post, we will examine characteristics of good pauses throughout the learning experience and we will see what the literature tells us about what constitutes a good starting pause, as well as a good middle pause, but let’s focus on closing pauses for now. Chip and Dan Heath (10) suggest that teachers can create memorable, meaningful moments by paying attention to endings—if something ends well, we tend to let that ending represent the entire experience. What we are discovering is that when we pause at the end, we can create these moments that will “add extra oomph into a lesson,” (11) or reverberate for hours after the lesson.(12)
Power of a Good Closing Pause: Get Students to Commit
Finley’s “extra oomph” will only happen when learners create and share their own meaning of the lesson. They must do more than merely think about it. If we seek behavior change, we will give students opportunities to make public commitments. We hear from motivational speakers and life change specialists that it is important to not only think about but also write or speak of our goals. An important study highlights this idea. Nearly 300 adults were asked about goals they had for the near future—things like losing weight, buying a house, selling a car. Some were asked to write down their goals. Others spoke to a partner about them and others were encouraged to simply think about their goals. After one month, the participants reported on their results. Those who had written down or spoken about their goals achieved those goals at a significantly higher rate than those who had merely thought about them.(13) This has such importance for us as instructors wanting to bring about significant change in our students. Why would we let them leave our classrooms or laboratories without tapping into this important strategy? Let’s get them to commit to more than just thinking about what they have learned. Let’s encourage them to write or talk about what they will do with what they are learning. In so doing, we not only increase retention, but we create the conditions for action. Those last few minutes of class are so powerful that we cannot afford to waste them with our summaries. Our students’ own written and spoken summaries and their ideas about how to apply their new knowledge —public commitments—will create memorable classes but also life-changing behavior change.
Questions We Can Ask in Closing
Writing or speaking answers to these kinds of questions will help students to create powerful endings:
What is one thing you learned today?
What concepts or insights were important to you from class?
What did you understand better after today’s class?
How would you summarize today’s class for someone who wasn’t here?
What will you do differently as a result of being in class today?
Why are you glad you were here today?
How will what we discussed today affect your professional or personal success?
What is your action plan after being here today?
Characteristics of a Good Closing Pause
These kinds of questions, posed at the close of learning, will contribute to an ideal closing pause, which may accomplish one or more of these goals:
Require personal reflection—develop new insights
Motivate action/follow up
Bring full circle and create a bookend(14)
James Lang, in his article, “Small Changes in Teaching: The Last 5 Minutes of Class,”(15) gives a good example of bringing full circle. At the beginning of class give students a few questions that you plan to address. Then at the end, have them take a look back at the questions and share how their thinking has developed.
Examples of Closing Pauses Making a Difference
Here is a good example of reviewing, reflecting, celebrating, and motivating. One of my colleagues looked for a powerful way to pause at the end of a workshop she presents to health professional students on the importance of being sensitive to the challenges of aging when they are providing care to patients. She has the students go through a series of activities, like putting Vicks in their nose and then trying to identify tastes or smells or putting a brace on their leg so it won’t bend and then trying to get in and out of an automobile. This was followed by a lecture/discussion. To close, she gave each student a sticky note on which they were to respond to the question, “What is the one thing you do not want to forget from our time together today?” Students completed their responses and then brought them to the front of the room. Finally, students were asked to select one of the sticky notes that best summarized all of the rest of the notes. The one they selected said, “I will never rush an old person again.” The student who had written this was applauded and given a small award.(16) It is hard to imagine any of the students present that day ever forgetting that class or the take-home messages they had publicly shared. I am sure that the closing pause resulted in at least one individual present who has never rushed an elderly patient since.
How many times have we taught classes with powerful messages but we lost much of the potential power of that class because we failed to provide that closing pause which asked the question, “What is the one thing you don’t want to forget from our time together today?” Because we didn’t ask it, students didn’t think about it and quickly forgot most of what we had hoped to achieve during our time together.
A Suggestion for the Basic Science Instructor
Here is an idea for our instructor of the large basic science class. During the last few minutes of class, she might ask students to write a multiple-choice exam question over the topic, share it with a neighbor, improve each other’s questions, write their names on the back of the card, and turn them in—with the promise of bonus points for each if a question is used on an upcoming exam. This would give them an opportunity to review the content of the hour, a chance to connect with a classmate, and an exciting opportunity to earn some recognition and a bonus point or two.(17) This pause is particularly well-suited for a content-heavy class, as it provides students with opportunities to review. It seems that this immediate review has advantages over the more typical review done some time later when students pull out their notes to prepare for the upcoming exam. When students review frequently during a class—most suggest every 15-20 minutes, but at least, at the close—retention is improved, both short- and long-term.
Importance of Keeping Pauses Fresh
Not only is it important to give our students those last few minutes of class to make meaning for themselves. It is also worth taking the time to change things up—to avoid becoming too predictable. We professors do tend to get into ruts fairly easily, and a minute paper every class period will get old quickly. Research supports the fact that the brain pays more attention to what is new or different than what has always been the same. So it is worth looking for different ways to help students review, value, and plan at the end of the learning session.
Sources for Ideas for Closing Pauses
There are a number of creative and fresh ideas for closing learning with a meaningful pause. Hitting Pause: 65 Lecture Breaks to Refresh and Reinforce Learning, provides stories of closing pauses in chapters 5 and 7 and gives 26 closing pause ideas in the appendix. (See discount offer from Stylus Publications, LLC, below). The book also cites a number of websites for additional ideas.
Conclusion: We all hope that our classes have life-changing effects on our students. But the reality is that this rarely, if ever, happens. Every class period is largely taken up with lecture—lecture without any breaks or pauses—lectures which are pretty much like all of the previous lectures. Lang suggests that improving teaching requires minimal changes to our lecture plans and yet has the power to boost the learning in substantive ways. Perhaps if we focused on creating oases of silent, powerful pauses during those last few minutes of class, we might transform our classes into unforgettably meaningful experiences.
I’ll be back on the Noba blog soon with a follow-up post focused on the why and how of creating good Starting and Middle pauses.
Dr. Gail Rice is a professor at Loma Linda University, where she directs faculty development for the campus. She teaches for the Harvard Macy Institute for Education in the Health Professions in Boston and the USC Keck Medical School Innovations in Medical Education conference. She presents for organizations and campuses worldwide and has published books and articles for peer reviewed journals on various topics relating to creative, effective teaching in higher education. Her most recent publication is Hitting Pause: 65 Lecture Breaks to Refresh and Reinforce Learning, published by Stylus Publications. Gail and her husband, Richard, have devoted their lives to finding fresh and effective ways to teach university students.
Hitting Pause: 65 Lecture Breaks to Refresh and Reinforce Learning - Book Discount
Pauses constitute a simple technique for enlivening and enhancing the effectiveness of lectures, or indeed of any form of instruction, whether a presentation or in an experiential setting. This book presents the evidence and rationale for breaking up lectures into shorter segments by using pauses to focus attention, reinforce key points, and review learning. It also provides 65 adaptable pause ideas to use at the opening of class, mid-way through, or as closers.
For 25% off Hitting Pause, follow the link below and use code HP25 when ordering.
1.Marken, R. (2008) Silences. Ottawa, Ontario, Canada: Council of 3M National Teaching Fellows of the Society of Teaching and Learning in Higher Education, p 115.
2.Ruhl KL, Hughes CA, Schloss PJ. 1987. Using the pause procedure to enhance lecture recall. Teach Educ Spec Educ. 10:14–18.
3.Ruhl KL, Suritsky S. 1995. The pause procedure and/or an outline: effect on immediate free recall and lecture notes taken by college students with learning disabilities. Learn Disabil Q. 18:2–11
4.Stuart J, Rutherford RJ. 1978. Medical student concentration during lectures. Lancet. 2:514–516.
5.Lukas W. Richards, Amy T. Wang, Saswati Mahapatra, Sarah M. Jenkins, Nerissa M. Collins, Thomas J. Beckman & Christopher M. Wittich (2017) Use of the pause procedure in continuing medical education: A randomized controlled intervention study, Medical Teacher, 39:1, 74-78, DOI: 10.1080/0142159X.2016.1230664
6.Ghorbani, A and Ghazvini, K. 2016. Using paper presentation breaks during didactic lectures improves learning of physiology in undergraduate students. Adv Physiol Educ. Mar;40(1):93-97. Pubmed/26873895.
7.Zhang, N, Henderson, CN. 2016. Brief, cooperative peer-instruction sessions during lectures enhance student recall and comprehension. J Chiropr Educ. 2016 Oct 30(2):87-93. Pubmed 26967766.
8.Bachhel, R., Thaman, RG. 2014. Effective use of pause procedure to enhance student engagement and learning. J Clin Diagn Res. 2014 Aug 8(8)XM01-XM03. Pub med/25302251
14.Lutsky, N. 2010. Teaching psychology’s endings: The simple gifts of a reflective close. In D. S. Dunn, B. B. Beins, M. A. McCarthy, & G. . Hill IV (Eds), Best practices for teaching beginnings and endings in the psychology major: Research, cases, and recommendations (pp. 331-349). New York, NY: Oxford University Press.
15.Lang, J. 2016. Small changes in teaching: The last 5 minutes of class. The Chronicle of Higher Education. March 7, 2016.
16.Rice, G. 2018. Hitting Pause: 65 lecture breaks to refresh and reinforce learning. Sterling, VA: Stylus Publications, LLC. (Sticky Note Closure, CP #45).
17.Rice, G. ibid. (Exam Question Challenge, CP #58).