• Appendix A.  Resources to ground pre-proposal discussions
• Appendix B.  Creating a course assessment plan
• Appendix C.  Working with an Active Learning Initiative fellow
• Appendix D.  A short research sampler on teaching

Appendix A.  Resources to ground pre-proposal discussions

The ALI was inspired by Carl Wieman’s science education initiatives at the University of British Columbia (CWSEI) and at the University of Colorado at Boulder (SEI), as well as Wieman’s book Improving How Universities Teach Science (Harvard, 2017). Having helped more than 250 faculty members introduce active learning into their teaching across several disciplines, the CWSEI and SEI have created rich online resources to help departments and faculty members redesign courses. While targeted at STEM teaching, much of this material is directly applicable in other disciplines. Useful links include:

• The Science Education Initiative Handbook: this handbook guides faculty on starting a course transformation project and effectively collaborating with embedded teaching specialists (postdocs).
• Transforming STEM Education: a collection of documents offering detailed advice for departments and faculty members on how to redesign courses.
• Instructor Resources: a collection of short guides for instructors—on assessment, clicker use, student engagement, etc.—that illustrates in concrete terms the pedagogical philosophy (active engagement of students) underlying these initiatives. The advice is highly practical.
• Inside the Classroom: a collection of videos that show, among other things, what active learning looks like.
• Recommended Reading: an annotated bibliography of papers on the research behind many aspects of active learning.

Top of page

Appendix B.  Creating a course assessment plan

Structured assessment is essential when redesigning a course. It provides information about what is working well and what could be improved. Assessment methods can help to target the development of teaching materials and methods by identifying which topics or skills students struggle to learn well. CTI can also assist departments through the process of obtaining permissions for human research studies from the Institutional Research Board.

Possible assessment strategies include:

Grades, scores, tracker questions, and concept inventories: Direct measures of learning are important when evaluating a new pedagogy. This data might include final grades, scores on exams or assignments, scores on components of a rubric, or points given to individual exam questions (such as tracker questions reused, in disguise, from one semester to another). Tests, often called concept inventories, can be given to students at the beginning of a course or a unit, and the same questions asked again at the end of the course or unit to measure how much students have learned.

Mid-semester feedback: Mid-semester feedback from students allows instructors to address and resolve issues with a course before the semester ends. CTI administers a mid-semester feedback program and can distribute survey questions to students and then report and discuss the information with faculty. Other options for mid-semester feedback include minute papers or polling questions asking for anonymous responses to questions.

Class observations using COPUS (Course Observation Protocol for Undergraduate STEM): With the COPUS protocol for class observations, a trained observer from CTI comes to class several times a semester and records the type of activity happening during two-minute intervals (for example: lecturing, group work, class discussion, polling question, students writing etc.). COPUS observations can help instructors find the right balance between time spent on different activities. They also document changes in the course over time.

Student attitudes, motivation, confidence, and sense of belonging: Surveys, focus groups, or interviews can assess factors such as student mindset, motivation, confidence, and attitudes towards the subject material and learning experiences in your course. For example, some initial findings indicate that Cornell ALI courses improve students’ sense of self-efficacy and confidence in science. Other ALI departments are researching student motivation and attitudes toward the discipline. Awareness of metacognitive strategies has been another area of interest for some departments in the initiative. Some departments are also using structured interviews to learn more about student misconceptions about course material and their experiences in learning.

Student reflections about their own learning: Reflection can take the form of “minute papers” (short responses to a prompt, for example, “what was the most interesting concept you learned today?”). Other options include reflective responses on an online class forum after a discussion, or a reflective essay at the end of the semester. Instructors can also ask students the same reflection question at the beginning of the course and the end of the course (for example: “What relevance do you think this course will have for your life outside of this class?”).

Faculty development and engagement with active learning: Faculty can measure and document changes in their teaching practices by completing the Teaching Practices Inventory survey before and after a project. Other options include documenting faculty perceptions and experiences implementing active learning, how their attitudes towards teaching have changed, and what changes they have noticed in their classes and with students. Surveys, interviews, and focus groups with faculty and TAs are possible.

Top of page

Appendix C.  Working with an Active Learning Initiative fellow 

Active Learning Initiative department grants give you the opportunity to work with an ALI fellow (postdoc) to improve your course and develop active learning questions, activities, assignments, and other materials. Based on many conversations with previous ALI faculty, we’ve distilled the following advice.

Five tips to make it a positive experience:

• Set up regular weekly meetings with the ALI fellow.
• Identify the trouble spots in the course – i.e., which topics or assignments do students struggle with the most? Focus on improving these areas.
• Enjoy the intellectual and creative process. In the best collaborations, ALI faculty and fellows bounce ideas off each other to develop engaging, innovative, and enjoyable activities that foster interaction and create a positive learning experience for students. 
• Choose one or two active learning strategies and become comfortable with incorporating them into each class. Set up a routine in class so students become accustomed to what is expected.
• Allow enough time for activities in class. Be willing to shorten parts of your lecture and let go of some content. Consider asking students to read ahead of time or complete other pre-class work to learn some of the basics so that there is more time for them to practice, discuss, and get feedback during the class time.

A few things that can cause difficulties:

• Don’t wait until the last minute to make changes and design activities with the ALI fellow. Start planning and meeting during the summer or the semester before you are teaching the course.
• Design learning activities that can be easily implemented and updated by yourself and other instructors in the future. Remember that the ALI fellow will eventually leave, and the changes should be sustainable.
• Take time to involve and train Teaching Assistants if they are expected to implement or help with active learning activities. TAs can also help design and give feedback on learning activities.

Generally, within a department, ALI fellows will:

• Collaborate with faculty on course transformation projects to implement active learning. Typically, an ALI fellow will focus on one course each semester, with secondary work for one or two other courses.
• Serve as a resource within their departments (consult with faculty, organize seminars or workshops about teaching)
• Collect and analyze evidence of student learning and experiences to guide course improvements

Typical timeline for a course transformation project:

• Before the semester starts, an ALI postdoc can collaborate with faculty to:
  • Refine or develop learning outcomes.
  • Identify aspects of the course that most need improvement: i.e., areas where students tend to struggle.
  • Decide on one or two main types of active learning strategies to implement throughout the semester (e.g., polling questions, think-pair-share, worksheet activities, group work, problem-solving, etc.).
  • Design learning materials (i.e., activities, worksheets, polling questions, assignments, etc.)
  • Develop assessments of student learning and experiences (pre/post quizzes, surveys, rubrics, revise prelims, etc.)
  • Review educational literature to identify best practices to implement
  • Develop TA instructions and training materials
• While the course is being taught the ALI fellow can:
  • Sit in and observe the classes, take notes on student engagement, length of activities, reflections on how things are going, and troubleshoot technology.
  • Move around the class to help students during activities.
  • Co-teach: some ALI fellows lead activities or teach parts of class sessions.
  • Help train and mentor TAs in implementing active learning in sections, labs, etc.
• After the semester ends:
  • Analyze student data to inform educational changes and course revisions.
  • Make improvements to activities, assignments, assessments, etc.
  • Develop new materials for the next iteration of the course.
  • Archive curricular materials for use in upcoming semesters.
  • Author or co-author publications and/or presentations.

Top of page

Appendix D.  A short research sampler on teaching

What follows is a small sample from the thousand plus research papers on active learning and teaching. Research in Cornell classes both confirms and extends these findings (selected ALI research publications). Books that offer more systematic surveys include: D. Schwartz et al, The ABCs of How We Learn (Norton, 2016); S. Ambrose et al, How Learning Works (Wiley, 2010); and J. Bransford et al, How People Learn, (National Academies Press, 2000).

S. Freeman et al, Active Learning Increases Student Performance in Science, Engineering and Mathematics, PNAS 111 (2014) 8410: Following meta-analysis practices familiar from medicine, these authors examined more than 200 articles, from 8 disciplines, on the impact of active learning. They find among other things that grades increased by half a letter grade with active learning (Fig. 2 in the paper), and failure rates decreased by a third (Fig. 1B).  They question whether, in light of these results, it is ethical to use conventional lecturing as a control in education research given the damage it inflicts on students in the control group. Carl Wieman’s commentary on this article provides an accessible and interesting summary of the paper’s results and their implications: PNAS 111 (2014) 8319.

Theobald et al. Active Learning Narrows Achievement Gaps for Underrepresented Students in Undergraduate Science, Technology, Engineering, and Math, PNAS 117 (2020) 6476. This is another recent meta-analysis study that shows that classes in which students spend a high proportion of time in active learning activities reduced the exam and grade performance gaps of underrepresented minority and low-income students when compared with traditional lecture classes. The authors argue that courses that combine opportunities for deliberate practice within an intentionally inclusive environment can increase equity in higher education attainment by improving the learning and retention of minority students.

S. Wineburg et al, What is Learned in College History Classes?, Journal of American History 104 (2018) 983: Much discipline-based education research is in STEM fields, but similar research is available in other disciplines. This paper is part of a multi-year study of differences between novice and expert historians in their evaluations and interpretations of historical sources. The study identifies analysis skills central to the discipline that are reflexive for experts but almost nonexistent for novices (i.e., undergraduates); and it provides tools for addressing these shortcomings through repeated deliberate practice, with immediate feedback, in history courses. For information about how this research is done (and another interesting example) see: S. Wineburg, Cognitive Science 22 (2010) 319. For a similar study, about reading poetry, see: J. Peskin, Cognition and Instruction 16 (1998) 235.

L. Deslauriers et al, Improved Learning in a Large-Enrollment Physics Class, Science 332 (2011) 862: This paper describes an experiment where a large introductory physics class was split in two for a week in mid-semester, with one group taught conventionally and the other using active learning. Student learning was assessed with an in-class test given after the intervention but designed beforehand. The entire grade distribution was shifted up two letter grades in the active-learning group (Fig. 1 in the paper). The authors also measured student attention levels during class (using a standard protocol) and showed it doubled in the new format. This kind of improvement has long been apparent in introductory physics courses: see R.R. Hake, Am. J. Phys. 66 (1998) 64, for a famous early study of 62 introductory courses by 62 different instructors at 62 institutions (see Fig. 2).

M.K. Smith et al, Combining Peer Discussion with Lecturer Explanation Increases Student Learning for In-Class Concept Questions, CBE—Life Sci. Ed. 10 (2011) 55: This semester-long controlled study, in two biology courses, compared the relative impacts of peer discussions (student-student) and instructor explanations on students’ ability to absorb new concepts in class. They found that peer discussion followed by an explanation from the instructor was twice as effective as either peer instruction or an instructor explanation alone (Fig. 4A in the paper). By itself, peer discussion was slightly more effective than instructor explanation, but the difference was not statistically significant except for the strongest students in the class—peer discussion was twice as beneficial for them as instructor explanation (Fig. 5A). These results indicate that lecturing can be very powerful provided the students are first engaged; and even strong students benefit from peer discussion. The benefits of having students struggle with a problem before they hear the expert solution (i.e., the lecture) is a theme in many other studies from a variety of disciplines: for example, D.L. Schwartz et al, A Time for Telling, Cognition and Instruction 16 (1999) 475 shows how having students analyze “contrasting cases” and then hear a lecture substantially improved their learning in psychology (Fig. 5). Lead author of the biology paper, Michelle Smith,  is a professor at Cornell in Ecology and Evolutionary Biology, and the Senior Associate Dean for undergraduate education in the College of Arts & Sciences. Smith also provides advice on writing effective questions for student learning through peer discussion (ch. 10 in Active Learning in College Science).

N.G. Holmes et al, Teaching Critical Thinking, PNAS 112 (2015) 11199: This controlled study, in a freshman-level physics lab, shows how directed practice taught students to make expert-like decisions about data: Do the data prove anything? How should the experiment be changed to improve the data? Do the data disprove the model? How must the model be changed?  And so on. The directed practice was phased out during the semester. By the end, students in the experimental group were outperforming the control group by factors of 5-10, and they continued to outperform in a subsequent course (Figs. 1 and 2 in the paper). Lead author, Natasha Holmes, joined the Physics faculty at Cornell and has been redesigning Physics laboratory classes as part of an ALI project. The redesigned labs provide students with more authentic experiences in scientific inquiry and discovery where they have agency to make decisions about their experiments (ch. 18 in Active Learning in College Science).

Top of page