Sheridan Center and Digital Learning & Design staff report on the May 2020 faculty survey, linking key themes to research and highlighting faculty examples for online/hybrid teaching.

In Spring 2020, the unprecedented scale of the move to remote instruction generated an array of creative responses by Brown University faculty to two challenges: continuing the delivery of high quality teaching and preserving a sense of a learning community. To capture faculty input about their experience, in May 2020, all faculty were asked to respond to a survey that invited them to reflect on three dimensions of the transition: “Thinking about the courses that you moved to remote instruction this spring, please tell us about one teaching approach or strategy that worked well for you and your students. Why do you think this was effective? What might you do differently next term?”* 

With a very high response rate (82%), this survey offers an opportunity to share a comprehensive picture of teaching approaches that faculty perceived to be effective at Brown. We -- Sheridan Center and Digital Learning & Design staff -- link key themes to research and highlight specific examples from 20 faculty. Although the faculty perspective is the key focus of this newsletter, for context, we do offer that student feedback was generally quite positive, with 92% of students agreeing that their spring 2020 courses were effective for their learning experience (83% response rate to course feedback).

Most frequently (76 comments), responses suggested a general experience of a relatively fluid transition to Zoom, without significant adjustments. In most cases, though not exclusively (see textbox below), faculty reported that this shift was made possible due to small class sizes or a graduate seminar format. Faculty appreciated the ability to “see faces and hear each others’ ideas voiced in real time” (Elizabeth Bryan, English) and noted that students valued features such as Zoom’s hand-raising and chat functions.

Moving to Zoom lecture went smoothly, and incorporating online questions, interactions, etc., all proceeded easier than I was expecting.

-Savvas Koushiappas, Physics (PHYS 0040: Electricity and Magnetism)

Below, we highlight other responses where Brown faculty describe specific adjustments that they perceived worked well for teaching and learning. Any of these ideas can also be adapted for the 2020-21 move to fully online and hybrid instruction. (Please see this document for a definition of these terms.) In Part I of this newsletter, we focus on the top three themes that arose in the faculty survey: asynchronicity, breakout discussions, and increased 1:1 contact with students. In Part II, we focus on other themes in faculty responses: change in assessments and written discussions.

Asynchronicity
In the Spring 2020 term, Brown students were located around the world in over 25 distinct time zones. Therefore, it is not surprising that a frequent comment (69 responses) focused on ways that faculty were able to transform course interactions into time- and space-independent learning experiences.

I think the Zoom recordings on our Canvas site provided a great way for students to review the material. Next year I will set up to capture all our lectures for the course regardless of whether we have in-person or remote teaching.

-Judith Bender, BIOL2150 (Scientific Communication)

Most frequently, this shift happened through distributions of Zoom lecture recordings, but in some cases, faculty used text. For example, Laura Bass (Hispanic Studies) and Evie Lincoln (History of Art and Architecture) posted minutes after each discussion for students who could not attend their Collaborative Humanities graduate seminar. (Although they typed up the minutes this term, they noted that the task of composing the minutes could be rotated among students and instructors.) Similarly, Joan Copjec (MCM) wrote a short lecture each week which she distributed to the students, along with specific questions she hoped the students would discuss. The TA for the class discussed with the students the lecture and questions and formulated questions of their own, which Copjec then responded to in writing.

In ideas about what faculty would do differently next time, asynchronicity surfaced as the most frequent theme (17 responses). As Dixa Ramírez D'Oleo (American Studies and English) remarked, this approach has the benefit of “accommodating students in various home environments and time zones.” 

A study of millions of students who engaged in MOOCs (Massive Online Open Courses) found that the following strategies were most effective for online lecture recordings (Guo, Kim, & Rubin, 2014):

• “Chunk” videos into shorter increments. In this study, students’ engagement waned after just six minutes.
• Videos do not need to be high production. Simple videos work well, as do whiteboard-style videos (e.g., recording problem-solving in STEM or marking up a text in the humanities) (Costa, 2020).
• Although this study of MOOC learners suggested that images of the instructor speaking and making “eye contact” can help with student engagement, it’s also OK if you are not comfortable recording yourself. One careful study of a more natural environment (a physics classroom) found that this image is not necessary to impact learning outcomes (Dey, Burn, & Gerdes, 2009).  

Additionally, embedding short low- or no-stakes quizzes into the recordings can help increase student engagement and performance on later course assessments (Kinsella, Mahon, & Lillis, 2017). This Panopto quizzes article describes a recommended approach for integrating quizzes into videos. If recording over slides, this resource offers research-based principles for effective multimedia presentations.

Breakout Discussions
Second most frequently named, faculty indicated that Zoom breakout rooms worked effectively (57 comments). These mirrored small-group discussions in a face-to-face classroom, often using Google Docs or screen sharing for students to document the outcomes of their work. For example, inspired by a technique used by Cindy Nguyen (History) in pre-pandemic teaching, Amy Remensnyder (History) gave students creative whiteboard tasks to complete in small groups, such as making collective drawings of key ideas in the readings. Prof. Remensnyder observed that this approach “allowed students the freedom to engage creatively with analytic materials at a time when their abilities to concentrate on more traditional forms of analysis were under considerable strain.”

I had a very good experience moving online. I used breakout rooms for students to work in small groups to create products on google docs that they could share with the class. 

-Laura Snyder, Education

Breakout rooms can be effective tools for helping students build community, engage deeply with course material, and decrease anxiety about the class discussion (Berry, 2019; Chi & Wylie, 2014; Clark et al., 2019; Eddy, et al., 2019). Like with face-to-face small-group activities, it is useful to offer clear guidelines (task to be completed, time students will be in the small group, reporting out expectations, and student roles) for students before they enter the breakout rooms (Brown et al., 2016). Small-group activities should also be intentionally facilitated because evidence-based approaches for online breakout discussions are often counterintuitive:

• While 5-6 students is often the recommended number for face-to-face small-group discussions, the optimal size for virtual breakout groups is likely much lower -- about three students per group -- to help maintain high discussion quality (Lowry et al., 2006).
• While mixing up groups throughout the term is a common practice, research on equity in group discussion suggests that it may be more useful to maintain membership over the term, to allow group members to build connections (Eddy et al., 2015).
• While many instructors encourage small groups early in the term to allow students a chance to get to know each other, it is equally or more important to highly structure breakouts later, because student networks tend to dissolve over the term (Brown, 2019).

Additionally, if using small groups for higher-stakes collaborative projects or papers, it can be helpful to have group members regularly self-evaluate their own interactions, processes and procedures. Martin et al. (2019) offer useful questions for group members to examine their team practice on four justice-related dimensions: distributive, procedural, interpersonal, and interactional.

Increased 1:1 Contact With Students
Although faculty reported increasing their use of asynchronous approaches for inclusivity, many faculty (56 responses) also indicated that it was important to enhance personalized, synchronous interactions with students through increased outreach and office hours. For example, Charles Morton (Chemistry), noted that the “critical move” in his CHEM0330 and CHEM0350 courses (Organic Chemistry) “was redistributing both the professors’ office hours and the TAs’ problem-solving sessions to cover more time zones.” This outreach is helpful for the student experience because increased instructor communication is associated with persistence in online courses (Hart, 2012).

One approach that I've found effective is to increase my one-on-one mentorship with students as they work on final research papers or final media projects and then to integrate their individual works-in-progress into class meetings... Going forward, I will make this kind of collaborative approach even more central to my teaching.

-Finnian Moore-Gerety, Religious Studies

While these meetings often took place via Zoom, some faculty noted that students preferred other modalities for checking in. For example, David Christensen (Philosophy) indicated that some students in his 72-person PHIL0100 (The Place of Persons) course preferred to speak with him by phone. (A Jabber account can be requested from the university, allowing instructors to use their Brown phone numbers for calls to students.)

Change in Assessment: Assignments and Grading Approaches

Many faculty noted that they changed assignment modalities and their approaches to grading (56 responses), and when asked what they would do differently, this was the second most frequent theme (16 responses).  While there was a wide range of approaches to faculty’s rethinking of their course assessment practices, the most frequent change was to allow students more choice and flexibility in their assignments. For example, David Laidlaw (Computer Science) “maintained expectations, but flexibly, [which] kept students engaged and productive.” Ruth Colwill (Cognitive, Linguistic, and Psychological Sciences) offered “a lot of variety and choice in online assignments,” with options such as a FFP (Fun Final Project), which asked students to get creative with course material (e.g., create a game, write a children’s story or Public Service Announcement, create a new course assignment). Opportunities to exercise some choice in assignments helps to heighten student motivation to learn deeply through them (Deci & Ryan, 2008; Pintrich, 2003). 

A number of faculty also modified the type of assignments and assessments, most frequently changing high-stakes exams or large projects into more frequent, lower-stakes quizzes or papers. Indeed, one of the most powerful ways to help students learn is by offering them more opportunities to practice key ideas and skills, with opportunities for feedback (Dunlosky, et al., 2013; National Research Council, 2000; Schrank, 2016). Neuroscience professor Monica Linden used an online portfolio to collect smaller assignments over time and allow students to reflect on their work cumulatively at the end of the term (see textbox below).

I used Google Sites to allow students to create and share an online portfolio of work over the time that we were engaged in remote learning.  It allowed for creativity and flexibility and spaced out the workload.  Students could see each other's work and provide feedback.

-Monica Linden, Neuroscience

Others used contract (e.g., Inoue, 2019) or specifications grading (Nilson, 2015) strategies to adjust assessment approaches. Other modifications to assignments included changing the types of assessments. For example:

• Virginia Krause (French Studies) used a creative media assignment in lieu of an exam.
• Jessica Stair (History of Art and Architecture) asked students to co-curate an online exhibition.
• Jonathan Conant (History) had students create and post artwork about texts instead of written papers.

Because “people learn better when multiple modalities are used” (Bruff, 2020, p. 111), these assignments all exemplify effective ways to promote student learning.

Increased Use of Written Discussions

Use of writing, to prime verbal discussions or in lieu of them, was mentioned by 47 faculty. Multiple faculty reported that the Zoom chat feature allowed for broader engagement, and in-class silent writing can have that affordance as well.

A student proposed "silent meetings" wherein students responded to questions in a shared document before discussing class topics as a group. The written discussions led to in-depth exchanges between students that informed the Zoom group discussions that followed. I found this to be a great way of stimulating engagement -- particularly from shy students -- and hope to use it next term as well.

-Ben Armstrong, International and Public Affairs, Watson Institute

This post offers useful guidance for planning, facilitating, and assessing written discussions. If grading online discussions, clear guidelines, such as a rubric, can be helpful for students to understand expectations. Common criteria include cognitive dimensions (e.g., use of critical thinking and problem-solving capacities), mechanical aspects (e.g., clarity of language and use of citations), procedural considerations (e.g., timeliness of posts), and interactive components (e.g., synthesizing or prompting classmates' posts) (Penny & Murphy, 2009).

Other Changes

While this newsletter describes the most frequently reported instructional changes in Spring 2020, many other types of adaptations were also described. These include use of:
 

• Zoom’s whiteboard feature 
• New tools like Piazza or text annotation tools (like Hypothes.is)
• Collaborative learning strategies like group projects
• Heightened structure or organization, like using Canvas modules or more clearly defined breakout discussion tasks
• Student presentations
• Student-directed approaches, such as having students be discussion leaders
• Increased use of multimedia 
• Bringing the “outside in” through guest speakers
• New approaches to labs

(All of these responses were named by more than 10 faculty each.)

To discuss how these ideas might be implemented in your 2020-21 courses, please contact Digital Learning and Design ([email protected]) or the Sheridan Center ([email protected]). To subscribe to the Sheridan Center newsletter, please use this link.

 

*There were 805 respondents to the faculty survey. 138 respondents reported that they were not teaching in Spring 2020, or wrote “N/A” in the comment box. An additional 121 respondents left the box blank. Nine faculty noted that nothing worked in the spring term. Of the remaining responses, there were 990 distinct ideas that were coded by thematic frequency. In other words, one respondent might have multiple ideas, which were analyzed separately. For questions about the analytical process, please contact [email protected].

 

References

Berry, S. (2019). Teaching to connect: Community-building strategies for the virtual classroom. Online Learning Journal, 23(1): 164-183.

Brown, B., Schroeder, M., & Eaton, S.E. (2016). Designing synchronous online interactions and discussions. In M.A. Takeuchi, A.P. Preciado Babb, & J. Lock (Eds.). Proceedings of the IDEAS: Designing for Innovation, pp. 51-60. Calgary, Canada: University of Calgary. Available: https://files.eric.ed.gov/fulltext/ED573166.pdf

Brown, M. (2019). The push and pull of social gravity: How peer relationships form around an undergraduate science lecture. The Review of Higher Education, 43(2):603-630.

Bruff, D. (2019). Intentional tech: Principles to guide the use of educational technology in college teaching. Morgantown, VA: West Virginia University Press.

Chi, M.T.H., & Wylie, R. (2014). The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist, 49(4): 219-243.

Clark, C., Strudler, N., & Grove, K. (2015). Comparing asynchronous and synchronous video vs. text based discussions in an online teacher education course. Online Learning, 19(3): 48-69.

Costa, K. (2020). 99 tips for creating simple and sustainable educational videos: A guide for online teachers and flipped classes. Sterling, VA: Stylus.

Deci, E. L., & Ryan, R. M. (2008). Self-determination theory: A macrotheory of human motivation, development, and health. Canadian Psychology, 49(3): 182-185.

Dey, E., Burn, H.E., & Gerdes, D. (2009). Bringing the classroom to the web: Effects of using new technologies to capture and deliver lectures. Research in Higher Education, 50: 377-393.

Dunlosky, J., Rawson, K.A.,  Marsh, E.J.,  Nathan, M.J., & Willingham, D.T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest 14(1):4–58.

Eddy, S.L., Brownell, S.E., Thummaphan, P., Lan, M., & Wenderoth, M.P. (2015). Caution, Student experience may vary: Social identities impact a student’s experience in group discussions. CBE - Life Sciences Education, 14: 1-17.

Guo, P., Kim, J. & Rubin, R. (2014, March). How video production affects student engagement: An empirical study of MOOC videos. L@S ‘14: Proceedings of the first ACM conference on Learning @ Scale conference. 41-50. 10.1145/2556325.2566239. 

Hart, C. (2012). Factors associated with student persistence in an online program of study: A review of the literature. Journal of Interactive Online Learning, 11(1): 19-42.

Kinsella, G.K., Mahon, C., & Lillis, S. (2017). Using pre-lecture activities to enhance learner engagement in a large group setting. Active Learning in Higher Education, 18(3): 1-12.

Lowry, P.B., Roberts, T.L., Romano, N.C., Cheney, P.D., & Hightower, R.T. (2006). The impact of group size and social presence on small-group communication: Does computer-mediated communication make a difference? Small Group Research, 37(6): 631-661.

Martin, C.C., Newstetter, W.C., & LeDoux, J.M. (2019). .Inclusion requires a comprehensive understanding of justice. Journal of Engineering Education, 1–6. https://doi.org/10.1002/jee.20296 

National Research Council. (2000). How people learn: Brain, mind, experience, and school (Expanded Edition). Washington, DC: The National Academies Press. Available: https://www.nap.edu/download/9853

Penny, L, & Murphy, E. (2009). Rubrics for designing and evaluating online synchronous discussions. British Journal of Educational Technology, 40(5): 804-820.

Pintrich, P. R. (2003). A motivational science perspective on the role of student learning in teaching and learning contexts. Journal of Educational Psychology, 95(4): 667-686.

Schrank, Z. (2016). An assessment of student perceptions and responses to frequent low-stakes testing in introductory sociology classes. Teaching Sociology, 44(2): 118-127.