Educational Designer

With the goal of expanding case-method teaching via digital learning platforms in effective and engaging ways, we employed design strategies to transform an in-person course into an e-course. The curriculum of both versions of the course was intended to increase skills in developing a non-stigmatizing social marketing campaign for obesity prevention. We pilot-tested the e-course with 37 participants across the U.S. who completed pre- and post-course surveys with quantitative and qualitative components. The survey results indicated that participants’ knowledge significantly improved following the e-course. Participants gave high mean ratings on measures associated with level of engagement, satisfaction with the online format, and effectiveness of the course in increasing targeted skills. Qualitative feedback helped us evaluate strengths and areas for improvement in the course design. Key design strategies included a moderated discussion board; an illustrated, interactive version of the teaching case, which included prompts for reflection using technology available through the learning management system to enable small group work; and video conferencing to simulate face-to-face interaction. The findings of this e-course pilot study can inform future initiatives to deliver case-method teaching via online learning platforms.

Case-method teaching is an inquiry-based instructional design that requires students to imagine themselves in the role of decision-makers facing complex problems inspired by real-world examples. Students read a case narrative, analyze the challenge from the perspective of those impacted, and then work collaboratively to propose solutions to the problem that meet the needs of stakeholder communities. The Strategic Training Initiative for the Prevention of Eating Disorders (STRIPED) based at Boston Children’s Hospital and the Harvard T.H. Chan School of Public Health, designs teaching cases for public health professionals and students. Each case is aligned with U.S. national standards for core competencies in public health education (Council on Linkages Between Academia and Public Health Practice, 2014; Interprofessional Education Collaborative, 2016) and incorporates the principles of student-centered, active learning (Austin & Sonneville, 2013; Handelsman, Miller & Pfund, 2007; Herreid, 2011; Kober, 2015; Mazur, 2009; McLaughlin et al., 2014; Michael, 2006; Trevena, 2007). Active learning techniques require that students engage in learning through activities involving higher-order skills (e.g., creating, analyzing, and applying) rather than lower-order skills (e.g., memorizing facts) (Bloom, Engelhart, Furst, WH & Krathwohl, 1956; Krathwohl, 2002). These techniques have been shown to have comparative advantage over traditional teaching methods such as Socratic discussion and lecture (Handelsman et al., 2007; Herreid, 2011; Kober, 2015), to result in enhanced skill acquisition and longer-term knowledge retention (Dochy, Segers, Van den Bossche & Gijbels, 2003; Lord, 2007; Marantz, Burton & Steiner-Grossman, 2003), and to contribute to greater confidence when applying skills in the field (Dochy et al., 2003; Herreid, 2011; Marantz et al., 2003; Wittrock, 2010).

Traditionally, case-method teaching has occurred in in-person classroom settings, yet recent advances in digital pedagogy and online learning platforms have made it possible to bring this interactive learning experience to wider audiences. Online learning expands access to those who may otherwise lack opportunities in their communities, or for whom regular in-person classroom attendance is infeasible. Simple transfer of classroom teaching methods into an online format is inadequate. Given that collaborative, active learning elements are key to case-method teaching, innovative approaches to online course design are needed to maximize learner engagement.

The design of the Adobe Captivate illustrated, interactive case module drew from Universal Design for Learning (UDL) principles, specifically illustrating content through multiple media and offering points for reflection and self-assessment (Meyer, Rose & Gordon, 2014). In this online version, learners engage with the case multiple times in different formats with the goal of heightening engagement and bringing the case to life in a new way through illustration and interaction. Additionally, learners can step through the interactive case at their own pace, taking the time to consider the self-check questions throughout the case. We hoped that anchoring the activity structure to multiple learner engagements with the case would provide more opportunity for learners to grapple with the case problems so that they could effectively apply lessons learned from the case to their social marketing campaign brief.

Course Development Team: The digital materials and structure of the e-course evolved through a collaborative process of content development and iterative feedback among the course development team members, who contributed expertise in the areas of traditional and digital pedagogy, instructional design for online learning, and continuing education for professional development.

Canvas Learning Management System: The Canvas learning management system (LMS) supports dynamic, media-rich learning experiences. We utilized the LMS tools to present the case in both text and interactive media formats, including embedded video tutorials that directed participants in navigating the online platform, locating course materials, and submitting assignments. The discussion board provided opportunities to share insight and interact directly with the course instructors. We implemented a team-based learning model by dividing students into teams and creating chat rooms in Canvas. To ensure a streamlined user experience, the course content was divided into weekly modules with clearly defined tasks.

Addressing Challenge of Group Cohesion in Online Courses: A key challenge in online courses is establishing and maintaining a sense of connection among learners and between learners and the instructors. To stimulate learner engagement, we reached out to our pilot e-course participants via email to welcome them, provide detailed instructions, and inform participants that instructors were available for questions. The course included a welcome video from the STRIPED director and course instructor. An icebreaker activity encouraged participants to create a personal profile with an optional photograph and an introduction describing their work and reasons for taking the course. We divided participants into teams based on geographical time zone and set up team chat rooms and conference call lines. Broader group interaction on the general course discussion board was required for certain assignments and was actively facilitated by the instructors. Participants were asked to respond to other teams’ final projects to learn from each other and share recommendations. The course instructors provided detailed feedback on assignments via the Canvas LMS platform and actively facilitated the discussion board and corresponded with students via email to maximize depth and breadth of learner comprehension (Ertmer & Koehler, 2015).

Course Components: In designing the online version of the case, we sought to adapt the assignments and preserve the learning objectives, while addressing challenges in online course design and adding a robust course evaluation protocol.

Table 1 outlines the adaptations made to the in-person course structure to transform it into an online format making use of the technological capabilities of the LMS.

In-person Course	Online Course	Technological
Foundational Reading – pre-course reading materials	Introductory video – Week 1 video Introduce Yourself – Week 1 group discussion Foundational video – overview of course content and expectations Foundational Reading – Week 1 reading materials Reading Quiz – Week 1 assignment	LMS* embedded video LMS moderated discussion board LMS embedded video LMS embedded PDF LMS quiz module
Case Study Narrative – pre-course reading	Case Study Narrative – Week 1 reading Interactive Case – Week 2 assignment	LMS embedded PDF Adobe Captivate interactive module: illustrated, version of the teaching case including prompts for reflection
Media Strategy Brief – pre-course assignment	Media Strategy Brief – Week 2 assignment	LMS embedded video LMS assignment
Opening Discussion –activity	Addressing Our Assumptions and Biases – Week 2 assignment Considering All Perspectives – Week 2 discussion	LMS assignment LMS moderated discussion board
Communications Team Meeting –activity	Communications Team Meeting & RFP* – Week 3 assignment	Video and phone conference LMS Collaboration tools LMS assignment
Teams Report Back –presentations & discussion	Teams Report Back – Week 3 discussion	LMS moderated discussion board
Free Write on Ethical Issues –course activity	Discussing the Ethical Issues – Week 2 discussion	LMS moderated discussion board
Whole-Class Discussion & Wrap-Up –	Wrapping Up – Week 3 discussion & assignment	LMS moderated discussion board LMS assignment
N/A	Learning Your Audience – Week 2 assignment	LMS assignment

*Abbreviations: LMS – Learning Management System; RFP – Request for Proposal

Participant Recruitment: To recruit our sample of pilot study participants, we emailed program managers of national LEAH programs. We also contacted project officers of the regional Maternal and Child Health Bureaus, the Massachusetts Department of Public Health, and the Association for Teachers of Maternal and Child Health requesting that they inform their networks about the e-course pilot. Participants could earn twelve Certified in Public Health credits and $65 in gift cards. Interested individuals registered by emailing the STRIPED program manager.

Our pilot study received an exemption from human subjects research review from the Boston Children’s Hospital Committee on Clinical Investigations, as it qualified as research on educational instructional strategies.

Course Evaluation and Feedback Protocol: Participants were emailed a pre-course survey prior to beginning the e-course and the post-course survey online immediately after the e-course. Both surveys included 10 knowledge-based questions to assess the impact of the e-course on learning. Participants were asked to indicate their level of agreement on items representing key domains in the research literature regarding the impact of weight stigma on health and well-being, the links between obesity prevention and eating disorders, and concepts presented in the National Cancer Institute’s publication, Making Health Communications Programs Work (National Cancer Institute, 2002). Ratings could range from 1-7, with a lower number indicating a lower rating of agreement with the statement. Lower ratings of agreement were considered to indicate greater mastery of the course material. However, one question was written so that a higher rating of agreement indicated greater mastery of the material. This question was reverse-coded during analysis. The pre-course survey asked participants to indicate their reasons for taking the e-course. The post-course survey included questions to evaluate participants’ ratings of engagement with fellow participants, instructors, and the course content, e-course technology, content, and design, relevance to their work, and time spent on the course. Elements of the post-course survey were adapted from the Utrecht Work Engagement Scale, which measures the level of satisfaction individuals experience in their work (Schaufeli & Bakker, 2003), and the Web-based Learning Environment Instrument (WEBLEI), designed to capture students' perceptions of web-based learning environments (Chang, 1999, 2001; Chang & Fisher, 2003; Tobin, 1998). Participants also were asked open-text qualitative questions on the post-course survey to elicit feedback on strengths of the course, how it could be improved, and whether and how they plan to implement what they have learned in their professional lives.

Data Analyses: A paired sample t-test was conducted on 10 knowledge-based survey measures administered in the pre- and post-course surveys. Comparative pre-/post-course survey data were analyzed for those participants who completed both surveys. Means and standard deviations on ratings (5- or 7-point Likert scales) for each quantitative measure on the post-course survey were calculated. All qualitative responses were analyzed by two members of the course development team who categorized them into themes. The identified themes provided the basis to suggest improvements to future e-course design and implementation.

Results of Evaluation

A total of 43 participants registered for the e-course. Four individuals subsequently dropped out citing time constraints, and two were lost to follow up. Thus 37 adult public health students and professionals from around the United States actively participated in some or all of the online course components from January 8 through February 1, 2018. Fifteen (40.5%) of the participants completed all of the course requirements. Twenty-nine (78.4%) completed at least 8 of the 11 course components. Eight (21.6%) completed less than 8 of the course components, citing family health issues or unanticipated time constraints due to the extent of the course requirements and/or the challenge of balancing full-time work priorities with outside study.

Participants included learners specializing in nutrition, nursing, physical therapy, and public health. The majority of participants reported little to no experience with developing social marketing campaigns. Participants’ reasons for taking the e-course ranged from personal experience with weight stigma or eating disorders to a desire for professional development by learning ethical approaches to obesity prevention, as well as strategies for designing effective public health-related social marketing campaigns.

Reported familiarity with online courses was broad. Participants’ experience ranged from no online course experience (9/35 = 25.7%), 1-2 courses (10/35 = 28.6%), to 3+ online courses (16/35 = 45.7%), including some who were in the process of or had completed online master’s degree programs.

Thirty-five (94.6%) participants completed both the pre- and post-course surveys. There was a significant difference between pre and post scores on five of the ten knowledge-based survey measures, as shown in Table 2.

Statement	Pre-test means (N=35)	Post-test means (N=35)	t-scores	p-values
Research has shown that one of the most effective approaches to reducing childhood obesity is to focus on individual efforts to lose weight and exercise.	2.91	1.89	3.10	0.004
Studies have found that instilling a certain level of body image dissatisfaction is a good way to motivate people to change their behavior and engage in weight-loss activities.	2.06	1.60	1.40	0.170
Social marketing campaigns to reduce childhood obesity work best when they include clear public health messages that being overweight or obese is bad or unhealthy.	3.03	1.86	3.31	0.002
Obesity and eating disorders should not be addressed concurrently as they have separate and distinct risk factors.	2.51	1.66	2.32	0.026
A person with a negative body image is more likely to be motivated to engage in healthy weight-related behaviors than someone with a positive body image.	2.26	1.63	2.09	0.045
Weight-related teasing is unkind, but it is unclear whether there are any long-term impacts on mental or physical health.	1.89	1.37	1.50	0.143
A person’s body image is something that develops from within; it has little to do with external messages.	1.43	1.37	0.28	0.780
Health-related social marketing campaigns are more or less one-size-fits-all. There is little need to adapt campaigns for different communities and audiences.	1.97	1.40	2.38	0.023
Testing your campaign messages should occur only at the end of the campaign to appropriately evaluate their impact.	1.86	1.80	0.25	0.800
Community-based research and stakeholder buy-in are valuable to ensure that health communications campaigns are accepted, appropriate and effective.b	1.49	1.60	-0.42	0.680

Score range: 1=Strongly Disagree; 7=Strongly Agree; Lower ratings indicate better mastery
^aNational Cancer Institute, 2002
^b This question was reverse coded

Thirty-six participants of the original 43 (83.7%) completed the post-course survey. Ratings of the e-course are presented in Table 3. Mean ratings of participant agreement with the four statements assessing self-reported learner engagement and satisfaction were moderate to high. Participants gave high mean ratings of agreement with five statements associated with level of engagement with the instructors and moderate to high ratings on the eight statements assessing engagement with the technology and fellow participants. There was general consensus that the course was well-organized and the assignments supported the learning objectives, but ratings of agreement were lower on statements that the time commitment for the course was reasonable. The ratings indicated that the e-course content and design were well-received. Participants found the online platform to be reliable but were less likely to agree that the course was flexible enough to fit their schedule. Participants more strongly agreed that an online approach to learning can be a supplement rather than a substitute to a traditional classroom approach. Participants gave high ratings of agreement regarding the value of the e-course in teaching them more about the impact of weight stigma, as well as the e-course’s effectiveness in preparing them to implement a public health-related social marketing campaign strategy. Participants generally agreed that the e-course was worth their time and effort, and that they would recommend the e-course to colleagues/fellow students. Greater than 50% of participants reported spending more than the estimated 16 hours on course work, a finding that was bolstered by the qualitative feedback.

Self-reported Learner Engagement and Satisfaction (adapted from Schaufeli & Bakker, 2003)	1= Strongly disagree; 5= Strongly agree
I was enthusiastic about my studies in this course.	4.28 ± 0.70
My studies in this course inspired me.	4.31 ± 0.86
I felt happy when I was working intensely on this module.	3.75 ± 1.03
I am proud of my work in this module.	3.72 ± 1.00

Engagement With Instructors (adapted from Chang, 1999, 2001; Chang & Fisher, 2003; Tobin, 1998)	1 = Strongly disagree; 7 = Strongly agree
The course instructors were knowledgeable about the course content.	6.56 ± 0.70
The course instructors were accessible.	6.39 ± 0.93
The course instructors responded promptly to my inquiries.	6.50 ± 0.78
The course instructors provided detailed feedback on my assignments.	6.11 ± 1.20
I felt that the course instructors valued my participation in the course.	6.44 ± 1.00

Engagement with Technology and Fellow Participants (adapted from Chang, 1999, 2001; Chang & Fisher, 2003; Tobin, 1998)	1 = Strongly disagree; 7 = Strongly agree
I felt a sense of satisfaction and achievement within this online learning environment.	5.61 ± 1.50
I enjoyed learning in this online environment.	5.69 ± 1.55
The technology resources enhanced learning.	5.69 ± 1.26
I was able to access the materials without much difficulty.	6.14 ± 1.10
I had no difficulty using the technology.	6.06 ± 1.22
It was easy to move about among the different areas of course material.	5.78 ± 1.42
The web-based learning environment held my interest throughout the course.	5.61 ± 1.38
I felt connected with my fellow participants during my course of study.	5.33 ± 1.04

E-course Organization and Expectations (adapated from Chang, 1999, 2001; Chang & Fisher, 2003; Tobin, 1998)	1 = Strongly disagree; 7 = Strongly agree
The learning objectives for the course as whole were clear.	6.08 ± 1.11
The organization of each lesson was easy to follow.	6.22 ± 1.17
Expectations of assignments were clearly stated.	6.19 ± 0.98
Assignments supported the overall learning objectives.	6.50 ± 0.74
I was able to complete all of the assignments during the course.	4.97 ± 1.99
The time commitment for the course requirements was reasonable for this three-week course.	4.56 ± 1.61

E-course Content and Design (adapted from Chang, 1999, 2001; Chang & Fisher, 2003; Tobin, 1998)	1 = Strongly disagree; 7 = Strongly agree
The subject content was appropriate for delivery on the Internet.	6.47 ± 1.00
The presentation of the subject content was clear.	6.42 ± 0.94
The material showed evidence of originality and creativity in the visual design and layout.	6.33 ± 0.99
The multimedia technology (e.g., graphics, sound) contributed to the affective appeal of the material.	6.08 ± 1.18
The links provided in the material were clearly visible and logical.	6.42 ± 0.97
The readings provided were relevant and appropriate to the course.	6.56 ± 0.65

Web-based Format (adapted from Chang, 1999, 2001; Chang & Fisher, 2003; Tobin, 1998)	1 = Strongly disagree; 7 = Strongly agree
The web-based platform was reliable (i.e., no inactive links, technical glitches).	6.67 ± 0.54
This web-based approach to learning the course material can be a substitute for a traditional classroom approach.	4.89 ± 2.01
This web-based approach to learning the course material can be used to supplement a traditional classroom approach.	6.08 ± 1.34
I found the course to be flexible enough to fit my schedule.	5.58 ± 1.66
I found it easy to use the course discussion board.	6.03 ± 1.28
I found it easy to submit the assignments through the course platform.	6.53 ± 0.81

Learning Objectives and Relevance	1 = Strongly disagree; 7 = Strongly agree
This course helped me to learn more about the impact of weight-based stigma.	6.25 ± 0.84
This course helped me learn the steps to plan, implement, and evaluate an effective health-related social marketing campaign strategy.	5.97 ± 1.11
This course was worth my time and effort.	6.08 ± 1.23
I would recommend this course to my professional colleagues/fellow students.	6.03 ± 1.32
I feel equipped to use what I have learned in this course in my work/research.	5.94 ± 1.26

Findings from Qualitative Data

The feedback we received from participants throughout the course and the qualitative data collected in the post-course survey helped us to evaluate areas of strength and areas for improvement in course design, the functionality of the online LMS course platform, course content, workload, and learner engagement. We have summarized the key findings and proposed improvements in Table 4.

Qualitative Feedback from Participants	Proposed Improvements
Online Learning Management System Course Platform and Navigation
“I had a very easy time accessing the course online, submitting assignments and viewing discussion boards.” “The platform was easy to use and I had no trouble following the course and completing all assignments.” “The platform is great! I really liked the short supplementary videos.”	Implemented during pilot of e-course in response to participant feedback: Highlighted and provided easy access to an area of the platform where the syllabus and course timeline were laid out in a straightforward manner. For future course design: Add an interactive calendar with links to assignments/readings/relevant discussion board threads organized by due date. Explore options to use other e-course hosting platforms that offer more creative design options and flexibility. Through mail and phone communication with participants from the beginning of the e-course several participants indicated confusion with getting a sense of the “lay of the land” regarding e-course syllabus and timeline for assignments. We made real-time adjustments to the LMS platform in response to this feedback.

Discussion Board

“I really liked the discussion boards, but they were kind of a lot to keep track of. I felt like I had to keep checking back a lot, and still could not keep up with thoroughly reading everyone's insights (which were valuable and important!).” “The discussion boards were helpful, but it seemed like many people had similar responses phrased in different ways. With a large group I wonder if it would be more beneficial to offer multiple questions to which people could respond instead of one question that everyone responds to?” “Would love for there to be more distinction between "sub-replies" in a thread and "replies"- or notation of whether the reply was to an original poster or a responder to a poster.”

Allowing for threaded conversations, that is, having more distinction between initial “threads” and “sub-threads” with clear linear branching to facilitate dialogue and conversational interaction. Integrating the option to “tag” someone when replying to a person’s comment or providing feedback on an assignment. Limiting conversations to small groups of participants to make the experience more manageable and potentially enrich the discussion. Presenting a few questions to which participants can respond rather than just one to which all must respond.

Course Assignments

“Needed more direction re: final project; more clear expectations of assignment.” “The assignment to write a brief was educational as I now know what to look for in a campaign brief. It would have been helpful to have a sample or two to view when creating my brief, as I was unclear about the planning framework section.” “I have never done a media strategy background brief before. I found it exciting, although I definitely had to learn how to do it. It would have been nice to see an example before completing it.”

Provide examples of assignments with structures that may be unfamiliar (e.g., media brief and final RFP project).

Course Workload Requirements

“The number of assignments and time involved to complete everything in this course was quite a bit more than I expected. Therefore, I regret I was not able to complete all the assignments despite my interest in the topic and enjoyment of the case study.” “For me the module took longer than what was forecasted by the developers. This made it challenging to fit into my already busy schedule.” “This was a lot of work! It is hard to commit 6+ hours per week to something when you are working FT and have a family. This level of commitment took me by surprise.” “The work load was quite heavy and may be better received over a 5-6 week class period.” “Timeframe of course should be extended to allow students to produce quality assignments.”

We had estimated that participants would spend approximately 16 hours on the e-course over the duration of the three weeks. While the data show that the mode time was between 12 to less than16 hours, more than half of the students spent more than 16 hours, and about 37% spent 20 hours or more on the e-course work. Extend the duration of the course to allow for more time to complete the requirements. Make certain assignments optional.

Scheduling

“I would only recommend that groups be divided according to their availability during the day or week, rather than time zones. My other 2 group members were never able to meet at the same time, which I feel really delayed every aspect of us working together as a group.” “It was a bit difficult to schedule our group meeting, as well, due to time constraints and the short duration of the course.” “More call time with groups would have been helpful, especially time to review/edit final product pre-submission.” “Group dynamics always difficult - some do more work than others; divided opinions; lack of face to face communication (video is key).”

Have participants sign up for preferred weekly group call times at registration to streamline logistical challenges of group formation and scheduling. Incorporate video call capability. Schedule weekly call times from course onset.

Group Cohesion

“I think this online course did a great job of connecting everyone participating in the class. I wish there was a way to have a sort of face to face discussion or maybe using a webcam to have a discussion. I thought everyone had really good ideas that I learned a lot from. I would have enjoyed have face to face discussions because I feel it would have sparked more ideas.” “The group project was very challenging for our group and resulted in a mostly individual project. The final project may have been more effective for me if completed individually. I find group collaboration for full-time workers to be fairly overwhelming.” “Would have been helpful to assign one person to be the group facilitator during calls.” “I would have really liked it if there was a way to know my teammates better.” “I truly enjoyed this class so much and cannot believe it is over. I would have liked for it to be longer. It also would have been interesting if an optional web chat was held with the professor and coordinators. I would have liked to feel like I knew them.”

Require teams to develop and sign a team contract to establish expectations and peer evaluation standards. Incorporate a peer-to-peer review to evaluate individual contribution and engagement; make this part of final grade. Incorporate video call capability. Scheduled weekly call times from course onset. Have groups assign call facilitator for each week. Provide a weekly call agenda. Include small-group conversations to allow participants the opportunity to offer reflections or respond to questions posed by the instructor in a more natural, responsive environment. Include an optional synchronous Q&A session with instructors utilizing video conference call technology. Improve the peer review section by assigning groups for each participant to respond to using a specific comment rubric to guide their feedback to ensure that all of the groups receive equal amounts of quality feedback.

Additional Feedback

“I am in the midst of planning a campaign for our county. The information I learned in this module will directly impact how well I plan and how successful the campaign will be. I now see areas that I've missed in my research and plan to include these moving forward.” “I think I will always remember the “No Fat Kids” campaign, and I feel this module really opened my eyes to how much weight stigmatism [sic] is prevalent in today’s society. I will use this knowledge to make sure any campaigns that I work on in the future will include members of the intended community and will promote positive healthy lifestyle changes that everyone in the community can benefit from.” “I plan to share the materials I've saved from this course with colleagues who are working in healthy eating/physical activity work - because there are not a lot of training opportunities, or talking points at the very least, in the chronic disease prevention community re: body image, eating disorders prevention.” “I learned in what ways to be more critical of health campaigns and the role of stigma among various groups of people. I also learned how important evaluation is in the process of creating a campaign.” “Fantastic experience!! The world needed this!! Thank you for the opportunity to participate in this course.”

The innovative learning design of the e-course utilized current media and technology and a transmedia approach to transform the time-tested instructional approach of in-person case-method instruction into an engaging online experience. Our design elevated the learning experience by requiring that learners build upon their knowledge and employ increasingly complex skills to ensure mastery of the course learning objectives. The e-course design process involved a cross-institutional collaboration of faculty and staff from diverse areas of expertise based at the Harvard Graduate School of Education, the Harvard Division of Continuing Education, the Harvard T.H. Chan School of Public Health, and Boston Children’s Hospital, in addition to public health professionals and students from across the country. This effort allowed for interdisciplinary collaboration and provided valuable insights regarding how to effectively use technology to integrate active learning techniques into courses designed for online implementation, how to proactively engage students and maximize group cohesion, and how to design targeted and useful e-course evaluation components. In addition to participants’ feedback, our experience in the field implementing the pilot study demonstrated ways to improve design elements and facilitate participant interaction. Participants signed up for the course based on their own interest in the subject matter and not to receive a grade or to meet any external requirements. We did not charge for this course, which lowers the risk threshold for not completing the course. In fact, we offered a financial incentive for completing elements of the course, as well as CPH credits, though the incentives were small and the CPH credits were useful to only a small number of course participants. Compared to the completion rate for Massive Open Online Courses, which educational researcher Katy Jordan has found to range from 0.7% to 52.1%, with a median value of 12.6% (Jordan, 2015), our completion rate of 40.5% for this smaller, more personalized course is high. Therefore, although we intend to improve on the course design to enable and inspire participants to complete the course requirements, we feel the completion rate for this pilot course was promising.

Participants in this e-course were fairly positive in terms of how they rated their experience with the online course, as well as with group engagement. However, group work can be challenging when conducted with virtual teams of geographically dispersed e-course learners. The discussion board was intended to facilitate virtual interaction, yet some found it difficult to keep up with all of the responses and to differentiate between replies to the original question versus “sub-replies” to others’ comments. Helpful suggestions included establishing clear distinctions between “threads” and “sub-threads” in the conversation to facilitate conversational interaction in the large group and offering multiple questions to which students could reply within their teams, thus encouraging smaller group discussions. Several participants also described difficulties with communication, setting expectations, and achieving equitable workload balance among group members.

When considering the expansion of online instruction in the educational landscape, it was interesting to note that participants were more likely to agree that an online approach to learning can be a supplement for a traditional classroom approach rather than a substitute. Participants in this e-course were not exposed to the in-person version of the course, and thus their ratings of agreement with these statements likely reflect their view as to the differential value of the two formats in general. The possibility that a traditional classroom approach is more likely to be viewed as the primary educational venue could reflect the learners’ perception that in-person instruction and group collaboration have greater inherent impact and worth than online instruction. Comparatively, online courses can offer opportunities for those who cannot access in-person courses on particular topics, for those who require the convenience of more flexible scheduling, or for those who seek to supplement their learning. Our aim for this particular project was not to test the value of one format over the other, but rather to examine whether the active learning design strategies implemented to adapt the in-person course to an online format were effective in achieving sustained participant engagement and ensuring that participants met the learning goals for the course.

Two potentially useful strategies to enhance the quality of – and commitment to – group work in online courses were introduced to us by colleagues at the Harvard T.H. Chan School of Public Health after the completion of our e-course pilot study. The first is to encourage teams to develop and sign a contract to establish the team mission, criteria for success, guidelines for communication, expectations regarding individual contributions, and steps to self-correct when the contract is breached. This recommended activity was adapted from the Field Immersion Experiences for Leadership Development (FIELD Method) (Harvard Business School, https://www.hbs.edu/mba/academic-experience/Pages/the-field-method.aspx). A second strategy is to incorporate a peer review component following each group project that would count toward the final grade. The goal of peer review is to encourage collaborative engagement as peers are asked to rate each other on their contributions and whether each person is accountable to achieving an equitable workload distribution. We will incorporate both of these strategies in future course designs.

There seemed to be a tension between a desire to connect with peers and instructors and challenges of finding time to invest in the e-course. Participants expressed the wish to get to know their peers and the instructors better, but busy schedules, in addition to the course workload, posed a challenge. We will incorporate this feedback into future e-course development by including video conference capability, encouraging smaller group discussions, and offering an optional synchronous Q&A with the instructors. Several participants requested to view examples of key assignments, such as the media brief and the final RFP project, given the formats were unfamiliar and expectations unclear, a suggestion we will implement in a future course design. Participant feedback throughout and following the course made it clear that the coursework was heavier than the participants expected, making it challenging to complete all of the assignments. Therefore, we will consider extending the duration of the course and making some assignments optional.

Our ongoing communications with participants during the course allowed us to learn and improve upon our e-course design in real time, so as to be immediately responsive to their needs. The post-course feedback and evaluation provided additional detail regarding strengths and areas for improvement.

Limitations to this e-course pilot study included the limited sample size of our participant cohort. While the course size was appropriate for this instructional format, multiple pilot tests would allow for more informative evaluation. Several participants who initially signed up for the course did not end up participating due to time constraints. Of those who did participate, not all were able to complete all of the course assignments, which limited their ability to provide feedback. The course evaluation was completed immediately following the period of instruction, and we did not assess whether the observed changes in knowledge persisted over time among course participants. A future iteration of the course in which the proposed revisions summarized above are implemented should also include a longitudinal follow-up to assess knowledge and skill retention. In addition, ongoing study of learner outcomes in pre- and post-course knowledge and assessments of learner engagement will be key to solidifying the efficacy of this approach to online case-method professional education.

We have gathered evidence to support technological curricular innovation via translation of active learning techniques to enhance learner engagement in online educational environments. The design strategies we propose for converting cases from in-person to online application, such as the moderated discussion board; the illustrated and interactive version of the teaching case, which included prompts for reflection; using technology available through the LMS to enable small group discussion and project work; and utilization of video conferencing software to simulate face-to-face interaction, must be further tested to explore whether they are generalizable across a range of courses.

This study was supported by a Harvard Institute for Learning and Teaching Advance grant. Other support was provided by the Adolescent and Young Adult Health Research Network and the Ellen Feldberg Gordon Fund for Eating Disorders Prevention Research. S.B. Austin and H. Gooding were supported by the Leadership Education in Adolescent Health project, Maternal and Child Health Bureau, Health Resources and Services Administration grants T71-MC00009 and T76-MC00001.

Conflict of Interest and Financial Disclosure

No authors have any relevant financial relationships with industry. No authors have any other potential conflicts of interest to report.