Investigating Student Engagement in an Online Mathematics Course through Windows into Teaching and Learning

         Investigating Student Engagement in an Online Mathematics Course through Windows into Teaching and Learning

Teresa Petty
Associate Professor
Department of Middle, Secondary, and K-12 Education
University of North Carolina at Charlotte
Charlotte, NC 28223 USA
tmpetty@uncc.edu

Abiola A. Farinde
Graduate Research Assistant
Department of Middle, Secondary, and K-12 Education
University of North Carolina at Charlotte
Charlotte, NC 28223 USA
afarinde@uncc.edu

Abstract

The Windows into Teaching and Learning (WiTL) project is a method developed by researchers at the University of North Carolina at Charlotte that allows the facilitation of online clinical experiences for students during their content area methods course. WiTL was developed to address difficulties in securing meaningful clinical placements for interns during online summer coursework. WiTL encompasses both an asynchronous and a synchronous component. Through the use of dialogue, the purpose of WiTL is to engage interns with practicing classroom teachers and with each other as they consider various teaching methodologies and observe these methodologies in practice. The authors describe the WiTL process, its implementation, and ways in which the process has encouraged various levels of student engagement in an online mathematics methods course. The results indicate varying levels of student engagement and suggest that students are more engaged during the asynchronous portion of this study.

Keywords: online content methods courses, student engagement, distance learning, asynchronous online learning, synchronous online learning, online clinical experiences

Introduction

Researchers at one southeastern University were experiencing difficulties with clinical placements during online summer coursework. During a summer school session, the challenge to find either year-round schools or schools that have a later end of year so that methods students can be placed in a classroom to observe teaching practices is problematic. In the past, students were allowed to observe the behavior of adolescents in other settings such as summer camps and the Young Men's Christian Association (YMCA). These observations did not provide an ideal situation in which students could observe classroom teachers in their content area delivering instruction. In an effort to deliver meaningful learning experiences and bridge the gap between theory and practice, Windows into Teaching and Learning (WiTL) was conceptualized and implemented. WiTL offered online clinical observation experiences of practicing teachers in various grades and content areas.

The WiTL process utilized web-conferencing software supported by the University. Students were required to have access to a computer, the Internet, a headset, and a webcam to participate in WiTL activities. Students in methods courses observed the practices of teachers both synchronously and asynchronously. Following synchronous classroom observations, which were delivered via Wimba, methods students were given the opportunity to debrief with the practicing teachers, using a text chat feature in Wimba. Dialogue was exchanged regarding the pedagogical practices of the teachers and the rationale for using particular methodologies. Methods students also participated asynchronously by watching prerecorded videos of the practicing teachers. Following these viewings, they participated in an online threaded discussion, via NiceNet, with each other and the practicing teachers.

This research explored the engagement of university graduate students in an online middle grades/secondary mathematics methods course during summer clinical experiences. Student–student interactions (Moore, 1989) were closely examined through an analysis of the asynchronous threaded discussions and synchronous text chat communications using the engagement framework devised by Perkins and Murphy (2006). The examination revealed various levels of engagement among students in both the synchronous and asynchronous platforms. The guiding research question for this study was: To what extent do students in an online mathematics methods course engage in meaningful discourse and collaboration that encourages them to critically examine teacher pedagogy?

Literature Review

Teacher education programs, through the use of technology, offer online courses that seek to enhance student learning (Nandi, Hamilton, Harland, & Warburton, 2011). Of the many technology-based modes of instruction used in online teacher education courses, online discussion forums are often specifically used as a vehicle for the continual discussion of class topics and concepts. In addition, this learning environment, whether synchronous or asynchronous, facilitates the dissemination and acquisition of knowledge and enables student interaction and collaboration (Levine, 2007). Transforming the traditional learning context, these pedagogical tools allow greater access to classroom peers as well as a fluid exchange of content and resources with the intent of improving learning outcomes.

Although online teacher education courses are an innovative and unique pedagogical approach, their effectiveness is often questioned because student interaction in online discussion forums does not necessarily mean that students are actively engaged in the learning process (McLoughlin & Mynard, 2009; Robinson & Hullinger, 2008). In fact, solely fulfilling the specified participation and time requirements of an online course suggests that students are merely "doing time" rather than "doing education" (Zyngier, 2007). In doing time, students fail to go beyond the content, do not bridge theory and practice, and avoid authentic student engagement. Considering that deep learners are often more engaged than surface learners (Hockings, Cooke, Yamashita, McGinty, & Bowl, 2008) and that pre-service teachers' pedagogical training will influence the lives of their future students, engagement is a poignant topic for online teacher education courses.

When reviewing the literature, we found that at the collegiate level the definitions and measurement of online student-to-student engagement were limited. However, the literature did show that engagement is seen through numerous lenses (Zepke & Leach, 2010): student motivation (Schuetz, 2008), students' social and economic background (Pascarella & Terenzini, 2005), institutional support (Kuh et al., 2005), peer interaction (Moran & Gonyea, 2003), teacher behavior (Bryson & Hand, 2007), etc. Although engagement is often difficult to define and measure in online courses, its absence and presence is identifiable. Due to the study's focus on student-to-student engagement, for the purpose of this study engagement is defined by Kuh (2003) as the efforts of the student to study a subject, practice, obtain feedback, analyze, and solve problems. This definition illustrates that engagement cannot be captured through a snapshot but rather has an interpersonal component. Engagement is best observed through interactions with others and through a development of complex ideas. For instance, echoing this definition of engagement, Guthrie and Anderson (1999) state, "social interaction patterns in the classroom can amplify or constrict students' ... attainment of deep conceptual knowledge" (p. 20). Furthermore, within the confines of face-to-face (F2F) schooling, engagement is seen as a multidimensional concept: behavioral, cognitive, and emotional (Fredricks, Blumenfeld, & Paris, 2004). Behavioral engagement measures students' ability to follow school rules, their level of involvement in learning, and participation in extracurricular activity; cognitive engagement is students' investment in and active participation in learning as they move beyond required coursework; and emotional engagement is the positive and negative feelings students hold for school. Aligning – yet moving beyond – the aforementioned definitions, engagement in this study is measured in the context of students' interactions in an online mathematics methods course. Engagement can be identified in this particular learning environment as participants meaningfully contribute to online discussions. Within the asynchronous and synchronous forums, students commented, asked for clarification, posed questions, made inferences, assessed the learning environment, and collectively strategized about how to best bridge content and pedagogy. Their interactions with each other are the focal point of analysis and determine whether student-to-student engagement was present within the online discussions.

One of the challenges of online discussion forums is determining whether quality online engagement is actually occurring or whether students are passively interacting with their peers. In an era where more and more students are learning via online coursework (Murray, Pérez, Geist, & Hedrick, 2012), it is imperative that researchers determine if online learning can equally promote student engagement in comparison to a F2F learning environment. If student engagement is absent or minimized, then full cognitive development in a specific content area is unachievable. Because of this uncertainty regarding online engagement and due to the need to move beyond rote memorization of knowledge, an evaluation of online learning engagement levels must be performed to ensure positive learning outcomes for online students.

Synchronous (occurring in real time) and asynchronous (occurring over time) online discussion forums provide a space for open communication among students. Regardless of person or location, one's thoughts are transmitted to others, constructing new knowledge or further developing existing ideas. While both modes of communication serve a common purpose, asynchronous communication is often deemed more appropriate for facilitating active, meaningful engagement. Recognizing weakness in both communicative resources, Branon and Essex (2001) outline the disadvantages of both types of communication that may consequently hinder student engagement. They list the following restrictions when utilizing synchronous communication: (1) getting students online at the same time; (2) difficulty in moderating large-scale conversations; and (3) lack of reflection time. Similar to synchronous discussions, asynchronous communication equally places limitations on engagement due to the lack of immediate feedback, and the infrequency of students logging in to online discussion forums.

Students' level of engagement during synchronous and asynchronous communication is often challenged because students are not physically present in the same space, supposedly decreasing their opportunity to socially interact, collaborate, give and receive feedback, and render support (Tuckman, 2007). Although both modes of communication possess certain weaknesses, traditional F2F student communication does not guarantee meaningful student-to-student engagement. In fact, bodies in close proximity to one another, occupying a shared classroom, may promote interaction but may fail to facilitate active, engaged learning. Unlike traditional F2F student communication, there are numerous synchronous and asynchronous modes of communication, which are used to promote engagement. The diverse asynchronous (e-mail, listserv, discussion boards, and web logs) and synchronous (chat, instant messaging, and audio and web-based conferencing) modes of communication have the potential to increase interaction and improve online learning environments (Repman, Zinskie, & Carlson, 2005). Through the implementation of effective pedagogical learning principles delivered via online communication, a community of engaged, communicative learners may emerge (Peck, 2012).

As mentioned, although constraints are present in both communication tools, Schellens, Van Keer, and Valcke (2005) affirm that asynchronous discussion boards foster active engagement through a community of learners who teach one another and provide constructive feedback while sharing and gaining information. Im and Lee (2003) advocate the use of asynchronous over synchronous in generating a more effective learning environment. PytlikZillig et al. (2011) also compare synchronous F2F discussions and asynchronous computer-mediated discussions, asserting that computer-mediated discussions produce more effective engagement than F2F discussions.

Moreover, the flexibility and convenience of time, which allows for greater reflection and more collaboration among students, is often cited as a primary reason for why asynchronous discussion forums are a preferred learning resource (Ajayi, 2009; Nandi, Hamilton, & Harland, 2012). While extended time does not necessarily equate to student engagement, the opportunity to engage with others is greater, increasing the likelihood that such engagement may occur. The presence of this engagement, though logical, requires further empirical support. The need for more research on engagement within online forums (Su, Bonk, Magjuka, Liu, & Lee, 2005) reiterates the persistent question of whether students are actively engaged or passively participating. In seeking insight in this matter, we begin with the following research problem: Are students fully engaged in online synchronous and asynchronous courses?

Adding to the literature, this exploratory study examines synchronous and asynchronous student–student interactions (Moore, 1989) in an online middle grades and secondary mathematics methods course for graduate education students. Students' levels of engagement were coded using Perkins and Murphy's (2006) engagement framework. Perkins and Murphy's rubric measures individual engagement in critical thinking in an online asynchronous discussion. While this model was previously used in online asynchronous discussions, answering Perkins and Murphy's call for further research, this study sought additional empirical data by applying their model to asynchronous and synchronous discussions. Perkins and Murphy's model was deemed appropriate and applicable to asynchronous and synchronous modes of communication because different levels of engagement, which were illustrated by students' varying levels of critical-thinking questions and comments, were present in both discussion forums. Identification and measurement of student engagement in critical thinking in both settings revealed that regardless of the online medium, the implementation of sound, student-centered pedagogical practices will foster student engagement. Table 1 provides a detailed classification of the four engagement categories.

Table 1. Perkins and Murphy's (2006) model for identifying engagement in critical thinking

Clarification:
All aspects of stating, clarifying, describing (but not explaining), or defining the issue being discussed.Analyzes, negotiates, or discusses the meaning of the issue.
Identifies one or more underlying assumptions in a statement in the discussion.Identifies relationships among the statements or assumptions.Defines or criticizes the definition of relevant terms. Assessment:
Evaluating some aspect of the debate; making judgments on a situation, proposing evidence for an argument or for links with other issues.Provides or asks for reasons that proffered evidence is valid.Provides or asks for reasons that proffered evidence is relevant.Specifies assessment criteria, such as the credibility of the source. Makes a value judgment on the assessment criteria or a situation or topic.Gives evidence for choice of assessment criteria.Inference:
Showing connections among ideas; drawing appropriate conclusions by deduction or induction, generalizing, explaining (but not describing), and hypothesizing.Deduces relationships among ideas.Strategies:
Proposing, discussing, or evaluating possible actions.Predicts outcomes of proposed actions.Method

Project Description and Participants

The primary goal of WiTL was to provide a meaningful summer clinical experience for students enrolled in an online mathematics methods course. Data were collected from all students (n = 22) enrolled in the online summer mathematics methods course at the researchers' University. Researchers utilized a middle school and a high school based on faculty connections in working within the school and community. The three teachers of middle grades that were invited to participate in this project were identified by their principal as exemplary teachers. All three middle grade levels were represented: one teacher was a sixth-grade mathematics teacher, one a seventh-grade mathematics teacher, and one an eighth-grade mathematics teacher. The three high school teachers that were asked to join were also acknowledged by their principal as exemplary teachers. The three teachers collectively taught Algebra I, Algebra II, Geometry, Pre-Calculus, and Advanced Placement Calculus. Prior to the beginning of the summer methods course, each of the practicing teachers selected two exceptional lessons. These lessons were then videotaped during regular classroom instruction using a laptop, wireless headset, webcam, and TechSmith's Camtasia, a software application used for screen video capture.

The methods students viewed these 12 asynchronous videos (two per teacher) and then participated in an online dialogue regarding the practices they observed. These threaded discussions, facilitated through NiceNet, occurred over two weeks and allowed methods students to engage in conversation with the six practicing mathematics teachers and their peers. The conversations (examples provided in data analysis section) included dialogue between students regarding instructional methods and pedagogical thinking concerning the teaching and learning of mathematics for Grades 6 to 12.

The three middle school and three high school mathematics teachers were also asked to allow online mathematics methods students to participate in a live teaching observation. These synchronous teaching observations were facilitated using a computer connected to the Internet, a webcam, a wireless headset, and Saba Centra (the University-supported web conferencing software). These items were chosen because all online cohort students utilize these technological tools throughout their program coursework. Methods students logged on to Centra at the indicated class time to observe the teachers. They viewed six separate teaching episodes and participated in a text chat dialogue with their classmates during the observations. This viewing and interaction allowed students to ask one another questions regarding the classroom environment, classroom management, instructional methods, and pedagogical thinking.

Following the teaching observation, the classroom teacher joined the methods students in Centra to take comments about their teaching and answer questions regarding instructional decision making. Examples are provided in the Data Analysis section.

WiTL was a multifaceted project, and therefore, generated several data sources. These included individual interviews with each classroom mathematics teacher, focus group interviews with methods students at the conclusion of the semester, copies of the asynchronous threaded discussion forum, text chat logs of methods students conversations during the synchronous teaching observations, and archives of both the synchronous and asynchronous sessions. The data essential to this present study are the asynchronous threaded discussions, facilitated through NiceNet, and the text chat logs, facilitated during the synchronous teaching observations via Centra. The following research question guided the analysis of data:

To what extent do students in an online mathematics methods course engage in meaningful discourse and collaboration that encourage them to critically examine teacher pedagogy?

Data Analysis

The threaded discussions were analyzed to determine various levels of student-to-student engagement during online clinical experiences. Researchers used content analysis to quantify levels of engagement based on students' questions and comments in the threaded discussions. Students' levels of engagement with each other were coded using Perkins and Murphy's (2006) engagement framework presented in Table 1. Various levels of analysis occurred. Initially, the researchers evaluated the data independently, determining which level of engagement was represented in the dialogue. Next, the researchers compared their analyses and determined inconsistencies. Finally, inconsistencies were discussed until final consensus was reached. This method of analysis allowed the researchers to establish inter-rater reliability (Neuendorf, 2002). Frequencies for each category were then tabulated.

The text chat logs, generated during the synchronous teaching observations, were analyzed in a similar manner using content analysis. Again, students' levels of engagement with each other were coded using Perkins and Murphy's (2006) engagement framework (see Table 1). Researchers conducted an independent analysis of the data to decide which level of engagement was represented in the various comments and questions stated during text chat sessions. Researchers then compared their analyses to determine inconsistencies. Once more, inconsistencies were debated until consensus was reached, allowing for the establishment of intercoder reliability (Neuendorf, 2002). Frequencies for each category were then recorded.

Results

After the threaded discussions and text chat logs were analyzed, relative frequencies were tabulated. These data are presented in Table 2.

The data, although not conclusive, indicate varying levels of student engagement across both synchronous and asynchronous platforms. This distribution of student outputs across the engagement categories yields interesting results. The asynchronous forum postings show a higher percentage of Strategy engagement (15.2% vs. 7.9%) as well as a higher percentage of Clarification engagement (61% vs. 45.7%). The synchronous forum postings indicate something different. The categories of Assessment (35.3% vs. 17.5 %) and Inference (11.1% vs. 6.3%) indicate higher levels of engagement for the synchronous platform.

Table 2. Frequency of student output – synchronous vs. asynchronous

Asynchronous
(Threaded Discussion) It is important to note that the threaded discussion forum was available to students and classroom teachers for two weeks, while the text chat logs were offered during six synchronous teaching observations with durations of 75 to 90 minutes each. All four categories of engagement were observed in both the threaded discussions and text chat logs. Students in both settings engaged more at the level of clarification. This was apparent as students asked questions and made comments regarding the classroom context (i.e., configuration of the desks/classroom, decoration of the classroom, technology available to students, academic levels of students, etc.). As the level of engagement increased on Perkins and Murphy's (2006) engagement framework, the relative frequency of students' comments and questions decreased with the exception of the Strategies category. This could be warranted due to the time required for students to reflect on teachers' actions and offer possible justifications. All four categories were observed, but as noted above, engagement cannot be solely measured by frequency (doing time). A critical examination of the actual content presented by students during the threaded discussions and text chat dialogues must also be considered when determining students' engagement level.

The data presents examples of authentic student engagement. For example, the following are distinct comments and statements posted by one student and responded to by another student during the asynchronous online discussion forum (NiceNet). In this exchange students share questions they still have about slope and direct variation and suggestions for teaching systems of equations. This exchange reinforces the notion that students can learn important teaching strategies from one another when they have the opportunity to discuss their ideas. The level of engagement based on Perkins and Murphy's (2006) framework is indicated [in brackets, and rendered in bold].

Student 1: "I really like how you [the teacher] make the association between slope and direct variation because that is a topic that my ninth-grade Algebra I students have a hard time with. Do you have any other suggestions on how to make it clearer? [Strategies] Also why did they change the variable to "k" instead of leaving it as "m"? [Clarification] I am a math major (not an education major) and I still am not sure why the variables in direct variation and slope intercept changed. I know it is a little late in the year for this stuff and you may have done this before, but there is a cool activity to do with solving systems. You break the students in groups and have them do an advertisement for a particular method. I make my students show examples, write a definition, make a logo, a slogan and a valid argument as to why their method is better than all the others. I also make them present it in front of the class. They usually have fun with it. [Strategies]"

Student 2: "I am also a math major and have never understood why the variable changed. I look forward to the teacher's answer! [Clarification] Also, thanks for sharing that method! Systems of equations was the topic I struggled the most with teaching this year and am concentrating on to make it better for next year. I love your project idea and am definitely going to implement that into my classroom! [Assessment]"

The exchange below is also taken from the NiceNet threaded discussion. It exemplifies quality engagement during student-to-student interaction. The students really push each other to think about the topic of fractions and consider various situations in which fractions are used. Both students are making connections, providing examples, going beyond the content, and actively contributing to the discussion.

Student 3: "When in real life do we ever really use fractions? I can think of obvious ones: slices of pizza, talking about discounts and sales tax, but other than those, I'm not sure I really know if we DO use fractions. [Assessment]"

Student 4: "I would argue that we use fractions every time we have to use division. If you look at a class and want to know what percentage of the students did their homework, you are going to have to collect the raw data first. If 16 out of 20 students did their homework, that is a fraction, 16/20. After you know that fraction, you are able divide and determine what that number is as a decimal and percentage. I agree that numbers are not usually left as fractions, and that fractions do not always look as polished as decimals do, but they are something that is used in many people's daily lives. [Assessment]"

Student 3: "Thank you for pushing my thinking. However, if we are using a fraction to solve for a percent on a test, wouldn't we convert it to a decimal to understand it better? Maybe it is the way I think and not the way the world works, but I understand 80% much better than I understand 16/20 even though they represent the same amount. Also, will we ever have to multiply, divide, add, or subtract fractions in real life? I absolutely agree that students need to be able to convert fractions to decimals to percents and vice versa, but will it hurt them to use a calculator to always compute with fractions (as in adding, subtracting, multiplying, and dividing)? I am not sure I know the answer to that question ... [Assessment]"

The following are individual comments made by students to students during the synchronous teaching observations via the text chat feature in Centra. In this exchange, students are discussing the note taking methods used by the teacher they are observing. This exchange reveals the ideas that the students have regarding this technique. The level of engagement based on Perkins and Murphy's (2006) framework is indicated.

Student 1: "How structured do you think note taking should be? Right now, they're basically writing exactly what he dictates. [Clarification]"

Student 2: "I think it will depend on my students ... may have to feel it out ... [Assessment]"

Student 3: "I think that note taking is a skill that has to be explicitly taught. [Assessment]"

Student 4: "I agree that it has to be taught, but you can't learn if you always are just copying something ... [Assessment] Maybe this is a good method for younger students who are still in the beginning stages, though. [Inference]"

The exchange below is also taken from the text chat log. During this exchange, students are observing a teacher-centered classroom. The teacher is initiating traditional instructional methods. The students are offering suggestions for getting the middle school students motivated as many of them are visibly disengaged.

Student 1: "Maybe some cooperative learning activities and more game type learning would motivate them a little more. Make the students think they're playing instead of learning. [Strategies]"

Student 2: "They will think it is a reward for passing the test. [Strategies]"

Student 3: "The teacher should consider maybe not teaching new information but readdressing what they did not get. [Strategies]"

The comments/questions presented above demonstrate the various levels of engagement that students exhibited as they participated in both the threaded discussions and the text chat dialogues. Students discuss a variety of issues that they see as they observe teaching practices via synchronous and asynchronous observations.

Limitations

As with most research studies, this study does have its limitations. The most apparent limitation is the small sample size, making the results difficult to generalize. The study was conducted over a short period of time – one semester during the summer. In the course of this study, there were technical difficulties. During two of the synchronous teaching observations, the teachers had difficulty with their wireless headsets, which resulted in periods of no sound for the online methods students who were viewing the teaching observations. This issue was quickly resolved. Another possible limitation relates to the asynchronous threaded discussion. Since this is a monitored forum, students may have felt more inclined to participate and to provide thoughtful responses.

The use of the Perkins and Murphy (2006) framework could also be viewed as a limitation. It was first developed for use with the asynchronous platform. This study took it a step further and utilized it with both asynchronous and synchronous platforms.

Conclusion

In this study, the researchers explored the guiding research question: To what extent do students in an online mathematics methods course engage in meaningful discourse and collaboration that encourage them to critically examine teacher pedagogy? It was determined that students engage in all levels of engagement as defined by Perkins and Murphy (2006) in both asynchronous and synchronous platforms. While asynchronous forum postings show a higher percentage of Strategy engagement (15.2% vs. 7.9%) as well as a higher percentage of Clarification engagement (61% vs. 45.7%), the synchronous forum postings indicate something different. The categories of Assessment engagement (35.3% vs. 17.5 %) and Inference engagement (11.1% vs. 6.3%) presented higher levels of engagement for the synchronous platform. The settings of the individual platforms would certainly encourage this. In the asynchronous environment, students would have more time to reflect and therefore offer strategies. Clarification engagement may be more present in the asynchronous platform because students in the synchronous platform were afforded opportunities to get clarification to these questions during the synchronous settings that were not readily available in the asynchronous setting. This notable constraint does differ from the literature, which asserts that asynchronous is preferable. The results of the WiTL study suggest that the asynchronous and synchronous platforms are emphasizing different types of engagement. Future research studies are needed to determine which method is more beneficial, or if one method is better at engaging students in an online environment.

WiTL proved to be a beneficial experience for methods students. It offered outreach to students across the state through an innovative approach to clinical experiences. WiTL provided the online methods students not only the opportunity to view sound teaching practices, it also allowed them the chance to engage with practicing mathematics teachers to confirm their understandings of methodologies and pedagogical decision making. During a typical clinical observation experience, a student would be placed with one teacher and one classroom. In this online mathematics methods course, students were placed in different classrooms across a variety of schools. WiTL was a unique experience in that it allowed online methods students to observe six different teachers in two different schools and school systems, exposing them to an assortment of teaching styles in a variety of classroom environments. Since WiTL allowed the online methods students to observe the same teachers, an exchange of thoughts and ideas related to the instructional strategies they observed was also facilitated. The WiTL process was equally valuable to the practicing mathematics teachers as it allowed them the opportunity to critically reflect on their teaching practices as they considered questions that the methods students posed. Through WiTL, methods students were able to utilize technology to engage in online clinical experiences at various levels with their peers.

References

Ajayi, L. (2009). An exploration of pre-service teachers' perceptions of learning to teach while using asynchronous discussion board. Educational Technology & Society, 12(2), 86-100. Retrieved from http://www.ifets.info/journals/12_2/7.pdf

Branon, R. F., & Essex, C. (2001). Synchronous and asynchronous communication tools in distance education. TechTrends, 45(1), 36-42. doi:10.1007/BF02763377

Bryson, C., & Hand, L. (2007). The role of engagement in inspiring teaching and learning. Innovations in Education and Teaching International, 44(4), 349-362. doi:10.1080/14703290701602748

Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research. 74(1), 59-109. doi:10.3102/00346543074001059

Guthrie, J. T., & Anderson, E. (1999). Engagement in reading: Processes of motivated, strategic, knowledgeable, social readers. In J. T. Guthrie & D. E. Alvermann (Eds.), Engaged reading: Processes, practices, and policy implications (pp. 17-45). New York, NY: Teachers College Press.

Hockings, C., Cooke, S., Yamashita, H., McGinty, S., & Bowl, M. (2008). Switched off? A study of disengagement among computing students at two universities. Research Papers in Education, 23(2), 191-201. doi:10.1080/02671520802048729

Im, Y., & Lee, O. (2003). Pedagogical implications of online discussion for preservice teacher training. Journal of Research on Technology in Education, 36(2), 155-170.

Kuh, G. D. (2003). What we're learning about student engagement from NSSE: Benchmarks for effective educational practices. Change: The Magazine of Higher Learning, 35(2), 24-32. doi:10.1080/00091380309604090

Kuh. G. D., Kinzie, J., Schuh, J. H., & Whitt, E. J., & Associates. (2005). Student success in college: Creating conditions that matter. San Francisco, CA: Jossey-Bass.

Levine, S. (2007). The online discussion board. New Directions for Adult and Continuing Education, 113, 67-74. doi:10.1002/ace.248

McLoughlin, D., & Mynard, J. (2009). An analysis of higher order thinking in online discussions. Innovations in Education and Teaching International, 46(2), 147-160. doi:10.1080/14703290902843778

Moore, M. G. (1989). Three types of interaction. American Journal of Distance Education, 3(2), 1-6. doi:10.1080/08923648909526659

Moran, E. T., & Gonyea, T. (2003). The influence of academically-focused peer interaction on college students' development. Available from ERIC database. (ED478773)

Murray, M., Pérez, J., Geist, D., & Hedrick, A. (2012). Student interaction with online course content: Build it and they might come. Journal of Information Technology Education: Research, 11, 125-140. Retrieved from http://www.jite.org/documents/Vol11/JITEv11p125-140Murray1095.pdf

Nandi, D., Hamilton, M., & Harland, J. (2012). Evaluating the quality of interaction in asynchronous discussion forums in fully online courses. Distance Education, 33(1), 5-30. doi:10.1080/01587919.2012.667957

Nandi, D., Hamilton, M., Harland, J., & Warburton, G. (2011). How active are students in online discussion forums. In J. Hamer & M. de Raadt (Eds.), Proceedings of the Thirteenth Australasian Computing Education Conference: Vol. 114 Conferences in Research and Practice in Information Technology (pp. 125-134). Darlinghurst, Australia: Australian Computer Society. Retrieved from http://www.crpit.com/confpapers/CRPITV114Nandi.pdf

Neuendorf, K. A. (2002). The content analysis guidebook. Thousand Oaks, CA: Sage.

Pascarella, E. T., & Terenzini, P. T. (2005). How college affects students: A third decade of research. San Francisco, CA: Jossey-Bass.

Peck, J. J. (2012). Keeping it social: Engaging students online and in class. Asian Social Science, 8(14), 81-90. doi:10.5539/ass.v8n14p81

Perkins, C., & Murphy, E. (2006). Identifying and measuring individual engagement in critical thinking in online discussions: An exploratory case study. Educational Technology & Society, 9(1), 298-307. Retrieved from http://www.ifets.info/journals/9_1/24.pdf

PytlikZillig, L. M., Horn, C. A., Bruning, R., Bell, S., Liu, X., Siwatu, K. O., ... Carlson, D. (2011). Face-to-face versus computer-mediated discussion of teaching cases: Impacts on preservice teachers' engagement, critical analyses, and self-efficacy. Contemporary Educational Psychology, 36(4), 302-312. doi:10.1016/j.cedpsych.2011.07.002

Repman, J., Zinskie, C., & Carlson, R. D. (2005). Effective use of CMC tools in interactive online learning. Computers in the Schools, 22(1-2), 57-69. doi:10.1300/J025v22n01_06

Robinson, C. C., & Hullinger, H. (2008). New benchmarks in higher education: Student engagement in online learning. Journal of Education for Business, 84(2), 101-109. doi:10.3200/JOEB.84.2.101-109

Schellens, T., Van Keer, H., & Valcke, M. (2005). The impact of role assignment on knowledge construction in asynchronous discussion groups: A multilevel analysis. Small Group Research, 36(6), 704–745. doi:10.1177/1046496405281771

Schuetz, P. (2008). A theory-driven model of community college student engagement. Community College Journal of Research and Practice, 32(4-6), 305-324. doi:10.1080/10668920701884349

Su, B., Bonk, C. J., Magjuka, R. J., Liu, X., & Lee, S.-H. (2005). The importance of interaction in web-based education: A program-level case study of online MBA courses. Journal of Interactive Online Learning, 4(1), 1-19. Retrieved from http://www.ncolr.org/jiol/issues/pdf/4.1.1.pdf

Tuckman, B. W. (2007). The effect of motivational scaffolding on procrastinators' distance learning outcomes. Computers & Education, 49(2), 414-422. doi:10.1016/j.compedu.2005.10.002

Zepke, N., & Leach, L. (2010). Improving student engagement: Ten proposals for action. Active Learning in Higher Education, 11(3), 167-77. doi:10.1177/1469787410379680

Zyngier, D. (2007). (Re)conceptualising student engagement: Doing education not doing time. Teaching and Teacher Education, 24(7), 1765-1776. doi:10.1016/j.tate.2007.09.004

View the original article here

Online Student Support Services: A Case Based on Quality Frameworks

?

         Online Student Support Services:
A Case Based on Quality Frameworks

Barbara L. Stewart
Professor of Retailing and Consumer Science
Department of Human Development and Consumer Sciences
University of Houston
Houston, TX 77004 USA
bstewart@uh.edu

Carole E. Goodson
Professor of Technology
Department of Human Development and Consumer Sciences
University of Houston
Houston, TX 77004 USA
cgoodson@uh.edu

Susan L. Miertschin
Associate Professor of Information Systems Technology
Department of Information and Logistics Technology
University of Houston
Houston, TX 77004 USA
smiertsch@uh.edu

Marcella L. Norwood
Department Chair and Associate Professor of Retailing and Consumer Science
Department of Human Development and Consumer Sciences
University of Houston
Houston, TX 77004 USA
mnorwood@uh.edu

Shirley Ezell
Associate Professor of Retailing and Consumer Science
Department of Human Development and Consumer Sciences
University of Houston
Houston, TX 77004 USA
sezell@uh.edu

Abstract

Expansion of online teaching and learning has fostered the creation of numerous standards and frameworks to evaluate and promote quality in online instruction and learning. Review of these standards showed that there is consistency in indicators of quality for online programs. One indicator, student support, was selected for study. Quality frameworks were reviewed to determine if student support was included among the indicators. Then, to examine the inclusion of student support services within a real-world context, a case was selected for observation and illustration. Examination of the case showed that faculty and administrators had designed and were delivering critical student support services. Both new and existing support systems were needed and provided. This case indicated a primary need and provisions for student support in the following areas: admissions and registration, advising, orientation, learning support, scholarships and awards, library resources, computing and technology resources, articulation with other institutions, career placement, and communication.

Keywords: student services, online services, distance education services, quality standards, supporting student success

Introduction

Program and course evaluations provide a foundation for appropriate review and assessment of online educational pursuits. Expansion of online teaching and learning has fostered the creation of numerous standards, guides, and frameworks to evaluate and promote quality in online offerings. A selection of these frameworks provides indicators of the aspects of online delivery that are important for course quality.

While not always clearly differentiated, quality standards or frameworks can be applied to both programs and courses. Whether they are formatted into principles (Western Cooperative for Educational Telecommunications [WCET], 2001), scales (Walker & Fraser, 2005), categories (National Institute of Standards and Technology [NIST], 2009), clauses (Canadian Standards Association, 2009), benchmarks (Phipps & Merisotis, 2000), dimensions (Jung, 2011, 2012), elements (MarylandOnline, 2008), standards (MarylandOnline, 2008), or values (International Center for Academic Integrity, 1999), each can be used to capture and share a vision of the elements of online education that are critical to quality.

This paper is based on the authors' perceptions and experience that support is important for student success. Quality frameworks were studied to find whether support factors were included. A case was investigated to observe the application of student support services. Schematically, this paper proceeds in the following sequence: examination of quality frameworks, identification of student support, and case observation.

Quality Frameworks for Online Learning

Representative standards focus predominately on the evaluation of online programs. They include the following:

The Sloan Consortium's Quality Framework (Moore, 2005);WCET's Best Practices for Electronically Offered Degree and Certificate Programs (WCET, 2001);Distance Education Learning Environments Survey (Walker & Fraser, 2005);Quality Matters (QM) (MarylandOnline, 2008);American Distance Education Consortium (ADEC) Guiding Principles for Distance Teaching and Learning (ADEC, 2003);Quality Improvement Framework (Inglis, Ling, & Joosten, 2002);Institute for Higher Education Policy (IHEP) Benchmarks for Success in Internet-Based Distance Education (Phipps & Merisotis, 2000);Universitas 21 Global Quality Framework (Chua & Lam, 2007);Australasian Council on Open, Distance and E-learning (ACODE) Benchmarks (ACODE, 2010);Proactive Evaluation Framework (Sims, Dobb, & Hand, 2002);Quality Preference Framework (Ehlers, 2004);Model for Quality in Distance Education (Jung, 2012).Review of these standards by multiple authors has shown that there is consistency in indicators of quality components for online programs (Goodson, Miertschin, & Stewart, 2012; Inglis, 2008; Jung, 2011; Parker, 2008; Wang, 2006). Parker states, "Although the regulatory frameworks for quality assurance vary dramatically ... there is still enough common ground to establish some general characteristics for a scholarly approach to online teaching and learning" (p. 305). With this in mind, review of these quality standards, guides, and frameworks revealed great commonality. Specifically, student support was selected for study because the authors' experiences suggested it was important and because preliminary evidence in the literature indicated that support was perceived as an indicator of quality. Student support was consistently listed among the many important components of quality. Student support systems were found to be linked to quality factors in over half of the frameworks reviewed. For the frameworks that did not list student support services specifically, in nearly all cases, terminology either suggested or subsumed student support. For example, the 2009-2010 Malcolm Baldrige National Quality Award Education Criteria for Performance Excellence defined by NIST (2009) employ the term "customer focus," which implies inclusion of elements of student support. Examples of student support terms included "student support" (ACODE, 2010; McKinnon, Walker, & Davis, 2000; Phipps & Merisotis, 2000; Sims et al., 2002; Wang, 2006; WCET, 2001), "learner support" (MarylandOnline, 2008), "supporting the needs of learners" (Inglis et al., 2002), "tutor support" (Ehlers, 2004), "effective student services" (Goodson, Faulkenberry, Miertschin, & Stewart, 2004), and "support" (Phipps & Merisotis, 2000; Simelane, 2009).

Hence, the review of quality frameworks showed strong enough value for student services as a component of quality that the researchers judged the provision of these services as appropriate for further investigation. The excerpts from the literature that follow demonstrate both the perceived value shown for student support services and the position of student support among other indicators of quality.

Jung (2011) studied Korean students' perceptions of e-learning quality and included various aspects – technical, psychological, social, administrative, and complaint – of student support as one of her seven initial dimensions. By 2012, in a more comprehensive study of Korean and other Asian learners, she expanded her dimensions to 10, categorized these dimensions into three domains, and included the student support dimension within a domain she labels "supportive domain" (Jung, 2012, p. 5).

Three broadly recognized quality frameworks provide further illustration. The QM Program provides for assessment of 40 elements distributed across eight broad standards (MarylandOnline, 2008). "Learner support" and "accessibility" are two of the eight broad standards. IHEP (2010), striving to influence public policy on postsecondary education, published 24 benchmark criteria. The benchmarks are organized into seven standards. "Student support" is one of the seven standards. Similarly, the WCET (2001) published best practices including 27 principles across 51 activities. The five institutional level activities include "student support."

Methods

The literature review was designed to verify whether or not student support was included as an indicator of quality of online instruction. Since student support was found to be included as a factor of quality consistently in research studies and frameworks, it was retained as the focus of this investigation. Then, a method was sought to view the inclusion of student support and its impact on quality in a real-life context.

Operationally, student services were defined as the academic, administrative, social, and psychological policies and practices to enable and facilitate student success.

Case study as a qualitative empirical methodology was selected. Yin's (1994) recommendation of the case study method as an investigation of contemporary phenomenon within real-life contexts formed a foundational element of this study. This study was further undergirded by Lofland and Lofland's (1995) methodological recommendations to seek patterns in frequencies, magnitudes, structures, processes, causes, and consequences. From this list, this study emphasized Lofland and Lofland's recommendations to include frequencies, structures, and processes in investigations.

Yin (1994) proposes the formation of propositions from a research question as a link to what is examined in a study. He further suggests that these propositions define the units of analysis. This process guided the selection of a single case with which the researchers were very familiar and had access to germane details to answer the following research question:

What types of student support are needed and available to online students?

From this research question, following Yin's recommendation, four propositions were drawn:

Support services are available to online students.Existing support services can be used by online students.New support services are available for online students.Access to support services is needed by online students.Hence, as Yin (1994) recommends, these propositions guided investigation of the case as a study of student support in a real-world contextual online setting.

The researchers recognized the need in this study to foster validity and followed Yin's (1994) charge to use a prescribed protocol as a means of establishing reliability and validity for case studies. For this study, the protocol included: (1) formulating an overview of the project including objectives and readings; (2) evaluating the readings for the presence or absence of references to student services; (3) providing a sequence for the observation of student support services; and (4) adhering to the protocol in preparing the final report as outlined by Yin. Following Lofland and Lofland's (1995) orientations, both objective case details and the subjective observations of the researchers were recorded, tabulated, and categorized.

The following illustrative case is the basis for the discussion of supporting online students. While future research would benefit from the inclusion of additional cases, this work reflects Eisenhardt's (1989) value for theoretical rather than statistical sampling in case selection. The authors, in the roles of researchers-as-participants, were in ideal positions to review and reflect upon student support. Pettigrew (1990), for example, suggests that when the number of cases that could be studied is substantial, it makes sense to select cases for their particular value. Hence, this case was selected because of the experience of the authors.

More generally, case methodology was selected because of its capability to investigate contemporary phenomena in real-life contexts (Yin, 1994). Online education certainly is both contemporary and exists within a real-life context. Soy (1997) suggests that in addition to producing, disputing, and building upon theory, case study research design is useful in describing, explaining, and applying and evaluating solutions to objects, phenomena, and situations.

Identification of student services, both needed and offered, was enabled through student, faculty, and staff interviews as well as examination of University websites, catalogues, student orientation materials, and directories. Tracking and documenting student outcomes was facilitated by the familiarity of the researchers with the case due to their roles of researchers-as-participants. Tools used to observe student outcomes included course summative and formative evaluations, the college-wide instructional effectiveness instrument, online discussion boards and chat sessions, live and online conversations, feedback from a professional advisory board, a student survey of critical factors for online success, and elements of an intensive accreditation self-study and site visit process.

Case Description

The students of the Bachelor of Science in Retailing and Consumer Science (RCS) degree program at the University of Houston (UH) are part of a large urban institution located in the middle of an expansive metropolitan region. The program is well established, having been offered for more than 30 years. Ninety percent of the student population lives and works off campus. Specifically, more than 90% of RCS majors are employed, with the majority working full time. Commuting times to campus can be challenging. Increasingly, students who work and live in an electronic environment raise expectations and opportunities for the use of technology in their educational experiences. At the same time, faculty and administrators extend their knowledge and skills to applying technology to learning environments. Evidence of student use of technology provided early encouragement for the development of online classes for this program. Additionally, a statewide initiative toward a common general education core across public institutions of higher learning provided opportunity. Since students could complete their general education core in multiple locations and online, the RCS faculty could focus on developing major courses in an online format.

The first online course in the program was offered in 1997. By Fall 2002, six of the 12 courses in the RCS major were available to students online. Shortly after Spring 2003, all courses were taught online (Stewart, Norwood, Ezell, & Waight, 2006).

Case Outcomes

It is from this vantage point that this case offers experience that may be beneficial in considering best practices. Consistent cycles of assessment, change, and reassessment resulted in the emergence of multiple issues. Among the critical issues was the need for numerous and varied types of support for students.

The RCS faculty members, in their roles of researchers-as-participants, not only determined student needs and designed and executed systems to meet the support needs of students, but they also recorded and catalogued the systems and evaluated their impact for students. This process provided a record of student support initiatives and outcomes, which is described herein.

The following section contains a listing and description of the support services for students used in this case study. While it is somewhat lengthy, it is shared both to illustrate the scope of services needed and to demonstrate that in this case the collaborative efforts of university administrators with faculty as well as college, department, and program leaders provided the expansive range of services needed by online students.

Types of Student Support

Objective and subjective observations resulted in the identification of existing and needed student services. An analysis of these services enabled the creation of categories for them. In addition, the four propositions based on Yin (1994) outlined earlier were each confirmed. Furthermore, this case indicated a primary need for student support in the following areas::

admissions and registration;academic advising;orientations to the University and to online learning;academic support services;scholarships and awards;library resources;computing and technology resources;articulation and transfer from other institutions;career placement.Needs for communication and connectivity were found to be pervasive throughout the support systems.

Finally, from observations and analysis, categories of services emerged. These services were further categorized by whether they were provided at the (1) course; (2) Department/College; or (3) University levels. The resulting structure is presented in Figure 1.

Figure 1. Categories of student support services emerging from observations and analysis

Descriptions of Student Support and Student Support Outcomes

In some cases, systems or services were designed and implemented specifically to support online students. In other cases, accommodations were made to existing mainstream services to facilitate access for online students.

Course design elements that support students. The faculty involved in this case supported students through course design; they developed course elements and features to enable student success. Course structure and content delivery were the two major areas most carefully considered.

Course structure issues included: (1) systematic implementation of a learning management system (LMS); and (2) student information systems that were easy to access, easy to navigate, and easy to use without technical difficulty. To support course and content access, orientation materials and syllabi were created to provide students with a clear understanding of course objectives, materials, expectations, and assessments. These materials included information about objectives, content, contacts, texts and materials, technical requirements, policies and procedures, assignments and exams, grading, help, calendars, resources, and course features.

Course content organization within the online delivery format was the second major area of course support for students. For most courses, content modules were developed to facilitate delivery of content and to break the material into approachable units. Each module was set up so that students could easily access module objectives; content outlines and terms; content presentations in PowerPoint, streamed video, or other forms; module assignments; and module assessments. Students could easily "click through" the module sequence and resources to engage in the learning process. Enrichment links were also provided to enhance the learning experience.

Course design elements outcomes. Consistently, student reports from the multiple outcome measures indicated that well-designed course structures and delivery modes are important support components and are beneficial to student success.

Specifically, student success was enabled by the incorporation of a user-friendly course learning platform. In this case, the University adopted a standard commercial LMS to provide a uniform course format. The Department also employed a technical specialist to support faculty development of courses in a consistent format within the LMS. Students expressed appreciation for the consistency provided by a common format. A familiar working environment across courses freed them to focus on the course content rather than the operational system.

Paramount in students' perceptions of support through course design was appreciation of clear instructions. These facilitated easy access to course components, navigation within course sites, and use of instructional materials without technical difficulties. Course specific orientation materials and syllabi that clearly delineated course processes, expectations, and resources were highly valued.

Students responded positively when course delivery methods facilitated their success. Content modules, used to break the course into approachable units, provided students with the ability to progress through the course a step at a time, yet also allowed them to see the big picture of the course in its entirety. Completion of one unit at a time, for example, was reported as providing a sense of satisfaction and progression.

Department and College Support Services for Students and Outcomes

While course level design systems and features, created primarily by the instructors, were imperative for student success, additional needed services were provided by the Department and the College. These were in the areas of learning support, student organizations, academic advising, and instructional design. In some cases, divisions between Departmental/College services and University services were blurred since each level may have provided service.

Instructional Support Services Center. The primary mechanism for student learning support beyond the course level, yet within the Department, was the Instructional Support Services Center (ISS) Lab. The ISS Lab offered many services to aid with student learning, achievement, progression, and retention. Over the time period spanned by the case, these services evolved to accommodate the increase in online offerings. For example, individual course tutoring was initially offered on campus through the ISS Lab. As need arose, and over a two- to three-year period of iterative design–implement–evaluate cycles, online tutoring was incorporated (Miertschin, Goodson, & Schroeder, 2010). Currently, the ISS Lab offers the following student services: course software assistance;individual course tutoring (both face-to-face and online);Graduate Record Examinations (GRE) preparation;supplementary course materials;a meeting place for students working on group assignments;a worksite for teaching assistants to support online courses;distribution of student course materials;collection of student assignments, as requested by the instructor;provision for student supplies;proctoring of flexibly scheduled exams or makeup exams.The ISS Lab also provides faculty support for online courses.Student organizations. A professional student organization, Collegiate DECA (previously known as Delta Epsilon Chi and Distributive Education Clubs of America), supported students through membership and leadership opportunities. In addition to establishing connections to one another, students developed channels to the faculty, campus, and community. Student members engaged in diverse activities including sponsorship of speakers, student discussions, service projects, fundraising activities, career days, and campus events. In addition there were outreach opportunities where students applied professional skills to judge and organize local and regional Collegiate DECA events for high school students. Students judged competitive events in the areas of marketing, merchandising, and retailing.

In addition to the departmentally sponsored student organization, faculty members encouraged student participation in University-wide organizations and activities. This study found that affiliations with department and campus student organizations lessened the isolation that is sometimes experienced by online students.

Academic Services Center. Counseling through an Academic Services Center (ASC) provided support with academic counseling for students. Online students were served by academic advisors via e-mail, telephone, or on-campus visits. ASC advisors oriented students to academic processes and regulations, evaluated transfer credits, created degree plans, communicated with students regarding events and deadlines, monitored academic progress, provided referrals to other campus services, facilitated career planning, and evaluated readiness for graduation. Course design and technical support. The following areas of support for online instruction were targeted for faculty but are included here because a well-supported faculty is better equipped to help students: Course design. In addition to instructional design services provided by the University, the Department, and the College employed instructional design support staff members. These individuals trained faculty in the use of course software and online communication tools, trained and monitored graduate students who assisted with course design, and provided direct design services. Technical services. The Department and College employed technical support personnel to support the technology needs of faculty and staff. Their responsibilities included assisting faculty and staff with hardware and software needs and maintenance of computer labs for student use.Department and College Student Support Service Outcomes

Outcomes evaluation of support for students as provided by Department and College initiatives, including learning support, student organizations, academic advising, and instructional design, yielded positive effects for the students.

Learning support. Logs of student services, maintained by the ISS Lab, recorded individual contacts with students on a daily basis. Analysis of the logs provided evidence of extensive student use and problem resolution. Students used the ISS Lab as both a physical place of contact and as an online resource for assistance with challenges related to course content and software. Students used tutoring services in both face-to-face and online formats, which together met student needs for course support and flexibility. Mathematics tutoring and supplemental materials for courses were seen as particularly beneficial. For some online students, having a physical location on campus where they could receive face-to-face help was important. The ISS Lab supported students by providing a single highly available location where they could flexibly pick up or submit course materials and complete exams in a proctored setting. Flexible scheduling of exams and on-demand assistance accommodated students' busy lifestyles. Student organizations. Students reported multiple benefits from participation in Collegiate DECA. DECA activities generated excitement and connection, as evidenced by conversations in online forums and e-mails to faculty. Faculty members stated that they got to know students in a deeper way through connections made through DECA-sponsored events. Speakers' forums organized by DECA provided extracurricular, relevant content and triggered valuable links to the profession. Judging high school competitions and organizing fundraisers allowed students to apply and extend classroom knowledge, practice professional communication skills, and make social connections to peers. Affiliation with DECA reinforced a sense of community, especially for those studying remotely. These findings are consistent with Tinto's (1975) suggestion that social integration is important in student retention. Academic advising. As the site for academic advising, the ASC was used by almost all students in this case. ASC rosters and logs recorded student contact via e-mail, telephone, and on-campus appointments. Logs indicated that students benefited from support, having received advising about academic processes and regulations, evaluation of transfer credits, course sequencing, degree plans, and evaluation of progress toward graduation. Additionally, the ASC regularly made referrals to other campus services. Instructional design. Students responded favorably that online courses were well designed. Software training, professional course design support, and training of graduate students used by instructors to create effective courses yielded positive student outcomes. Students felt course structures were easy to navigate and content was readily accessible. Technical support. Technical support for faculty, including training, procurement services, applications, and troubleshooting of hardware and software, supported students by enhancing the capability of faculty members to teach. Added technical support in the form of assisted computer and learning laboratories for students, created an important additional direct support mechanism. University Support Services for Students and Outcomes

Online students benefited from numerous services provided by the University as well. Some of these services were designed and directed specifically for online students, while others were available to all potential and enrolled students. Existing services, in some cases, required careful attention in order to be made accessible for online students.

Orientation. Orientations were provided at a number of different levels: General University orientations. General University orientation programs were available for all new and transfer students in this case. Services included campus visits, registration assistance, placement testing and counseling, career guidance, and other support. Orientation programs also provided support and information for parents and families. Campus-wide online orientations. Beyond general campus orientations, the University developed an orientation system specifically directed to online students. Pathway to Distance Education was a mandatory online orientation designed for both prospective and current students to inform them about distance education at the University. It was separate from other mandatory freshman and transfer orientations.

Besides being provided with in-depth information about distance education at UH, students could access course materials posted by the instructor prior to the start of classes. In addition to basic information about courses, this online orientation system provided features and links with information on campus resources, University identification cards, the bookstore, tuition and fee payment, the libraries, career services, services for students with disabilities, campus student organizations, keys to success in online courses, study skills, and exam preparation strategies. Topics such as "What is distance education?," "Being a distance education student," "Getting started with Blackboard," "Keys to success in distance education classes," and "Getting ready for distance education" were included. Additionally, information about registration, buying books, technology, tuition, study skills, and online library access was available.

Course-specific online orientations. Each course instructor prepared and disseminated orientation materials specific to individual courses. These materials were provided in electronic format. For example, instructors posted items such as a "welcome" orientation letter to the course homepage or a visual presentation that was used to share course procedures, syllabus, calendar, tips for success, grading plans, tools, strategies, resources, etc. University student success and retention programs. Student success, satisfaction, and retention can be influenced by students' levels of connection and interactivity with University faculty, personnel, and systems. The University in this case supported several student success programs across campus that helped students make the transition to college and supported their academic pursuits once they arrived. These programs included the following and were accessible to online students: General University support services. Additionally, other critical support services were provided with attention to access for online students: Scholarships and financial aid. The University provided an array of financial aid options to assist online students in their educational pursuits. Additional information about available University undergraduate scholarships was available via the Office of Scholarships and Financial Aid website. The College also offered scholarships directed toward online students. Library resources. Off-campus access to library resources was a critical service for online students. Detailed procedures were developed to meet needs of these students. Resource guides and tutorials were available that outlined the procedures for requesting library materials to be delivered both electronically and/or physically to prescribed off-campus distance education sites. Librarians and electronic help desk services were available. Resources were available to assist students in learning how to remotely use research tools, identify and use databases, search for appropriate sources, request materials for use, and receive materials. Computing and technology help desks. The experiences of this case suggest that the technology support service most valued by online students was the help desk service. University Information Technology recently combined its Customer Services, Information Services, and University Media Services departments to form Technology Services and Support. For times when online students may have been on campus, the Department, College, and University also maintained numerous computer labs. University Student Support Services Outcomes

Orientations. While general University orientation provided support that was used by all students, additional orientation services prepared online students specifically. Help with registration, career decisions, and course selection are examples of the services that were used by all students. Beyond these services, specific electronically delivered orientations enabled online students to get an early start with the tools and strategies for distance learning. Students learned how to access and use campus resources such as the bookstore, libraries, advising, learning support services, and course tools from a distance. Students reported deriving value from these services in terms of helping them learn to use vital campus attributes.

At the course level, specific electronically delivered orientations provided roadmaps for students to succeed in classes. One instructor's use of a welcome letter to share course policies, procedures, and expectations was seen as a friendly way to start a learning relationship. Other students felt they benefitted from orientations provided in the form of PowerPoint or other visual presentations.

University student success and retention. Numerous support programs to facilitate students' transition to and success in college were used. While, in general, tracking of outcomes was difficult, two of the support programs provided for regular, systematic feedback among students, faculty, and program advisors. Both the Urban Experience and Challenger programs yielded strong evidence of positive impacts on students through interim grade reports and mentoring sessions.

Learning Support Services were found to support students by enhancing students' study skills, test-taking strategies, and learning style awareness. For specific content areas, such as Mexican American studies, English writing, natural sciences, and mathematics, additional services were proffered.

General University support services. Access for online students was facilitated for core University services. For example, the resources provided by University Career Services were popular, and, even in a tight job market, they yielded employment contacts and placements. Similarly, student use of the Center for Disabilities provided services and contacts with faculty that led to positive academic outcomes that would not otherwise have been possible.

The University's Counseling and Psychological Services, while providing critical services, could not share student-use data regarding outcomes due to the personal nature and confidentiality of psychological services. Faculty supported students by providing referrals for these services when need was indicated or interest was expressed.

A relatively recent University addition to the general services was Profs with Pride. Interested faculty members network with students through personal and electronic tools to encourage awareness and participation in campus events. For online students faculty used e-mails, links to campus updates, discussion board triggers and comments, and announcements posted to the course website to notify students of events and encourage school spirit. Event attendance showed that students used and valued this faculty-driven support system. An ancillary outcome of this service was that students not only learned about and attended events, but that attendance and communication fostered stronger connections including student–student, student–faculty, and student–campus relationships.

More traditionally, providing bookstore, admissions, administrative, registration, finance, and other basic services for online students ensured that all students could effectively adhere to university processes, requirements, and deadlines. Out-of-state students and those in international locations relied on access to these services. Such capability, for example, allowed several U.S. military personnel to continue their education while serving in remote areas with Internet access provided by the armed services.

Scholarships and awards. To support themselves financially, online students accessed websites that outlined scholarship and financial aid opportunities and application processes. The obvious outcome was that students were able to gain or retain capability to continue their studies. Library resources. Library services enabled students to electronically receive library materials. As an early participant in online student support, the library staff offered a well-developed selection of services that students used remotely. Computer and technology help desks. To meet the challenges of learning online, students made use of computing and technology help desks. Technical problems, while definitely unwanted, were a part of the student experience. Appropriately, the campus had designed a well-supported system of help desks to answer student's questions and service their needs. The help desk intercept logs and communications with course faculty showed that students needed and used these services to solve their technical problems.Discussion and Applications

In the present study, the authors performed a review of the literature, which they used as a means of verifying whether student services were identified as a component of quality in online education. This indicated value for further study of student services. The authors, through their subsequent data collection, categorization, tabulation, and analysis, found that many services were needed and offered. Thus, review of the literature regarding frameworks for quality in online learning systems and the experience of this case, including systems developed and student outcomes, suggests that student support is a recognized and valued component of quality in online programs. Further, it appears important that support be pervasive through all aspects of student engagement in the learning process. Support services are needed at the course, department/college, and university levels. Additionally, it is imperative that accommodation be made to existing student support services to guarantee their accessibility for online students.

Analysis and reflection on the findings of this study lead the researchers to conclude that student support services play a valued and integral part in quality online educational delivery. Directed by the research question "What types of student support services are needed and available to students?," the authors have found the methodological propositions to be confirmed.

Conclusion and The Future

Considerations of the future support needs for online students suggest that adaptations will be needed as students, faculty, and technology become more sophisticated with the supporting technologies and with the mode of instruction itself. Course design and emerging technology, as well as the instructional design imagined and created by faculty, will need to keep pace with increasing levels of sophistication. Students' expectations will expand through generalized exposure to greater technology in all aspects of life, including mobile information and communication technologies. Faculty training, development, and utilization must keep pace with and exceed that growth. Both students and faculty will have evolving needs.

While the needs of students and faculty change and evolve, so too, will the university environment. New technologies will emerge and become available to institutions of higher learning. The concept of "university" may also change, as more completely online institutions emerge. Competition from online-only programs may force changes in traditional universities and their support of students. Funding issues may both improve and present problems, based on new or expanded models of online education. For example, in Australia a lack of public funding has focused interest in online education as an alternate funding source (Gururajan, 2002).

Changes in online education will continue to stimulate changes in the roles of traditional student affairs and student services professionals as they adapt and develop services to meet evolving needs in conjunction with instructors, course designers, and personnel at all levels of the institution.

Traditionally, universities have played a primary role as caretakers of learning. In that context, as faculty continue to care about student learning, the view of support for students in online environments will continue to evolve, and it will require future investigation. Usun (2004), following review of online student support systems in Turkey, suggests that the institutions, not just the courses, must actually be designed or redesigned to enhance learner support. In sum, online education requires both the development of dedicated support services, specifically created for online students, as well as attention to or adaption of existing support to ensure access by online learners. The statement by Thompson and Hills (2005) is true: "Careful planning and adequate resourcing, especially long-term funding and staffing, are necessary to ensure the long-term viability of providing worthwhile learning support services" (p. 664). Similarly, Frieden (1999) states that "the creation and operation of a distance education support infrastructure requires the collaboration of virtually all administrative departments" (p. 48). The provision of these support services, in turn, contributes to overall quality as recognized by the inclusion of student support in the numerous frameworks and standards for online course and program quality.

References

American Distance Education Consortium. (2003). ADEC guiding principles for distance teaching and learning. Retrieved from http://www.adec.edu/admin/papers/distance-teaching_principles.html

Australasian Council on Open, Distance and E-learning. (2010). ACODE benchmarks. Retrieved from http://www.acode.edu.au/resources/ACODE_benchmarks.pdf

Canadian Standards Association. (2009). The ISO 9000 essentials: A practical handbook for implementing the ISO 9000 standards (4th ed.). Toronto, Canada: Author.

Chua, A., & Lam, W. (2007). Quality assurance in online education: The Universitas 21 global approach. British Journal of Educational Technology, 38(1), 133-152. doi:10.1111/j.1467-8535.2006.00652.x

Ehlers, U.-D. (2004). Quality in e-learning from a learner's perspective. European Journal of Open, Distance and E-learning, 2004(1). Retrieved from http://www.eurodl.org/materials/contrib/2004/Online_Master_COPs.html

Eisenhardt, K. M. (1989). Building theories from case study research. Academy of Management Review, 14(4), 532-550. doi:10.5465/AMR.1989.4308385

Frieden, S. (1999). Support services for distance education. Educational Technology & Society, 2(3), 48-54. Retrieved from http://www.ifets.info/journals/2_3/frieden.html

Goodson, C. E., Faulkenberry, L. M., Miertschin, S. L., & Stewart, B. L. (2004). Comprehensive program assessment: The whys and wherefores. In Proceedings of the 2004 ASEE Annual Conference and Exposition. Washington, DC: American Society for Engineering Education. Retrieved from http://search.asee.org/search/fetch?url=file%3A%2F%2Flocalhost%2FE%3A%2Fsearch%2Fconference%2F28%2FAC%25202004Paper492.pdf&index=conference_papers&space=129746797203605791716676178&type=application%2Fpdf&charset=

Goodson, C. E,  Miertschin, S. L., & Stewart, B. L. (2012). On-line delivery of courses: What components are important to students? Computers in Education Journal, 22(1), 47-58.

Gururajan, V. (2002, February). Knowledge management a new game in higher education: A discussion paper on the new trends of technology influencing the higher education landscape. Paper presented at the 11th Annual Teaching and Learning Forum, Perth, Australia. Abstract retrieved from http://otl.curtin.edu.au/professional_development/conferences/tlf/tlf2002/abstracts/gururajan-abs.html

Inglis, A. (2008). Approaches to the validation of quality frameworks for e-learning. Quality Assurance in Education, 16(4), 347-362. doi:10.1108/09684880810906490

Inglis, A., Ling, P., & Joosten, V. (2002). Delivering digitally: Managing the transition to the knowledge media (2nd ed.). London, UK: Kogan Page.

Institute for Higher Education Policy. (2010). About IHEP. Retrieved from http://www.ihep.org/About/about-IHEP.cfm

International Center for Academic Integrity. (1999). The fundamental values of academic integrity. Retrieved from http://www.academicintegrity.org/fundamental_Values_project/pdf/FVProject.pdf

Jung, I. (2011). The dimensions of e-learning quality: From the learner's perspective. Educational Technology Research & Development, 59(4), 445-464. doi:10.1007/s11423-010-9171-4

Jung, I. (2012). Asian learners' perception of quality in distance education and gender differences. The International Review of Research in Open and Distance Learning, 13(2), 1-25. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1159

Lofland, J., & Lofland, L. H. (1995). Analyzing social settings: A guide to qualitative observation and analysis (3rd ed.). Belmont, CA: Wadsworth.

MarylandOnline. (2008). The Quality Matters program rubric. Retrieved September 13, 2010, from http://www.qmprogram.org/rubric/ (archived at http://web.archive.org/web/20100911002421/http://www.qmprogram.org/rubric/)

McKinnon, K. R., Walker, S. H., & Davis, D. (2000, February). Benchmarking: A manual for Australian universities. Canberra, Australia: Department of Education, Training and Youth Affairs. Retrieved from http://foi.deewr.gov.au/system/files/doc/other/benchmarking_a_manual_for_australian_universities.pdf

Miertschin, S. L., Goodson, C. E., & Schroeder, S. (2010). Online tutoring support service for STEM. In Proceedings of the 2010 ASEE Annual Conference and Exposition. Washington, DC: American Society for Engineering Education. Retrieved from http://search.asee.org/search/fetch?url=file%3A%2F%2Flocalhost%2FE%3A%2Fsearch%2Fconference%2F32%2FAC%25202010Full1299.pdf&index=conference_papers&space=129746797203605791716676178&type=application%2Fpdf&charset=

Moore, J. C. (2005). The Sloan Consortium quality framework and the five pillars. Retrieved from http://www.sloanconsortium.org/publications/books/qualityframework.pdf

National Institute of Standards and Technology. (2009). 2009-2010 education criteria for performance excellence. Retrieved from http://www.nist.gov/baldrige/publications/upload/2009_2010_Education_Criteria.pdf

Parker, N. K. (2008). The quality dilemma in online education revisited. In T. Anderson (Ed.), The theory and practice of online learning (2nd ed., pp. 305-342). Edmonton, Canada: Athabasca University Press. Retrieved from http://www.aupress.ca/books/120146/ebook/13_Anderson_2008-Theory_and_Practice_of_Online_Learning.pdf

Pettigrew, A. M. (1990). Longitudinal field research on change: Theory and practice. Organizational Science, 1(3), 267-292. doi:10.1287/orsc.1.3.267

Phipps, R. A., & Merisotis, J. P. (2000). Quality on the line: Benchmarks for success in Internet-based distance education policy. Washington, DC: The Institute of Higher Education Policy. Retrieved from http://www.ihep.org/assets/files/publications/m-r/QualityOnTheLine.pdf

Simelane, S. (2009). Success indicators in implementing technology-enhanced courses during a professional development programme. In P. A. van Brakel (Ed.), Proceedings of the 11th Annual Conference on World Wide Web Applications. Cape Town, South Africa: Cape Peninsula University of Technology. Retrieved from http://www.zaw3.co.za/index.php/ZA-WWW/2009/paper/viewFile/25/20

Sims, R., Dobb, G., & Hand, T. (2002). Enhancing quality in online learning: Scaffolding planning and design through proactive evaluation. Distance Education, 23(2), 135-148. doi:10.1080/0158791022000009169

Soy, S. K. (1997). The case study as a research method. Unpublished manuscript, School of Information, The University of Texas at Austin, Austin, TX. Retrieved from http://www.ischool.utexas.edu/~ssoy/usesusers/l391d1b.htm

Stewart, B. L., Norwood, M., Ezell, S., & Waight, C. L. (2006). Case study: Collaborative creation of an online degree program. Innovations in Education and Teaching International, 43(3), 197-210. doi:10.1080/14703290600618399

Thompson, J., & Hills, J. (2005). Online learning support services for distance education students: Responding to and maintaining the momentum. In H. Goss (Ed.), Balance, fidelity, mobility: Maintaining the momentum? Proceedings of the 22nd ASCILITE Conference (pp. 661-665). Brisbane, Australia: Queensland University of Technology. Retrieved from http://www.ascilite.org.au/conferences/brisbane05/blogs/proceedings/76_Thompson.pdf

Tinto, V. (1975). Dropout from higher education: A theoretical synthesis of recent research. Review of Educational Research, 45(1), 89-25. doi:10.3102/00346543045001089

Usun, S. (2004). Learner support services in distance education system (A case study of Turkey). Turkish Online Journal of Distance Education, 5(4). Retrieved from http://tojde.anadolu.edu.tr/tojde16/articles/s_usun.htm

Walker, S. L., & Fraser, B. J. (2005). Development and validation of an instrument for assessing distance education learning environments in higher education: The Distance Education Learning Environments Survey (DELES). Learning Environments Research, 8(3), 289-308. doi:10.1007/s10984-005-1568-3

Wang, Q. (2006). Quality assurance – best practices for assessing online programs. International Journal on E-Learning, 5(2), 265-274. Retrieved from EdITLib Digital Library. (5601)

Western Cooperative for Educational Telecommunications. (2001). Best practices for electronically offered degree and certificate programs. Retrieved July 24, 2006 from http://www.wcet.info/Accrediting%20-%20Best%20Practices.pdf (archived at http://web.archive.org/web/20060724103942/http://www.wcet.info/Accrediting%20-%20Best%20Practices.pdf)

Yin, R. K. (1994). Case study research: Design and methods (2nd ed.). Thousand Oaks, CA: Sage.

View the original article here

Wrapping a MOOC: Student Perceptions of an Experiment in Blended Learning

         Wrapping a MOOC: Student Perceptions of an Experiment in Blended Learning

Derek O. Bruff
Director, Center for Teaching
Senior Lecturer, Department of Mathematics
Vanderbilt University
Nashville, TN 37235 USA
derek.bruff@vanderbilt.edu

Douglas H. Fisher
Associate Professor of Computer Science and of Computer Engineering
Department of Electrical Engineering and Computer Science
Vanderbilt University
Nashville, TN 37235 USA
douglas.h.fisher@vanderbilt.edu

Kathryn E. McEwen
Graduate Assistant, Center for Teaching
Doctoral Candidate – German, Department of Germanic and Slavic Languages
Vanderbilt University
Nashville, TN 37235 USA
kathryn.e.mcewen@vanderbilt.edu

Blaine E. Smith
Doctoral Candidate – Language, Literacy, and Culture
Department of Teaching and Learning
Vanderbilt University
Nashville, TN 37235 USA
blaine.smith@vanderbilt.edu

Abstract

Although massive open online courses (MOOCs) are seen to be, and are in fact designed to be, stand-alone online courses, their introduction to the higher education landscape has expanded the space of possibilities for blended course designs (those that combine online and face-to-face learning experiences). Instead of replacing courses at higher education institutions, could MOOCs enhance those courses? This paper reports one such exploration, in which a Stanford University Machine Learning MOOC was integrated into a graduate course in machine learning at Vanderbilt University during the Fall 2012 semester. The blended course design, which leveraged a MOOC course and platform for lecturing, grading, and discussion, enabled the Vanderbilt instructor to lead an overload course in a topic much desired by students. The study shows that while students regarded some elements of the course positively, they had concerns about the coupling of online and in-class components of this particular blended course design. Analysis of student and instructor reflections on the course suggests dimensions for characterizing blended course designs that incorporate MOOCs, either in whole or in part. Given the reported challenges in this case study of integrating a MOOC in its entirety in an on-campus course, the paper advocates for more complex forms of blended learning in which course materials are drawn from multiple MOOCs, as well as from other online sources.

Keywords: massive open online course (MOOC), blended learning, online learning, wrapper, flipped classroom, course cohesion, subject coupling, task coupling, local learning communities, global learning communities, course customization

Introduction

Technology continues to transform education in traditional and online settings (Baldwin, 1998), as the recent proliferation of massive open online courses (MOOCs) demonstrates (Guthrie, 2012; Mangan, 2012; Pappano, 2012). Although MOOCs are seen to be, and in fact are designed to be, standalone online courses (Hill, 2012), their introduction to the higher education landscape has expanded the space for possible blended or hybrid course designs (those that combine online and face-to-face learning experiences). Creating a blended course that incorporates another instructor's MOOC simplifies the blended course design problem in some respects, by fixing the online component of the blended course, while allowing the blended course instructor to shape the in-class components. However, fitting in-class modules into an existing MOOC in a way that optimizes student engagement, satisfaction, and ultimately learning, can be challenging.

This paper reports a case study of a blended graduate course in machine learning at Vanderbilt University in Fall 2012, which incorporated a Stanford University MOOC. It reports student perceptions of the blended course and identifies elements of the blended course design that the authors think are responsible for these perceptions. Drawing on these findings, the paper suggests a number of design considerations of potential interest to instructors wishing to build blended learning experiences around MOOCs or to integrate online and face-to-face components of blended courses more generally. Although the blended course in this study adopted the entirety of one particular MOOC, the paper suggests that other customizations may well be both possible and desirable, particularly those that select from and mix multiple MOOC sources.

Background

Blended Learning

Blended or hybrid approaches to teaching integrate face-to-face (offline) instruction with online materials, creating what can be a flexible and effective model for instruction (Aycock, Garnham, & Kaleta, 2002; Bowen, Chingos, Lack, & Nygren, 2012; Hill, 2012). By leveraging online modes of content delivery outside of class time, blended courses can free face-to-face sessions for instructor feedback, applications, and interaction (Aycock et al., 2002; Hill, 2012). Indeed, a 2010 meta-analysis prepared by the United States Department of Education reports that in recent experimental and quasi-experimental studies, blended instruction has been found to be more effective than either face-to-face or fully online instruction (Means, Toyama, Murphy, Bakia, & Jones, 2010). However, caveats are in order, as the meta-analysis notes "it was the combination of elements in the treatment conditions (which was likely to have included additional learning time and material as well as additional opportunities for collaboration) that produced the observed learning advantages" (Means et al., 2010, p. xviii).

As Aycock et al. (2002) outline, blended approaches demonstrate wide variations not only in the distribution of face-to-face and online time, but also in course design, which reflect and accommodate differences in teaching style and course content. Although there is no "standard" approach to blended courses, they often involve a rigorous, time-intensive redesign of traditional face-to-face courses to fully integrate face-to-face and online learning (Aycock et al., 2002; Stone & Perumean-Chaney, 2011). Students' work online must be made clearly relevant to their work in the classroom, just as the face-to-face sessions must draw on and apply the online materials (Babb, Stewart, & Johnson, 2010; Gilbert & Flores-Zambada, 2011; Toth, Amrein-Beardsley, & Foulger, 2010). Building on this research, this paper will argue, based on the authors' case study, that the degree and type of coupling between online and face-to-face components is an important dimension along which blended courses can be varied.

MOOCs and Blended Learning

Despite variations in format, the "traditional" blended course assumes a common designer of both face-to-face and online learning: namely, the on-campus instructor(s) (Aycock et al., 2002; Gilbert & Flores-Zambada, 2011; Rodriguez & Anicete, 2010). For example, in one version of what is often called a "flipped" or "inverted" classroom (Lage, Platt, & Treglia, 2000), students gain first exposure to course content through online video lectures created by their instructor, then explore that content more deeply during class through active learning exercises also designed by their instructor (Talbert, 2012). Although some versions of the flipped classroom involve materials created by others, such as the use of textbooks for the pre-class first exposure to content (Mazur, 2009), the blend of online and face-to-face learning activities in such courses is designed by the students' on-campus instructor.

MOOCs present a new option for blended course design. Instead of "flipping" one's course by producing online lecture videos or leveraging textbooks, instructors can "wrap" their courses around existing MOOCs (Caulfield, 2012a; Fisher, 2012; Koller, 2012; Mangan, 2012; Shirky, 2012). In this approach, students in an on-campus course are asked to participate in part or in whole in a MOOC hosted at another institution, with the local instructor supplementing that online learning experience with face-to-face classroom interactions. Since MOOCs are designed externally and intended to function as stand-alone courses (Hill, 2012), incorporation of a MOOC in a blended learning experience constrains the face-to-face instructor's course design decisions: the online component is relatively fixed, and only the in-class component can be varied. The online component is, however, only relatively fixed because the instructor of the wrapper can always choose to use only parts of the MOOC, a possibility that the authors return to later in discussing customization around more than one MOOC and other online content.

The challenges posed by "wrapping" a course around a MOOC are not unlike those posed by incorporating a textbook, authored by another, into a course. However, given the variety and interactivity of learning experiences available on most MOOCs – lecture videos, automatically graded quizzes, discussion forums – the use of externally hosted MOOCs in blended courses involves design questions not raised by the use of textbooks. These are the questions explored in the current case study.

Methods

Instructional Context
The setting for the present case study (Yin, 2003) was a graduate-level course on machine learning taught at Vanderbilt University, a research university, by co-author Fisher. The Machine Learning graduate course was typically only offered every other year by the computer science program. Due to demand for the course from another graduate academic program of the University, a special section of the course was run during the Fall 2012 on an "off" year, and as an overload course for Fisher. As a result, many of the 10 students in the course were from outside computer science though all were graduate students with some computing sophistication, but new to machine learning. In order to maintain a sustainable workload across all his courses, Fisher decided to draw on some of the educational resources provided by MOOCs for this course, building on his experience incorporating open educational resources (Wiley & Gurrell, 2009), such as online lecture videos created by other faculty, into previous courses (Fisher, 2012). In fact, in an earlier Spring 2012 Machine Learning course, Fisher used online lectures by Stanford professor Andrew Ng, director of the Stanford Artificial Intelligence Lab and co-founder of Coursera.

Students in the Fall 2012 course, however, were asked to go well beyond simply watching Ng's online lectures; they actually enrolled in and were required to complete Ng's Machine Learning MOOC on the Coursera platform. This involved watching lecture videos, completing quizzes and programming assignments, and, optionally, participating in discussion forums. Students were asked to take screenshots of their submitted quizzes and programming assignments and send those to Fisher, allowing that work to contribute to the students' grades in the Vanderbilt course.

The start of the 10-week Stanford MOOC happened to coincide with the beginning of the Vanderbilt semester, one of the reasons Fisher chose to use it as part of his course. However, there were topics in machine learning not addressed by the MOOC that were of potential use to students in their research at Vanderbilt. Thus, students were also assigned additional readings, which were discussed in weekly face-to-face class sessions led by Fisher. Where the MOOC provided an introduction to some classic and widely used methods of machine learning, the readings were journal papers, consisting of both recent and seminal research in the field, chosen to build on the topics covered in the MOOC and to introduce other important areas of machine learning. During the final four weeks of the semester, after the MOOC ended, students worked individually on projects of their own design, receiving guidance and feedback from Fisher and each other, during the remaining in-class meetings. Student projects of this sort had been components of previous offerings of Fisher's Vanderbilt Machine Learning course; it was the first 10 weeks of the 14-week semester that differed in the Fall 2012 offering.

Figure 1 shows the rough layout of machine learning topics throughout the Fall 2012 course. The left column gives the topics covered by journal readings, which were the focus of weekly in-class discussions; the right column lists the online video topics from Andrew Ng's MOOC. The one-week offset in the readings (left column) was intended to allow Ng's MOOC videos to first introduce concepts before they were then expanded upon in the readings. Arrows between the columns indicate some of the dominant conceptual correspondences between the readings and MOOC videos, though synergies along these lines could only be realized at relatively high levels of abstraction and not at a "nuts-and-bolts" level. Fisher adapted his experience with these same videos in the Spring 2012 Machine Learning course in arriving at this sequencing for the Fall 2012 course.

Figure 1. Topics covered in the wrapper course by readings (left) and MOOC (right)

In many cases there are no correspondences shown. In some cases, none are shown because the connections are pervasive. For example, though no correspondences are shown for Week 6 of the MOOC lectures on experimental evaluation, this material had relevance across nearly all in-class discussions and online topics, and, in fact, the readings and their discussions presaged and reflected on this material throughout the course. Similarly, Week 7 of the in-class discussions on projects also referenced material throughout both online and in-class topics. In contrast, some in-class topics, such as relational learning, inductive logic programming, and knowledge-biased learning, simply had no strong linkage – certainly not at the nuts-and-bolts level – with MOOC topics, though in some cases the contrasts suggested by paradigmatic differences were a topic of discussion.

Although the topic references listed in Figure 1 are course specific, the implication that the figure conveys that the readings for in-class discussion were selected with some care has broad applications. They also reflect Fisher's priorities to include certain topics, which although having little or no substantive connection to the online topics, he thought important particularly in the Vanderbilt context. And in all cases, even linkages that did exist were treated at a relatively high level of abstraction.

Fisher describes this course structure as a "wrapper" approach, a term adopted from the machine learning research literature, referring to an algorithm that is wrapped around another in order to extract the most salient features from the environment, and therefore to improve overall learning. With this approach in mind, he "wrapped" his on-campus course around the Machine Learning MOOC offered on the Coursera platform. The in-class lectures and low-stakes homework assignments Fisher provided in previous offerings of this course were replaced by the MOOC's online lecture videos, automatically graded quizzes, and programming assignments. Doing so enabled Fisher to use class time differently, focusing it more on interactive discussions and more challenging material. This structure is a version of the flipped classroom referenced above. In Fisher's case, the MOOC played the role of the earlier video lectures or textbook, providing students with a structured introduction to some of the course content.

Data Collection and Analysis

In order to explore student experiences learning in this wrapped course, a focus group was conducted with the students during one of the weekly class sessions just after the MOOC ended. The focus group, with all 10 students participating, was conducted during the first half hour of class that day. Students were informed that the focus group was part of a research project exploring hybrid teaching models and that their instructor was interested in their feedback on the course. The focus group was audio recorded and transcribed for later analysis. An informal and de-identified summary of the student remarks was shared with Fisher shortly after the focus group, before the end of the semester.

Later in the semester, students were asked to complete the standard end-of-course evaluation forms used widely at the University. Students' responses to two holistic, Likert-scale questions on these forms are discussed below. Additionally, a few weeks after the course had concluded (after winter break), students were asked by Fisher to complete a post-course survey, which was designed by the researchers to further explore some of the themes that emerged from the focus group. The survey consisted of 14 Likert-scale questions and three open-ended questions, and was taken by the students anonymously. Only five of the 10 students in the course completed the survey, yielding a 50% response rate.

Qualitative data analysis for this study involved the constant comparative method (Strauss & Corbin, 1998) and the development of case studies (Yin, 2003). During the initial phase, the transcripts of the focus group and students' responses for the open-ended survey questions underwent line-by-line coding in order to establish categories and subcategories related to students' experiences and views of the Machine Learning class and the wrapper approach. These overarching themes were triangulated (Strauss & Corbin, 1998) with Fisher's perspective as the course instructor. During this iterative process, the researchers met regularly to discuss the emergent categories, refine themes, and connect ideas.

Findings

Overall, student response to the wrapper approach in the Machine Learning course was enthusiastic. They described Ng's lecture videos as designed effectively, presented clearly, and informative; they described the MOOC as generally useful for self-paced learning. The students did not engage actively in the online community of peer learners created through the MOOC, preferring to interact with the local learning community provided by the on-campus component of the course. Although their overall response to the wrapper approach was positive, students pointed to challenges in integrating the online and face-to-face components of the course. Student perspectives on these issues are described in the following section.

Value of Self-Paced Learning

According to students, the major advantage of the MOOC over a traditional lecture-based course was its greater flexibility, customization, and accessibility, which students saw as encouraging structured self-paced learning.

Students valued the flexibility offered through the MOOC, which allowed for them to watch the weekly video lectures at their own pace and on their own schedule. As one student described:

"I really, really like the absorbing information on your own time at your own speed, and through this sort of video format with someone that you know is a really good lecturer, has really carefully prepared these topics, and I think that's much more efficient [than traditional lectures]." (Focus group transcript, November 14, 2012)

Along with students finding "being able to [watch videos] on your own schedule" as "very valuable," students also described that the videos' shorter length, typically between five and 15 minutes, helped them to keep their attention focused and to better digest the lecture content (Focus group transcript, November 14, 2012).

Various features of the online platform also allowed for students to customize the way in which they viewed lecture videos, which they found to be more efficient and conducive for learning. For example, one student explained how the variable viewing speed, captions, and embedded quizzes helped to "make [Coursera] a wonderful learning experience":

"I love the way Coursera is set up, and that you can kind of set your own schedules, watch it when you want. In addition to being able to watch at 2X [double speed], you also have captions throughout, so 2X plus captions makes it really easy to understand what's going on. And they also have questions based throughout the videos, and quizzes and homework assignments through there also, to totally keep you fully integrated with what's going on, and makes it a wonderful learning experience." (Focus group transcript, November 14, 2012)

Another student described the MOOC lecture videos as "basically the best thing ever," having watched the online lectures at "twice the speed," which helped the student "stay focused" and "feel like [he/she] got a lot more out of the material" (Focus group transcript, November 14, 2012). In addition to being able to speed up video playback and customize features, students also found the almost immediate feedback on quizzes and programming assignments to be helpful. One student explained that this "instant feedback" allowed for him/her to gauge his/her understanding and "make changes" accordingly.

Although students believed the flexibility of MOOC's self-paced environment to be effective, they also described it to be a challenge to stay on schedule. One student explained, "You have to be very disciplined to make sure you're keeping up on the material. If not, you'll find that you're trying to play catch-up a lot of times" (Focus group transcript, November 14, 2012). Another student, however, found that the self-paced environment enabled him/her to work ahead. Despite these differences, students described the face-to-face sessions with Fisher as helping to keep them on track with the material online.

Local vs. Global Learning Communities

Although students participated regularly in the Machine Learning MOOC to complete and submit assignments – for example, the programming assignments and quizzes, also submitted to Professor Fisher – they did not actively participate in either the Coursera discussion forums or the study groups formed online. Students cited time constraints as the main reason for not participating more actively in the online discussion forums. Instead, they used the discussion boards to check for course errata or to quickly troubleshoot questions or problems, but tended to ask questions among their local peers.

Additionally, students found the discussion boards helpful for solving problems they encountered, including sharing strategies and solutions pertaining to those problems. Although no students described posting a question on the discussion boards, students did describe the forums as useful for learning about "other people who were having the same problem" and applying their solutions to the problem. One student reported, "I knew that if I was stuck on something, thousands of other students were trying to do the same thing. In all cases I could find my specific questions in the online forums" (Survey response, January 19, 2013). Another student affirmed, "Whenever I had trouble on an assignment, I could almost always just go to the forum and look at the answers provided by people who had already run into similar stumbling blocks" (Survey response, January 17, 2013).

Instead of utilizing the online discussion boards, students preferred to ask questions about and discuss course content during the face-to-face class sessions. As one student explained in the focus group:

"I think when I had a question, I tended to ask the other people in here, before I would probably ask it on the discussion board. I mean, me and [another student] would talk before class about some of the material review." (Focus group transcript, November 14, 2012)

Students liked the structure of the wrapper format because it opened up space for productive class discussions related to the content. They also described in-class discussions as valuable for generating new ideas and new research projects. As one student described in the focus group:

"One of the things I liked is that, since you did the lecture material at home whenever you had time for it, it saved the class time for discussion. And so we didn't always discuss stuff that was exactly following along with the course, but whenever we did, I found that a lot more helpful." (Focus group transcript, November 14, 2012)

Another student echoed a similar belief: "So, I really liked doing that [online content] sort of outside [of class], and then coming in and sort of like taking all of the knowledge that supposedly you sort of download into your brain and apply." A third student expressed a view of class time as "it's more like you ask questions, you learn." (Focus group transcript, November 14, 2012)

Interestingly, three of the five responses to the online survey question asking how to improve the course suggested even more discussions of the MOOC material during the face-to-face class meetings. One student explained that discussions were valuable because they facilitated "instant feedback from [the] instructor and classmates," (Survey response, January 28, 2013) and another explained,

"I would recommend that you discuss more of the Coursera material in the class. I don't think you should give a repeat lecture of the material, but rather spend some time talking about the methods presented and the main ideas of the methods. This was done in our class to a degree, but I would like even more discussion from Coursera." (Survey response, January 19, 2013)

In the focus group, students also suggested more in-class discussion of the material presented in the MOOC. These suggestions included "short discussion for the first 15 minutes of class to ask questions or consolidate ideas before moving on" and more "discussion of applications" of the online content (Focus group transcript, November 14, 2012).

Misalignment between Face-to-Face and Online Components

According to students, one challenge in this offering of the on-campus Machine Learning course was that the topics covered in class did not always line up with the material covered in the video lectures on a week-to-week basis. Students mentioned that they would have preferred a greater degree of alignment between online and on-campus offerings, so that the material in-class would more directly address, and expand upon, the topics covered online. As one student explained in the focus group:

"I felt like the topics we covered in class – because we'd read some like outside papers – they didn't line up very well with a lot of the online material. I mean, not that it wasn't valuable stuff, but it seemed kind of disjointed to me." (Focus group transcript, November 14, 2012)

The misalignment was particularly problematic for students in terms of the research papers discussed in class. One student commented that the information in the papers was presented in a "less structured format" than the information in the MOOC materials, making the papers seem "less accessible." However, as another student pointed out, the research papers required a "different kind of learning" than the highly structured video lectures. And as that student described, although the papers raised more questions than the online lecture material, the face-to-face sessions provided a space for discussion.

Students emphasized that they were new to machine learning and reported feeling ill-prepared and lacking in context to adequately understand the papers. As beginners, they would have preferred to read papers more directly connected to the video lectures and material covered online. They suggested supplementing reading with review articles assigned before each paper, or by including an outline or key points to guide assigned reading. Even though there was a consensus among students that the papers were challenging, they described the reading in the terms of application of knowledge, an exercise in "Can you get something from it?" – that is, the real-life negotiation of meaning. And, in fact, one student reported learning to read machine-learning papers in the course of the seminar, despite the challenges:

"One thing I was just going to say about the papers is for me, they were kind of a bitter pill to swallow. I went through and I read these papers, but looking back, I'm glad I did it because I feel like I can go to a machine learning paper, and I can read it, and I won't be as intimidated by it because I've kind of struggled through it all semester reading these things. So, I feel like I'm in a better place now than I was before I started this course." (Focus group transcript, November 14, 2012)

Student Perceptions of the Instructors

Valuing both Ng's and Fisher's contributions, students viewed each as having different roles in the Machine Learning course. Overall, students perceived Ng, a "world-renowned researcher and teacher," as the lead lecturer of the course and explained that they found his teaching style to be effective (Focus group transcript, November 14, 2012). Students explained that "he did a really good job with the course" (Focus group transcript, November 14, 2012). Specifically, one student pointed out: "Andrew Ng does a great job of teaching the skills necessary and highlighting potential problems" (Survey response, January 17, 2013).

In contrast, students perceived Fisher's role in the face-to-face sessions as that of a "facilitator." They described him as following up on their work in the MOOC, explaining concepts and providing background for the papers, and leading class discussions. In the focus group, one student explained:

"I thought he [Fisher] was a facilitator, and he would try to facilitate discussions. He would introduce papers for us to read, and then just kind of follow up and make sure we're doing the Coursera stuff by having us submit everything to him each week. That's the word that comes to my mind." (Focus group transcript, November 14, 2012)

Another student built on this comment about Fisher's role as facilitator:

"He did a really good job in facilitating the discussion of the research papers, I thought. And he made sure that everybody talked, even when we didn't want to. He would tease something out of us to get us to talk about the paper and what we thought or what we didn't understand." (Focus group transcript, November 14, 2012)

As noted above, students in the course were asked to complete the University's standard end-of-semester course evaluation. Six of 10 students responded to the two holistic questions: "Give an overall rating of the instructor" and "Give an overall rating of the course." Each question had an average response of 4.17 (on a 5-point scale – 3 being average, 4 being very good, and 5 being excellent), with a standard deviation of 0.68. These ratings were comparable to Fisher's Spring 2012 Machine Learning course, in which students viewed Ng's lectures, but the rest of the MOOC was not used. Before 2012 (Spring and Fall), the last offering of the Machine Learning course by Fisher (or any instructor) was in the Spring of 2006. That offering, occurring well before MOOCs were available, was taught using a more traditional face-to-face approach. The average end-of-semester ratings of instructor and course in 2006 were 3.83 (standard deviation: 0.89) and 3.66 (standard deviation: 1.11), respectively, with six of six students responding. These data, although based on small sample sizes, indicate that the hybrid course of 2012 was somewhat better received than the more traditionally taught course of 2006. While suggestive only, the increase in means and the decrease in standard deviations (from 2006 to 2012) are measures that warrant continued tracking in future wrapper courses.

Discussion and Conclusion

While these numbers are too small to support strong conclusions on the efficacy of the authors' wrapper design, the experience is suggestive and can guide research going forward. In this section, after reviewing key observations from the case study, the beginnings of a categorization scheme for the kinds of couplings that can arise between the online and in-class components of a blended course are introduced. The present study is then framed with this nascent categorization, and an argument is put forward that customizing a wrapper around parts of multiple MOOCs – and other online resources – can both leverage the advantages of MOOC platforms, and also soften design constraints that stem from adopting a MOOC en masse.

MOOCs as Learning Resources

It is clear that the students in this Machine Learning course found the online lecture videos provided by the MOOC to be useful, thanks to both content and form. While it is possible that less experienced students (say, first-year undergraduates) might not find online lecture videos, with their lack of instructor-student interaction, as useful, it is clear that at least in this teaching context, the online lectures were a valuable resource for the students.

Interestingly, "outsourcing" the lecture component of the course to the MOOC instructor did not diminish the students' view of the on-campus instructor as an effective teacher. They noted that Fisher's role was changed from a lecturer to a facilitator, but the students had an overall positive view towards Fisher and Ng. This is perhaps not surprising given the greater (graduate) experience level of the students in this course. It is also possible that Fisher's inclusion of research papers he selected helped students see him as an expert, even if he was not fulfilling that role in the traditional way of lecturing. Less experienced students (again, consider first-year undergraduates) might not place as much value on the facilitator role taken by the on-campus instructor.

Moreover, it is possible that having two instructors, with different points of view on the course content, helped the students better understand debates within the discipline. Again, expert differences could be challenging for underclassmen and/or students in fields where a single "right" answer is not typically mandated, such as literature or history; however, they could also provide a useful tool for helping students move from what Kuhn (1992, pp. 167-168) describes as "absolutist" or "relativist" modes of thought to more "evaluative" modes as they grapple with observations that experts in a field can and do disagree. Though there were no overt disagreements between instructors on the material that was covered, Fisher's inclusion of material unrelated to the MOOC coverage reflected (probably) Fisher's different prioritization of material; different prioritizations among machine learning researchers and practitioners was a topic for some in-class discussion.

Also valuable, at least for some students, were the discussion forums provided within the MOOC. Their use of the online learning community, albeit limited to selective reading and no posting, points to the value of the "M" and the "C" in the acronym MOOC: with thousands of other students ("massive") working through the same material at the same time ("course"), it was highly likely that any difficulty encountered by one of Fisher's students was raised and addressed on the forums.

Although the present case study's results indicate that MOOCs can serve as useful learning resources as part of a blended course, student comments in the study also point to the design challenges involved in wrapping a face-to-face course around a MOOC. The misalignment that students perceived between the online and face-to-face components of the wrapped course speaks to the recommendation that students' work online must be made clearly relevant to their work in the classroom, and vice versa (Babb et al., 2010; Gilbert & Flores-Zambada, 2011; Toth et al., 2010). This design challenge seems particularly difficult when building a blended course around a MOOC, given that the online component is relatively fixed, potentially inviting a schism between the online and face-to-face components of the course. This design challenge is explored in the following subsections.

Coupling between Online and In-Class Components

The authors' study suggests that hybrid courses are characterized and distinguished by the coupling that occurs between the online and face-to-face components, as well as the cohesion of the hybrid course in total. Coupling refers to the kinds and extent of dependency between online and in-class components of a hybrid course, whereas cohesion refers to the relatedness of the course content overall.

There was a relatively low degree of coupling (or loose coupling) in Fisher's course, by the instructor's design, and to the apparent dissatisfaction of some students. The factors behind the low-coupling design were: (1) that there were material and skills that the on-site instructor wanted to cover that was not covered by the MOOC, with limited synergies possible; (2) that the on-site instructor, a machine learning expert himself, felt that the MOOC modules were excellent and self-contained; (3) that those modules were certainly within the grasp of graduate students taking the course (indeed, class assessments confirmed this); and (4) that the instructor needed to maintain a sustainable workload (this was an overload class), and greater coupling generally requires greater time and effort (Aycock et al., 2002). Under (1), the skills at issue are those of reading and understanding journal papers published in the literature, skills in which graduate students must become practiced.

The coupling that did exist in the course involved limited in-class discussion of MOOC material as it related to some of the readings. These interactions, in which part of a class session was used to synthesize across readings and MOOC lectures, was perhaps closest to a traditional flipped classroom. The authors call this subject coupling, because subject matter is shared across the online and face-to-face components of a course. While most periodic (e.g., weekly) assessments were done through the MOOC, Fisher did give a weekly quiz on the readings, and in some cases these quizzes would draw upon both reading and video material (e.g., students received an in-class quiz that asked them to combine concepts from the reading on regression trees with the MOOC material on multivariate regression).

The instructor also intended that final projects by students would draw on methods presented through the MOOC and the readings, and that the final projects would also allow students to practice research methodologies, which they were learning about through both readings and the MOOC. This is an example of what the authors call task coupling, since online and face-to-face components contribute to the completion of a task, typically by learning and applying complementary subject content and/or skills. (These categories, subject and task, are not intended to be mutually exclusive; indeed, much of what would be characterized as active and experiential learning would involve both types.)

Implications of Coupling on Student Satisfaction

Student feedback suggests that students would have liked stronger subject coupling between MOOC and face-to-face components, in which the MOOC material is reviewed in class. It remains an open question as to whether students would have been equally satisfied with task coupling, if only it had been distributed throughout the semester (e.g., regular in-class activities in which students applied what they learned in the MOOC) rather than reserved for the project at the end of the semester.

The previous offering of the Machine Learning course, in the Spring 2012 semester, offers a contrast here. As noted earlier, during that semester Fisher incorporated most of the online lecture videos from the Stanford MOOC into the course, but did not require students to participate in the MOOC itself. Indeed, the MOOC was not offered at that time; only the archived lecture videos were available. (This points to another design constraint in wrapping a course around an external MOOC: the times at which the MOOC is offered might not align well with the local academic calendar.) The Spring 2012 offering of the course had higher subject and task coupling, since the online videos were addressed more directly during class (including through quizzes) and the student projects were started earlier in the semester. Even though end-of-semester students' ratings for both Spring and Fall 2012 course offerings were comparable, none of the Spring students mentioned any kind of misalignment between the online and face-to-face components of the course. This experience offers some evidence for what seems a natural conclusion, that higher coupling results in fewer student concerns about low coupling.

Despite some student discomfort with it, the low coupling approach nonetheless resulted in a very satisfactory class as measured by end-of-semester evaluations. Nonetheless, creative approaches to course redesigns that more highly couple the online and in-class components are certainly of interest. Below, plans are described for a blended approach that draws resources from multiple online resources, including MOOCs, which may have positive implications for coupling.

Cohesion and Coupling

The low coupling design of the Fall 2012 Machine Learning wrapper contrasts with the holistically designed, highly coupled, blended pre-MOOC courses surveyed earlier. As noted, the instructor of a wrapper can craft in-class activities that more significantly connect with the existing MOOC, through subject, task, and other forms of coupling; however, learning and teaching goals and scope necessarily guide and constrain course design.

In addition to coupling between online and face-to-face components, the content cohesion of a blended course, or of any course for that matter, must also be addressed. Indeed, the degree of course cohesion will influence the degree and types of coupling that are most natural within a course. Fisher's Machine Learning course was a survey course, designed to cover a wide range of machine learning methods, many of which are quite disparate. As such, the course cohesion was low, as compared to say, Ng's stand-alone MOOC, in which topic choice and scaffolding created a strong sense of synergy. Thus, in blended environments there can be not only a low coupling between online and face-to-face components of a course, such as Fisher's wrapper, but also a low coupling between the face-to-face modules, or between the online modules. Indeed, to the extent that there exists, in wrapper courses, a difference in the degree of scaffolding provided for materials in the online and the on-campus components, it is possible that additional scaffolding of materials in the on-campus classroom could increase students' perception of cohesion, even without incorporating additional forms of coupling.

Furthermore, the Fall 2012 wrapper combined a graduate seminar course (which included journal readings) delivered through the in-class component with a more structured, lecture-based course delivered through the online component. Including both components in one course is important in the authors' setting, but the very different modalities for learning may have been more responsible for perceptions of schism than the hybrid online and face-to-face structure per se. Moreover, student perceptions of schism may have been magnified because "the two faces" (Professors Fisher and Ng) of the two respective course components were different. An open question is whether students would have been as concerned with schism (between content) in a traditional, one-instructor, entirely face-to-face, low-cohesion survey course, as they were in the authors' wrapper version of a survey. Generally, student expectations regarding a wrapper should be characterized and addressed, because having multiple instructors in multiple modalities is not in the experience of most students.

Finally, this paper has introduced the barest categorization scheme for coupling and cohesion, but the authors expect that this can be usefully expanded and deepened. For example, subject coupling (across or within modalities) may be broken down into repetitive (or reinforcing) treatment of the same material, or connected by prerequisite relationships, with some material building on others. Likewise, forms of task coupling might be further distinguished into modules looking at complementary subject content and/or complementary skill content.

Customization and Other Future Work

The Fall 2012 wrapper used the totality of the Stanford MOOC, but customization strategies might choose to wrap a course around only part of a MOOC, or more ambitiously, to wrap a course around parts of multiple MOOCs. The possibilities to explore this latter type of customization continue to emerge as the Coursera platform continues to expand; for example, the platform hosts the lectures of a second Machine Learning MOOC (Domingos, 2013) from the University of Washington. While this course has some intersections with the Stanford MOOC, it also intersects the additional content that Fisher included in his wrapper. A next step would be to design a wrapper around two or more MOOCs, with the instructor selecting and mixing lectures and assignments from each MOOC, as well as using other online content, some perhaps even produced by the instructor of the wrapper. This is an exciting possibility, which does not require that a MOOC be adopted in its entirety, as is. Currently this process of selection and mixing is technically easy, and such use cases may further drive MOOC providers to design for piecemeal use, accelerating customization and a co-evolution of online and blended course designs. Indeed, some recent ideas in this area highlight the possibilities opened by "mixable" elements (Caulfield, 2012b). In any case, it is expected that a process of mixing online resources will require greater attention to course design, perhaps resulting in certain kinds of coupling between online and in-class components, thus reducing perceptions of schism.

In general, the focus of this paper has been the design of in-class components to complement existing online components (a MOOC in this case study). However, the more typical perspective of designing online components to complement in-class components is equally important. In the case where the online components are MOOCs, however, the question becomes novel: how should MOOCs be designed to best take advantage of in-class component designs? More generally, how can MOOCs be best designed to best leverage differently designed local learning communities?

Finally, drawing on the findings of earlier studies (Aycock et al., 2002; Mehaffy, 2012), the authors believe that greater customization will also lead to a greater realization among instructors that they are members of instructional communities, further promoting open opportunities for collegiality and collaboration among instructors and across disciplines. As members of community, the authors expect that in many cases, instructors of wrapper courses will create and add content to the world's repository as well, which may in turn be picked up by students outside the wrapper. While Fisher did not create and contribute video himself for the Fall 2012 Machine Learning course, he has done so for other courses, and for his graduate course in artificial intelligence, he requires students to create and post content. Fisher (2012) reports that students taking a MOOC in artificial intelligence hosted by another institution visited his YouTube channel for clarification on some concepts, posting a link to the channel on the MOOC discussion board, bringing still other visitors.

While such data is anecdotal, it suggests the fascinating possibility for characterizing student and faculty interactions beyond any single MOOC, to include interactions across MOOCs and across media. Longitudinal studies for understanding the nature, extent, and evolution of ad hoc communities, perhaps MOOC centered but not restricted to a single MOOC, would undoubtedly require data mining across larger spheres of Web interactions than is currently easy to do. Nonetheless, the possibilities for understanding and leveraging student patterns in seeking remedial and advanced material, instructor incentives for creating and posting material, and the movement of people between student and teacher roles, are exciting and within current technical abilities – if only the data could be accessed.

References

Aycock, A., Garnham, C., & Kaleta, R. (2002). Lessons learned from the hybrid course project. Teaching with Technology Today, 8(6). Retrieved from http://www.uwsa.edu/ttt/articles/garnham2.htm

Babb, S., Stewart, C., & Johnson, R. (2010). Constructing communication in blended learning environments: Students' perceptions of good practice in hybrid courses. MERLOT Journal of Online Learning and Teaching, 6(4), 735-753. Retrieved from http://jolt.merlot.org/vol6no4/babb_1210.htm

Baldwin, R. G. (1998). Technology's impact on faculty life and work. New Directions for Teaching and Learning, 76, 7-21. doi:10.1002/tl.7601

Bowen, W. G., Chingos, M. M., Lack, K. A., & Nygren, T. I. (2012). Interactive learning online at public universities: Evidence from randomized trials. New York, NY: Ithaka S+R. Retrieved from http://www.sr.ithaka.org/sites/all/modules/contrib/pubdlcnt/pubdlcnt.php?file=http://www.sr.ithaka.org/sites/default/files/reports/sr-ithaka-interactive-learning-online-at-public-universities.pdf&nid=464

Caulfield, M. (2012a, November 9). How Coursera could walk the talk about MOOC-wrapping [Web log post]. Retrieved from http://mikecaulfield.com/2012/11/09/how-coursera-could-walk-the-talk-about-mooc-wrapping/

Caulfield, M. (2012b, December 11). Threads and the wrappable MOOC [Web log post]. Retrieved from http://www.hapgood.us/2012/12/11/threads-and-the-wrappable-mooc/

Domingos, P. (2013). Machine learning. Retrieved July 25, 2013, from https://www.coursera.org/course/machlearning

Fisher, D. H. (2012, November 6). Warming up to MOOC's. [Web log post]. Retrieved from http://www.chronicle.com/blogs/profhacker/warming-up-to-moocs/44022

Gilbert, J. A., & Flores-Zambada, R. (2011). Development and implementation of a "blended" teaching course environment. MERLOT Journal of Online Learning and Teaching, 7(2), 244-260. Retrieved from http://jolt.merlot.org/vol7no2/gilbert_0611.htm

Guthrie, K. M. (2012). Barriers to the adoption of online learning systems. EDUCAUSE Review, 47(4), 50-51. Retrieved from http://www.educause.edu/ero/article/barriers-adoption-online-learning-systems

Hill, P. (2012, November 1). Online educational delivery models: A descriptive view. EDUCAUSE Review, 47(6), 84-97.Retrieved from http://www.educause.edu/ero/article/online-educational-delivery-models-descriptive-view

Koller, D. (2012, November 7). How online courses can form a basis for on-campus teaching. Forbes. Retrieved from http://www.forbes.com/sites/coursera/2012/11/07/how-online-courses-can-form-a-basis-for-on-campus-teaching/

Kuhn, D. (1992). Thinking as argument. Harvard Educational Review, 62(2), 155-178.

Lage, M. J., Platt, G. J., & Treglia, M. (2000). Inverting the classroom: A gateway to creating an inclusive learning environment. The Journal of Economic Education, 31(1), 30-43. doi:10.1080/00220480009596759

Mangan, K. (2012, October 1). Massive excitement about online courses. The Chronicle of Higher Education. Retrieved from http://www.chronicle.com/article/Massive-Excitement-About/134678/

Mazur, E. (2009). Farewell, lecture? Science, 323(5910), 50-51. doi:10.1126/science.1168927

Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. (2010). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. Washington, DC: U.S. Department of Education, Office of Planning, Evaluation, and Policy Development. Retrieved from http://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf

Mehaffy, G. L. (2012). Challenge and change. EDUCAUSE Review, 47(5). Retrieved from http://www.educause.edu/ero/article/challenge-and-change

Pappano, L. (2012, November 2). The year of the MOOC. The New York Times, ED26. Retrieved from http://www.nytimes.com/2012/11/04/education/edlife/massive-open-online-courses-are-multiplying-at-a-rapid-pace.html

Rodriguez, M. A., & Anicete, R. C. R. (2010). Students' views of a mixed hybrid ecology course. MERLOT Journal of Online Learning and Teaching, 6(4), 791-798. Retrieved from http://jolt.merlot.org/vol6no4/rodriguez_1210.htm

Shirky, C. (2012, November 12). Napster, Udacity, and the academy [Web log post]. Retrieved from http://www.shirky.com/weblog/2012/11/napster-udacity-and-the-academy/

Stone, M. T., & Perumean-Chaney, S. (2011). The benefits of online teaching for traditional classroom pedagogy: A case study for improving face-to-face instruction. MERLOT Journal of Online Learning and Teaching, 7(3), 393-400. Retrieved from http://jolt.merlot.org/vol7no3/stone_0911.htm

Strauss, A. L., & Corbin, J. M. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory (2nd ed.). Thousand Oaks, CA: Sage.

Talbert, R. (2012, July 13). Screencasting for the inverted proofs class [Web log post]. Retrieved from http://www.chronicle.com/blognetwork/castingoutnines/2012/07/13/screencasting-for-the-inverted-proofs-class/

Toth, M. J., Amrein-Beardsley, A., & Foulger, T. S. (2010). Changing delivery methods, changing practices: Exploring instructional practices in face-to-face and hybrid courses. MERLOT Journal of Online Learning and Teaching, 6(3), 617-633. Retrieved from http://jolt.merlot.org/vol6no3/toth_0910.htm

Wiley, D. A., & Gurrell, S. (2009). A decade of development ... . Open Learning: The Journal of Open and Distance Learning, 24(1), 11-21. doi:10.1080/02680510802627746

Yin, R. K. (2003). Case study research: Design and methods (3rd ed.). Thousand Oaks, CA: Sage.

View the original article here