Showing posts with label Online. Show all posts
Showing posts with label Online. Show all posts

Sunday, 24 November 2013

Investigating Student Engagement in an Online Mathematics Course through Windows into Teaching and Learning

         Investigating Student Engagement in an Online Mathematics Course through Windows into Teaching and Learning


Teresa Petty

Associate Professor
Department of Middle, Secondary, and K-12 Education
University of North Carolina at Charlotte
Charlotte, NC 28223 USA
tmpetty@uncc.edu

Abiola A. Farinde
Graduate Research Assistant
Department of Middle, Secondary, and K-12 Education
University of North Carolina at Charlotte
Charlotte, NC 28223 USA
afarinde@uncc.edu

Abstract

The Windows into Teaching and Learning (WiTL) project is a method developed by researchers at the University of North Carolina at Charlotte that allows the facilitation of online clinical experiences for students during their content area methods course. WiTL was developed to address difficulties in securing meaningful clinical placements for interns during online summer coursework. WiTL encompasses both an asynchronous and a synchronous component. Through the use of dialogue, the purpose of WiTL is to engage interns with practicing classroom teachers and with each other as they consider various teaching methodologies and observe these methodologies in practice. The authors describe the WiTL process, its implementation, and ways in which the process has encouraged various levels of student engagement in an online mathematics methods course. The results indicate varying levels of student engagement and suggest that students are more engaged during the asynchronous portion of this study.

Keywords: online content methods courses, student engagement, distance learning, asynchronous online learning, synchronous online learning, online clinical experiences

Introduction

Researchers at one southeastern University were experiencing difficulties with clinical placements during online summer coursework. During a summer school session, the challenge to find either year-round schools or schools that have a later end of year so that methods students can be placed in a classroom to observe teaching practices is problematic. In the past, students were allowed to observe the behavior of adolescents in other settings such as summer camps and the Young Men's Christian Association (YMCA). These observations did not provide an ideal situation in which students could observe classroom teachers in their content area delivering instruction. In an effort to deliver meaningful learning experiences and bridge the gap between theory and practice, Windows into Teaching and Learning (WiTL) was conceptualized and implemented. WiTL offered online clinical observation experiences of practicing teachers in various grades and content areas.

The WiTL process utilized web-conferencing software supported by the University. Students were required to have access to a computer, the Internet, a headset, and a webcam to participate in WiTL activities. Students in methods courses observed the practices of teachers both synchronously and asynchronously. Following synchronous classroom observations, which were delivered via Wimba, methods students were given the opportunity to debrief with the practicing teachers, using a text chat feature in Wimba. Dialogue was exchanged regarding the pedagogical practices of the teachers and the rationale for using particular methodologies. Methods students also participated asynchronously by watching prerecorded videos of the practicing teachers. Following these viewings, they participated in an online threaded discussion, via NiceNet, with each other and the practicing teachers.

This research explored the engagement of university graduate students in an online middle grades/secondary mathematics methods course during summer clinical experiences. Student–student interactions (Moore, 1989) were closely examined through an analysis of the asynchronous threaded discussions and synchronous text chat communications using the engagement framework devised by Perkins and Murphy (2006). The examination revealed various levels of engagement among students in both the synchronous and asynchronous platforms. The guiding research question for this study was: To what extent do students in an online mathematics methods course engage in meaningful discourse and collaboration that encourages them to critically examine teacher pedagogy?

Literature Review

Teacher education programs, through the use of technology, offer online courses that seek to enhance student learning (Nandi, Hamilton, Harland, & Warburton, 2011). Of the many technology-based modes of instruction used in online teacher education courses, online discussion forums are often specifically used as a vehicle for the continual discussion of class topics and concepts. In addition, this learning environment, whether synchronous or asynchronous, facilitates the dissemination and acquisition of knowledge and enables student interaction and collaboration (Levine, 2007). Transforming the traditional learning context, these pedagogical tools allow greater access to classroom peers as well as a fluid exchange of content and resources with the intent of improving learning outcomes.

Although online teacher education courses are an innovative and unique pedagogical approach, their effectiveness is often questioned because student interaction in online discussion forums does not necessarily mean that students are actively engaged in the learning process (McLoughlin & Mynard, 2009; Robinson & Hullinger, 2008). In fact, solely fulfilling the specified participation and time requirements of an online course suggests that students are merely "doing time" rather than "doing education" (Zyngier, 2007). In doing time, students fail to go beyond the content, do not bridge theory and practice, and avoid authentic student engagement. Considering that deep learners are often more engaged than surface learners (Hockings, Cooke, Yamashita, McGinty, & Bowl, 2008) and that pre-service teachers' pedagogical training will influence the lives of their future students, engagement is a poignant topic for online teacher education courses.

When reviewing the literature, we found that at the collegiate level the definitions and measurement of online student-to-student engagement were limited. However, the literature did show that engagement is seen through numerous lenses (Zepke & Leach, 2010): student motivation (Schuetz, 2008), students' social and economic background (Pascarella & Terenzini, 2005), institutional support (Kuh et al., 2005), peer interaction (Moran & Gonyea, 2003), teacher behavior (Bryson & Hand, 2007), etc. Although engagement is often difficult to define and measure in online courses, its absence and presence is identifiable. Due to the study's focus on student-to-student engagement, for the purpose of this study engagement is defined by Kuh (2003) as the efforts of the student to study a subject, practice, obtain feedback, analyze, and solve problems. This definition illustrates that engagement cannot be captured through a snapshot but rather has an interpersonal component. Engagement is best observed through interactions with others and through a development of complex ideas. For instance, echoing this definition of engagement, Guthrie and Anderson (1999) state, "social interaction patterns in the classroom can amplify or constrict students' ... attainment of deep conceptual knowledge" (p. 20). Furthermore, within the confines of face-to-face (F2F) schooling, engagement is seen as a multidimensional concept: behavioral, cognitive, and emotional (Fredricks, Blumenfeld, & Paris, 2004). Behavioral engagement measures students' ability to follow school rules, their level of involvement in learning, and participation in extracurricular activity; cognitive engagement is students' investment in and active participation in learning as they move beyond required coursework; and emotional engagement is the positive and negative feelings students hold for school. Aligning – yet moving beyond – the aforementioned definitions, engagement in this study is measured in the context of students' interactions in an online mathematics methods course. Engagement can be identified in this particular learning environment as participants meaningfully contribute to online discussions. Within the asynchronous and synchronous forums, students commented, asked for clarification, posed questions, made inferences, assessed the learning environment, and collectively strategized about how to best bridge content and pedagogy. Their interactions with each other are the focal point of analysis and determine whether student-to-student engagement was present within the online discussions.

One of the challenges of online discussion forums is determining whether quality online engagement is actually occurring or whether students are passively interacting with their peers. In an era where more and more students are learning via online coursework (Murray, Pérez, Geist, & Hedrick, 2012), it is imperative that researchers determine if online learning can equally promote student engagement in comparison to a F2F learning environment. If student engagement is absent or minimized, then full cognitive development in a specific content area is unachievable. Because of this uncertainty regarding online engagement and due to the need to move beyond rote memorization of knowledge, an evaluation of online learning engagement levels must be performed to ensure positive learning outcomes for online students.

Synchronous (occurring in real time) and asynchronous (occurring over time) online discussion forums provide a space for open communication among students. Regardless of person or location, one's thoughts are transmitted to others, constructing new knowledge or further developing existing ideas. While both modes of communication serve a common purpose, asynchronous communication is often deemed more appropriate for facilitating active, meaningful engagement. Recognizing weakness in both communicative resources, Branon and Essex (2001) outline the disadvantages of both types of communication that may consequently hinder student engagement. They list the following restrictions when utilizing synchronous communication: (1) getting students online at the same time; (2) difficulty in moderating large-scale conversations; and (3) lack of reflection time. Similar to synchronous discussions, asynchronous communication equally places limitations on engagement due to the lack of immediate feedback, and the infrequency of students logging in to online discussion forums.

Students' level of engagement during synchronous and asynchronous communication is often challenged because students are not physically present in the same space, supposedly decreasing their opportunity to socially interact, collaborate, give and receive feedback, and render support (Tuckman, 2007). Although both modes of communication possess certain weaknesses, traditional F2F student communication does not guarantee meaningful student-to-student engagement. In fact, bodies in close proximity to one another, occupying a shared classroom, may promote interaction but may fail to facilitate active, engaged learning. Unlike traditional F2F student communication, there are numerous synchronous and asynchronous modes of communication, which are used to promote engagement. The diverse asynchronous (e-mail, listserv, discussion boards, and web logs) and synchronous (chat, instant messaging, and audio and web-based conferencing) modes of communication have the potential to increase interaction and improve online learning environments (Repman, Zinskie, & Carlson, 2005). Through the implementation of effective pedagogical learning principles delivered via online communication, a community of engaged, communicative learners may emerge (Peck, 2012).

As mentioned, although constraints are present in both communication tools, Schellens, Van Keer, and Valcke (2005) affirm that asynchronous discussion boards foster active engagement through a community of learners who teach one another and provide constructive feedback while sharing and gaining information. Im and Lee (2003) advocate the use of asynchronous over synchronous in generating a more effective learning environment. PytlikZillig et al. (2011) also compare synchronous F2F discussions and asynchronous computer-mediated discussions, asserting that computer-mediated discussions produce more effective engagement than F2F discussions.

Moreover, the flexibility and convenience of time, which allows for greater reflection and more collaboration among students, is often cited as a primary reason for why asynchronous discussion forums are a preferred learning resource (Ajayi, 2009; Nandi, Hamilton, & Harland, 2012). While extended time does not necessarily equate to student engagement, the opportunity to engage with others is greater, increasing the likelihood that such engagement may occur. The presence of this engagement, though logical, requires further empirical support. The need for more research on engagement within online forums (Su, Bonk, Magjuka, Liu, & Lee, 2005) reiterates the persistent question of whether students are actively engaged or passively participating. In seeking insight in this matter, we begin with the following research problem: Are students fully engaged in online synchronous and asynchronous courses?

Adding to the literature, this exploratory study examines synchronous and asynchronous student–student interactions (Moore, 1989) in an online middle grades and secondary mathematics methods course for graduate education students. Students' levels of engagement were coded using Perkins and Murphy's (2006) engagement framework. Perkins and Murphy's rubric measures individual engagement in critical thinking in an online asynchronous discussion. While this model was previously used in online asynchronous discussions, answering Perkins and Murphy's call for further research, this study sought additional empirical data by applying their model to asynchronous and synchronous discussions. Perkins and Murphy's model was deemed appropriate and applicable to asynchronous and synchronous modes of communication because different levels of engagement, which were illustrated by students' varying levels of critical-thinking questions and comments, were present in both discussion forums. Identification and measurement of student engagement in critical thinking in both settings revealed that regardless of the online medium, the implementation of sound, student-centered pedagogical practices will foster student engagement. Table 1 provides a detailed classification of the four engagement categories.

Table 1. Perkins and Murphy's (2006) model for identifying engagement in critical thinking

Clarification:
All aspects of stating, clarifying, describing (but not explaining), or defining the issue being discussed.Analyzes, negotiates, or discusses the meaning of the issue.
Identifies one or more underlying assumptions in a statement in the discussion.Identifies relationships among the statements or assumptions.Defines or criticizes the definition of relevant terms. Assessment:
Evaluating some aspect of the debate; making judgments on a situation, proposing evidence for an argument or for links with other issues.Provides or asks for reasons that proffered evidence is valid.Provides or asks for reasons that proffered evidence is relevant.Specifies assessment criteria, such as the credibility of the source. Makes a value judgment on the assessment criteria or a situation or topic.Gives evidence for choice of assessment criteria.Inference:
Showing connections among ideas; drawing appropriate conclusions by deduction or induction, generalizing, explaining (but not describing), and hypothesizing.Deduces relationships among ideas.Strategies:
Proposing, discussing, or evaluating possible actions.Predicts outcomes of proposed actions.Method

Project Description and Participants

The primary goal of WiTL was to provide a meaningful summer clinical experience for students enrolled in an online mathematics methods course. Data were collected from all students (n = 22) enrolled in the online summer mathematics methods course at the researchers' University. Researchers utilized a middle school and a high school based on faculty connections in working within the school and community. The three teachers of middle grades that were invited to participate in this project were identified by their principal as exemplary teachers. All three middle grade levels were represented: one teacher was a sixth-grade mathematics teacher, one a seventh-grade mathematics teacher, and one an eighth-grade mathematics teacher. The three high school teachers that were asked to join were also acknowledged by their principal as exemplary teachers. The three teachers collectively taught Algebra I, Algebra II, Geometry, Pre-Calculus, and Advanced Placement Calculus. Prior to the beginning of the summer methods course, each of the practicing teachers selected two exceptional lessons. These lessons were then videotaped during regular classroom instruction using a laptop, wireless headset, webcam, and TechSmith's Camtasia, a software application used for screen video capture.

The methods students viewed these 12 asynchronous videos (two per teacher) and then participated in an online dialogue regarding the practices they observed. These threaded discussions, facilitated through NiceNet, occurred over two weeks and allowed methods students to engage in conversation with the six practicing mathematics teachers and their peers. The conversations (examples provided in data analysis section) included dialogue between students regarding instructional methods and pedagogical thinking concerning the teaching and learning of mathematics for Grades 6 to 12.

The three middle school and three high school mathematics teachers were also asked to allow online mathematics methods students to participate in a live teaching observation. These synchronous teaching observations were facilitated using a computer connected to the Internet, a webcam, a wireless headset, and Saba Centra (the University-supported web conferencing software). These items were chosen because all online cohort students utilize these technological tools throughout their program coursework. Methods students logged on to Centra at the indicated class time to observe the teachers. They viewed six separate teaching episodes and participated in a text chat dialogue with their classmates during the observations. This viewing and interaction allowed students to ask one another questions regarding the classroom environment, classroom management, instructional methods, and pedagogical thinking.

Following the teaching observation, the classroom teacher joined the methods students in Centra to take comments about their teaching and answer questions regarding instructional decision making. Examples are provided in the Data Analysis section.

WiTL was a multifaceted project, and therefore, generated several data sources. These included individual interviews with each classroom mathematics teacher, focus group interviews with methods students at the conclusion of the semester, copies of the asynchronous threaded discussion forum, text chat logs of methods students conversations during the synchronous teaching observations, and archives of both the synchronous and asynchronous sessions. The data essential to this present study are the asynchronous threaded discussions, facilitated through NiceNet, and the text chat logs, facilitated during the synchronous teaching observations via Centra. The following research question guided the analysis of data:

To what extent do students in an online mathematics methods course engage in meaningful discourse and collaboration that encourage them to critically examine teacher pedagogy?

Data Analysis

The threaded discussions were analyzed to determine various levels of student-to-student engagement during online clinical experiences. Researchers used content analysis to quantify levels of engagement based on students' questions and comments in the threaded discussions. Students' levels of engagement with each other were coded using Perkins and Murphy's (2006) engagement framework presented in Table 1. Various levels of analysis occurred. Initially, the researchers evaluated the data independently, determining which level of engagement was represented in the dialogue. Next, the researchers compared their analyses and determined inconsistencies. Finally, inconsistencies were discussed until final consensus was reached. This method of analysis allowed the researchers to establish inter-rater reliability (Neuendorf, 2002). Frequencies for each category were then tabulated.

The text chat logs, generated during the synchronous teaching observations, were analyzed in a similar manner using content analysis. Again, students' levels of engagement with each other were coded using Perkins and Murphy's (2006) engagement framework (see Table 1). Researchers conducted an independent analysis of the data to decide which level of engagement was represented in the various comments and questions stated during text chat sessions. Researchers then compared their analyses to determine inconsistencies. Once more, inconsistencies were debated until consensus was reached, allowing for the establishment of intercoder reliability (Neuendorf, 2002). Frequencies for each category were then recorded.

Results

After the threaded discussions and text chat logs were analyzed, relative frequencies were tabulated. These data are presented in Table 2.

The data, although not conclusive, indicate varying levels of student engagement across both synchronous and asynchronous platforms. This distribution of student outputs across the engagement categories yields interesting results. The asynchronous forum postings show a higher percentage of Strategy engagement (15.2% vs. 7.9%) as well as a higher percentage of Clarification engagement (61% vs. 45.7%). The synchronous forum postings indicate something different. The categories of Assessment (35.3% vs. 17.5 %) and Inference (11.1% vs. 6.3%) indicate higher levels of engagement for the synchronous platform.

Table 2. Frequency of student output – synchronous vs. asynchronous

Asynchronous
(Threaded Discussion) It is important to note that the threaded discussion forum was available to students and classroom teachers for two weeks, while the text chat logs were offered during six synchronous teaching observations with durations of 75 to 90 minutes each. All four categories of engagement were observed in both the threaded discussions and text chat logs. Students in both settings engaged more at the level of clarification. This was apparent as students asked questions and made comments regarding the classroom context (i.e., configuration of the desks/classroom, decoration of the classroom, technology available to students, academic levels of students, etc.). As the level of engagement increased on Perkins and Murphy's (2006) engagement framework, the relative frequency of students' comments and questions decreased with the exception of the Strategies category. This could be warranted due to the time required for students to reflect on teachers' actions and offer possible justifications. All four categories were observed, but as noted above, engagement cannot be solely measured by frequency (doing time). A critical examination of the actual content presented by students during the threaded discussions and text chat dialogues must also be considered when determining students' engagement level.

The data presents examples of authentic student engagement. For example, the following are distinct comments and statements posted by one student and responded to by another student during the asynchronous online discussion forum (NiceNet). In this exchange students share questions they still have about slope and direct variation and suggestions for teaching systems of equations. This exchange reinforces the notion that students can learn important teaching strategies from one another when they have the opportunity to discuss their ideas. The level of engagement based on Perkins and Murphy's (2006) framework is indicated [in brackets, and rendered in bold].

Student 1: "I really like how you [the teacher] make the association between slope and direct variation because that is a topic that my ninth-grade Algebra I students have a hard time with. Do you have any other suggestions on how to make it clearer? [Strategies] Also why did they change the variable to "k" instead of leaving it as "m"? [Clarification] I am a math major (not an education major) and I still am not sure why the variables in direct variation and slope intercept changed. I know it is a little late in the year for this stuff and you may have done this before, but there is a cool activity to do with solving systems. You break the students in groups and have them do an advertisement for a particular method. I make my students show examples, write a definition, make a logo, a slogan and a valid argument as to why their method is better than all the others. I also make them present it in front of the class. They usually have fun with it. [Strategies]"

Student 2: "I am also a math major and have never understood why the variable changed. I look forward to the teacher's answer! [Clarification] Also, thanks for sharing that method! Systems of equations was the topic I struggled the most with teaching this year and am concentrating on to make it better for next year. I love your project idea and am definitely going to implement that into my classroom! [Assessment]"

The exchange below is also taken from the NiceNet threaded discussion. It exemplifies quality engagement during student-to-student interaction. The students really push each other to think about the topic of fractions and consider various situations in which fractions are used. Both students are making connections, providing examples, going beyond the content, and actively contributing to the discussion.

Student 3: "When in real life do we ever really use fractions? I can think of obvious ones: slices of pizza, talking about discounts and sales tax, but other than those, I'm not sure I really know if we DO use fractions. [Assessment]"

Student 4: "I would argue that we use fractions every time we have to use division. If you look at a class and want to know what percentage of the students did their homework, you are going to have to collect the raw data first. If 16 out of 20 students did their homework, that is a fraction, 16/20. After you know that fraction, you are able divide and determine what that number is as a decimal and percentage. I agree that numbers are not usually left as fractions, and that fractions do not always look as polished as decimals do, but they are something that is used in many people's daily lives. [Assessment]"

Student 3: "Thank you for pushing my thinking. However, if we are using a fraction to solve for a percent on a test, wouldn't we convert it to a decimal to understand it better? Maybe it is the way I think and not the way the world works, but I understand 80% much better than I understand 16/20 even though they represent the same amount. Also, will we ever have to multiply, divide, add, or subtract fractions in real life? I absolutely agree that students need to be able to convert fractions to decimals to percents and vice versa, but will it hurt them to use a calculator to always compute with fractions (as in adding, subtracting, multiplying, and dividing)? I am not sure I know the answer to that question ... [Assessment]"

The following are individual comments made by students to students during the synchronous teaching observations via the text chat feature in Centra. In this exchange, students are discussing the note taking methods used by the teacher they are observing. This exchange reveals the ideas that the students have regarding this technique. The level of engagement based on Perkins and Murphy's (2006) framework is indicated.

Student 1: "How structured do you think note taking should be? Right now, they're basically writing exactly what he dictates. [Clarification]"

Student 2: "I think it will depend on my students ... may have to feel it out ... [Assessment]"

Student 3: "I think that note taking is a skill that has to be explicitly taught. [Assessment]"

Student 4: "I agree that it has to be taught, but you can't learn if you always are just copying something ... [Assessment] Maybe this is a good method for younger students who are still in the beginning stages, though. [Inference]"

The exchange below is also taken from the text chat log. During this exchange, students are observing a teacher-centered classroom. The teacher is initiating traditional instructional methods. The students are offering suggestions for getting the middle school students motivated as many of them are visibly disengaged.

Student 1: "Maybe some cooperative learning activities and more game type learning would motivate them a little more. Make the students think they're playing instead of learning. [Strategies]"

Student 2: "They will think it is a reward for passing the test. [Strategies]"

Student 3: "The teacher should consider maybe not teaching new information but readdressing what they did not get. [Strategies]"

The comments/questions presented above demonstrate the various levels of engagement that students exhibited as they participated in both the threaded discussions and the text chat dialogues. Students discuss a variety of issues that they see as they observe teaching practices via synchronous and asynchronous observations.

Limitations

As with most research studies, this study does have its limitations. The most apparent limitation is the small sample size, making the results difficult to generalize. The study was conducted over a short period of time – one semester during the summer. In the course of this study, there were technical difficulties. During two of the synchronous teaching observations, the teachers had difficulty with their wireless headsets, which resulted in periods of no sound for the online methods students who were viewing the teaching observations. This issue was quickly resolved. Another possible limitation relates to the asynchronous threaded discussion. Since this is a monitored forum, students may have felt more inclined to participate and to provide thoughtful responses.

The use of the Perkins and Murphy (2006) framework could also be viewed as a limitation. It was first developed for use with the asynchronous platform. This study took it a step further and utilized it with both asynchronous and synchronous platforms.

Conclusion

In this study, the researchers explored the guiding research question: To what extent do students in an online mathematics methods course engage in meaningful discourse and collaboration that encourage them to critically examine teacher pedagogy? It was determined that students engage in all levels of engagement as defined by Perkins and Murphy (2006) in both asynchronous and synchronous platforms. While asynchronous forum postings show a higher percentage of Strategy engagement (15.2% vs. 7.9%) as well as a higher percentage of Clarification engagement (61% vs. 45.7%), the synchronous forum postings indicate something different. The categories of Assessment engagement (35.3% vs. 17.5 %) and Inference engagement (11.1% vs. 6.3%) presented higher levels of engagement for the synchronous platform. The settings of the individual platforms would certainly encourage this. In the asynchronous environment, students would have more time to reflect and therefore offer strategies. Clarification engagement may be more present in the asynchronous platform because students in the synchronous platform were afforded opportunities to get clarification to these questions during the synchronous settings that were not readily available in the asynchronous setting. This notable constraint does differ from the literature, which asserts that asynchronous is preferable. The results of the WiTL study suggest that the asynchronous and synchronous platforms are emphasizing different types of engagement. Future research studies are needed to determine which method is more beneficial, or if one method is better at engaging students in an online environment.

WiTL proved to be a beneficial experience for methods students. It offered outreach to students across the state through an innovative approach to clinical experiences. WiTL provided the online methods students not only the opportunity to view sound teaching practices, it also allowed them the chance to engage with practicing mathematics teachers to confirm their understandings of methodologies and pedagogical decision making. During a typical clinical observation experience, a student would be placed with one teacher and one classroom. In this online mathematics methods course, students were placed in different classrooms across a variety of schools. WiTL was a unique experience in that it allowed online methods students to observe six different teachers in two different schools and school systems, exposing them to an assortment of teaching styles in a variety of classroom environments. Since WiTL allowed the online methods students to observe the same teachers, an exchange of thoughts and ideas related to the instructional strategies they observed was also facilitated. The WiTL process was equally valuable to the practicing mathematics teachers as it allowed them the opportunity to critically reflect on their teaching practices as they considered questions that the methods students posed. Through WiTL, methods students were able to utilize technology to engage in online clinical experiences at various levels with their peers.

References

Ajayi, L. (2009). An exploration of pre-service teachers' perceptions of learning to teach while using asynchronous discussion board. Educational Technology & Society, 12(2), 86-100. Retrieved from http://www.ifets.info/journals/12_2/7.pdf

Branon, R. F., & Essex, C. (2001). Synchronous and asynchronous communication tools in distance education. TechTrends, 45(1), 36-42. doi:10.1007/BF02763377

Bryson, C., & Hand, L. (2007). The role of engagement in inspiring teaching and learning. Innovations in Education and Teaching International, 44(4), 349-362. doi:10.1080/14703290701602748

Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research. 74(1), 59-109. doi:10.3102/00346543074001059

Guthrie, J. T., & Anderson, E. (1999). Engagement in reading: Processes of motivated, strategic, knowledgeable, social readers. In J. T. Guthrie & D. E. Alvermann (Eds.), Engaged reading: Processes, practices, and policy implications (pp. 17-45). New York, NY: Teachers College Press.

Hockings, C., Cooke, S., Yamashita, H., McGinty, S., & Bowl, M. (2008). Switched off? A study of disengagement among computing students at two universities. Research Papers in Education, 23(2), 191-201. doi:10.1080/02671520802048729

Im, Y., & Lee, O. (2003). Pedagogical implications of online discussion for preservice teacher training. Journal of Research on Technology in Education, 36(2), 155-170.

Kuh, G. D. (2003). What we're learning about student engagement from NSSE: Benchmarks for effective educational practices. Change: The Magazine of Higher Learning, 35(2), 24-32. doi:10.1080/00091380309604090

Kuh. G. D., Kinzie, J., Schuh, J. H., & Whitt, E. J., & Associates. (2005). Student success in college: Creating conditions that matter. San Francisco, CA: Jossey-Bass.

Levine, S. (2007). The online discussion board. New Directions for Adult and Continuing Education, 113, 67-74. doi:10.1002/ace.248

McLoughlin, D., & Mynard, J. (2009). An analysis of higher order thinking in online discussions. Innovations in Education and Teaching International, 46(2), 147-160. doi:10.1080/14703290902843778

Moore, M. G. (1989). Three types of interaction. American Journal of Distance Education, 3(2), 1-6. doi:10.1080/08923648909526659

Moran, E. T., & Gonyea, T. (2003). The influence of academically-focused peer interaction on college students' development. Available from ERIC database. (ED478773)

Murray, M., Pérez, J., Geist, D., & Hedrick, A. (2012). Student interaction with online course content: Build it and they might come. Journal of Information Technology Education: Research, 11, 125-140. Retrieved from http://www.jite.org/documents/Vol11/JITEv11p125-140Murray1095.pdf

Nandi, D., Hamilton, M., & Harland, J. (2012). Evaluating the quality of interaction in asynchronous discussion forums in fully online courses. Distance Education, 33(1), 5-30. doi:10.1080/01587919.2012.667957

Nandi, D., Hamilton, M., Harland, J., & Warburton, G. (2011). How active are students in online discussion forums. In J. Hamer & M. de Raadt (Eds.), Proceedings of the Thirteenth Australasian Computing Education Conference: Vol. 114 Conferences in Research and Practice in Information Technology (pp. 125-134). Darlinghurst, Australia: Australian Computer Society. Retrieved from http://www.crpit.com/confpapers/CRPITV114Nandi.pdf

Neuendorf, K. A. (2002). The content analysis guidebook. Thousand Oaks, CA: Sage.

Pascarella, E. T., & Terenzini, P. T. (2005). How college affects students: A third decade of research. San Francisco, CA: Jossey-Bass.

Peck, J. J. (2012). Keeping it social: Engaging students online and in class. Asian Social Science, 8(14), 81-90. doi:10.5539/ass.v8n14p81

Perkins, C., & Murphy, E. (2006). Identifying and measuring individual engagement in critical thinking in online discussions: An exploratory case study. Educational Technology & Society, 9(1), 298-307. Retrieved from http://www.ifets.info/journals/9_1/24.pdf

PytlikZillig, L. M., Horn, C. A., Bruning, R., Bell, S., Liu, X., Siwatu, K. O., ... Carlson, D. (2011). Face-to-face versus computer-mediated discussion of teaching cases: Impacts on preservice teachers' engagement, critical analyses, and self-efficacy. Contemporary Educational Psychology, 36(4), 302-312. doi:10.1016/j.cedpsych.2011.07.002

Repman, J., Zinskie, C., & Carlson, R. D. (2005). Effective use of CMC tools in interactive online learning. Computers in the Schools, 22(1-2), 57-69. doi:10.1300/J025v22n01_06

Robinson, C. C., & Hullinger, H. (2008). New benchmarks in higher education: Student engagement in online learning. Journal of Education for Business, 84(2), 101-109. doi:10.3200/JOEB.84.2.101-109

Schellens, T., Van Keer, H., & Valcke, M. (2005). The impact of role assignment on knowledge construction in asynchronous discussion groups: A multilevel analysis. Small Group Research, 36(6), 704–745. doi:10.1177/1046496405281771

Schuetz, P. (2008). A theory-driven model of community college student engagement. Community College Journal of Research and Practice, 32(4-6), 305-324. doi:10.1080/10668920701884349

Su, B., Bonk, C. J., Magjuka, R. J., Liu, X., & Lee, S.-H. (2005). The importance of interaction in web-based education: A program-level case study of online MBA courses. Journal of Interactive Online Learning, 4(1), 1-19. Retrieved from http://www.ncolr.org/jiol/issues/pdf/4.1.1.pdf

Tuckman, B. W. (2007). The effect of motivational scaffolding on procrastinators' distance learning outcomes. Computers & Education, 49(2), 414-422. doi:10.1016/j.compedu.2005.10.002

Zepke, N., & Leach, L. (2010). Improving student engagement: Ten proposals for action. Active Learning in Higher Education, 11(3), 167-77. doi:10.1177/1469787410379680

Zyngier, D. (2007). (Re)conceptualising student engagement: Doing education not doing time. Teaching and Teacher Education, 24(7), 1765-1776. doi:10.1016/j.tate.2007.09.004


View the original article here

Learner Participation and Engagement in Open Online Courses: Insights from the Peer 2 Peer University

         Learner Participation and Engagement in Open Online Courses:
Insights from the Peer 2 Peer University


June Ahn

Assistant Professor
College of Information Studies and College of Education
University of Maryland, College Park
College Park, MD 20740 USA
juneahn@umd.edu

Brian S. Butler
Associate Professor
College of Information Studies and Robert H. Smith School of Business
University of Maryland, College Park
College Park, MD 20740 USA
bsbutler@umd.edu

Alisha Alam
Graduate Research Assistant
College of Information Studies
University of Maryland, College Park
College Park, MD 20740 USA
alisha@umd.edu

Sarah A. Webster
Graduate Research Assistant
College of Information Studies
University of Maryland, College Park
College Park, MD 20740 USA
websters@umd.edu

Abstract

Recent developments in massive open online courses (MOOCs) have brought renewed attention to online learning, but most of the attention has been centered on courses offered through platforms such as Coursera and edX. This paper focuses on the importance of examining alternative, large-scale learning initiatives that promote more participatory modes of education production and delivery. It presents a comprehensive description of the Peer 2 Peer University (P2PU), a social computing platform that promotes peer-created, peer-led, online learning environments. Using log data from the P2PU platform, the ecosystem of this learner-generated education platform is described. The descriptive analysis highlights P2PU's growth in terms of the participatory learning environments that have been created – such as online study groups, courses, and challenges – and also describes the participation patterns of P2PU members. This paper provides one of the first empirical descriptions of an emerging open learning platform and illuminates how log data from the platform, particularly in relation to open courses, open badges, and learning tasks embedded in courses, can be used to track development of courses and engagement in learning activities across P2PU. The analyses reported here are aimed at helping researchers understand the P2PU ecosystem and identify potential areas for future study as P2PU works to open its data for public analysis.

Keywords: massive open online course (MOOC), open education, peer-to-peer learning, participatory learning, crowdsourced education

Introduction

While recent platforms for massive open online courses (MOOCs) such as Coursera and edX have garnered substantial public attention, MOOCs were originally conceived of as participatory environments where groups of learners (from small to massive) collaborate to aggregate self-created learning experiences (McAuley, Stewart, Siemens, & Cormier, 2010). In this model of peer-based, open learning, a course might organize topics and guide the schedule. However, individual learners coordinate the learning activities by contributing blog posts, links to resources, and other media. This model brings to the foreground a bottom-up, crowd-generated approach to delivering online education versus the top-down approach implemented in popular platforms. Yet in spite of their differences, learner participation and engagement is a concern in both types of MOOCs (Waters, 2012).

At a basic level, any learning platform requires participant engagement to achieve its goals. Individuals must participate in the provided activities in order to have the learning experiences that are the focus of the platform. Beyond this basic form of participation, crowd-generated learning platforms require learners to engage in many other activities such as creating courses and learning materials, joining online courses, contributing comments and discussion, and persisting in the group's learning activities. Creating a participatory MOOC environment requires both more coordinated work (Butler & Ahn, 2013) and more types of participation than top-down MOOCs that rely primarily on individual learner engagement (Ahn, Weng, & Butler, 2013; Kafai & Peppler, 2011). However, in spite of its importance, there has been little empirical work done on conceptualizing and measuring participation in large scale, open learning platforms.

In the following study, the patterns of participation and engagement present in a prominent participatory, open online learning community, the Peer 2 Peer University (P2PU), are examined. An outline is first provided of how P2PU is an example of a peer-generated open online course platform that speaks to the original notion of participatory MOOCs. Then, statistics derived from a comprehensive dataset about P2PU courses and participant activity are presented, with the aim of exploring the question of how learners have participated and engaged with open online courses in P2PU.

The data in this study describe the course ecology that exists on the P2PU platform using metrics such as the number of member-created courses, types of courses, patterns of course enrollment, and active contributions to these peer-to-peer courses over time. A descriptive analysis of the diverse learning environments that have arisen in this participatory, open education platform is also offered.

This paper makes several contributions to research on open online learning communities. Recent developments in top-down MOOCs (Coursera, edX, etc.) have brought renewed energy and attention to online learning. It highlights the importance of examining alternative, and equally exciting, open learning initiatives such as P2PU. This paper provides the first empirical descriptions of an emerging open learning platform, illuminating how log data from P2PU, particularly in relation to open courses, open badges, and learning tasks embedded in courses, can be used to track the development of courses and engagement in learning activities across P2PU. Different flavors of online education are developing at a rapid pace, bringing to the fore numerous research questions and concerns. The descriptions presented in this paper are aimed to help researchers understand the P2PU ecosystem and identify potential areas for future study as P2PU works to open its data for public analysis.

Related Work

Understanding the different definitions of the term MOOC is vital to develop a nuanced understanding of how educational experiences are developed and delivered in various online platforms. Recent, high profile MOOC examples such as Coursera, edX, and Udacity represent what some term "xMOOCs," with their institutional, top-down, content-delivery driven models (Siemens, 2012). Courses in these platforms are typically designed as a weekly syllabus of video lectures followed by quizzes or other assignments that evaluate whether students understood the content. Discussion boards are also often provided for students to ask questions and seek clarification on information related to the course. These courses mirror the structure of what was seen in most online education platforms. The major innovations have been the ability of these platforms to curate and freely deliver content from elite universities and garner "massive" enrollments of thousands and tens of thousands of learners.

In contrast to this traditional model of teaching and learning is what educators have referred to as connectivist MOOCs or "cMOOCs" (McAuley et al., 2010; Siemens, 2012). This original form of MOOC was viewed as an experiment that leveraged the distributed, networked information available on the Internet with the learners themselves generating most of the learning activities. Learners might write their own reflections about what they are learning on blogs and other social media platforms, share these back with their peers, create social networks with others in the MOOC, share information, and aggregate their ideas to create an emergent learning experience.

This form of collective learning differs from the xMOOC model in several significant ways (Kafai & Peppler, 2011). First, the modes of production differ dramatically. In xMOOCs, elite institutions such as Stanford University, Harvard University, Massachusetts Institute of Technology, and partner universities develop courses. In cMOOCs, the underlying philosophy is that any individual should be empowered to design a learning experience. Second, course implementation and the role of learners are conceptualized very differently. In an xMOOC, learner responsibilities focus on consuming the course content and completing evaluations to assess understanding of that content. In a cMOOC, learners may utilize the course-provided content, but as one resource among many that they may access through their own online searching and research. Learners take on the dual role of learners and teachers, as they help their peers through the learning process they are also undertaking. In a cMOOC, the content of the course is user-generated and emergent, arising from the persistent contributions of the learners themselves. Leadership is distributed and the learning experience is severely limited if there are only a few students generating ideas and information.

Connectivist or collectivist MOOC environments build on broader trends in computing and technology-mediated social interaction. These versions of MOOCs leverage the capabilities of social computing, where networked media and technology enable individuals to easily share information, communicate with others, and drive online content-creation (Parameswaran & Whinston, 2007). Thus, instead of treating learning as information transfer from instructor to student, learning is conceptualized as arising from social interaction and information sharing via networked tools. A collectivist philosophy of online education also represents a particular instantiation of crowdsourcing. The term crowdsourcing describes the use of technology to outsource a task typically done by a particular agent or person by distributing the task to a large group of people to do collectively (Howe, 2006). In the MOOC context, one could observe the process of course creation as a crowdsourcing example. In xMOOC platforms, course creation is crowdsourced, in a sense, to individual universities and their instructors, but aggregated in a central platform. Yet at a micro-level, courses are still designed and created by a single source, the instructor. In a cMOOC context, the course content and design are further crowdsourced to the learners themselves. Rather than one source of information and content, learners form online social networks in which they share and aggregate content across multiple channels. Through the use of social computing platforms, education could potentially be a crowdsourced endeavor, where learners create, manage, and collectively implement large-scale, online courses.

The Peer 2 Peer University

P2PU is non-profit organization and online platform that began in 2009. The online community promotes experimentation in open learning and peer-led education. The platform allows any member to design and create an educational course, which can then be taken by any other member in the online community. In P2PU, learning environments come in a variety of formats (with courses being only one instantiation). The general term used in P2PU is learning "project," and this term will be used in the present paper to speak generally to different P2PU learning situations. More specific terms refer to particular learning environments. In P2PU, members can create study groups to bring together learners around topics of interest for a set period of time. Members can also create more formal courses that are structurally similar to study groups. The most recent iteration of P2PU adds challenges, which are asynchronous courses that remain persistent and act as guides for learners through an educational experience (rather than being time-bound). Learners in P2PU can join, complete, and leave challenges at any moment. P2PU has also introduced new forms of collective assessment through open badges. Learners can earn badges associated with different learning tasks and courses, which tie into the Mozilla Open Badges framework (The Mozilla Foundation & P2PU, 2012). Study groups, courses, challenges, and badges are all important constructs that P2PU provides to organize peer-generated learning experiences, and each will be considered in the analysis presented below.

In many ways, P2PU is a social computing platform that crowdsources the creation and implementation of informal online education. Instead of formal teachers or established institutions acting as the sole developers of online courses, any stakeholder can create a course in P2PU. In addition, the mission and ethos of the P2PU community encourages a social process of learning. Courses are not designed merely as content delivery mechanisms, but as spaces that ask members to engage in active learning projects, share their progress and resources, start and participate in discussions, and collaborate to gain knowledge of a topic area. Similarly, P2PU's experiments with open badges are a way to observe how social assessments of learning can be recognized and credited in open, informal learning environments. As a crowdsourced education platform, the success of the community is critically dependent on each member's active and persistent participation.

In this study, the authors sought to better understand the notions of learner participation and engagement in open online courses, and explore how researchers could observe these activities using log and trace data from P2PU. The goal was to take the raw data provided by P2PU, and use descriptive analytics to understand the broader ecosystem of the P2PU community. The main research question guiding this study was:

How have learners participated and engaged with open online learning in P2PU?

In a participatory, open education setting like P2PU, the learning environments themselves are not givens. There is a substantial amount of member effort needed to create the basic learning contexts (e.g., courses and challenges) and assessments (e.g., badges) required for the platform to function. As Butler and Ahn (2013) observe, a great amount of cooperative work and coordination is needed to create materials, mobilize peers, and collaborate in productive ways, before learning can even occur. Thus, the descriptive analysis of the P2PU log data began with an exploration of exploring questions such as:

How many courses and challenges comprise the P2PU community?What patterns of participation can be observed in these learning contexts?What are the characteristics of learning environments within P2PU?How can learners' participation and engagement in these contexts be measured?These descriptive analyses of the P2PU community can then inform future researchers seeking to explore complex relationships between learning context characteristics and learner participation, and engagement.

Method

The study reported here was part of an ongoing partnership with P2PU, in which the main goals are to: (1) create policies and processes to publicly share datasets of the P2PU platform; and (2) conduct analyses of this widely used open education platform to shed light on the challenges of supporting large-scale, participatory, online learning. The project includes substantial data work to transform raw P2PU log data into usable variables and datasets for researchers in education and other related fields. In addition, the research team is exploring various learning analytics techniques for illuminating the learning processes that occur in this online community (Bienkowski, Feng, & Means, 2012).

In October 2012, P2PU provided the research team with a raw dataset containing a complete record of all elements of the platform (courses, challenges, users, learning activities, etc.). This dataset covered the entire history of P2PU from its beginning in late 2009 to October 2012. The data was a copy of the back-end mySQL database that is the operational foundation for P2PU.org and consisted of 82 tables. In an initial phase of data exploration, the research team developed a high-level mapping of the tables that corresponded to the primary components of the P2PU platform and their relationships to one another (Figure 1). This study utilized log data from those tables that described P2PU Schools, Projects (e.g., the different types of learning contexts), Badges, Users, and user Content contributions.


Figure 1. High-level description of the P2PU relational database structure

Although the database theoretically provided a rich, fine-grained record of P2PU schools, projects, users, their activity, and badges, the raw log data was structured in terms of operational transactions and entity status records which did not readily correspond to the constructs most likely to be of interest to researchers and analysts. To overcome this mismatch the research team first conducted a phase of data exploration and descriptive analytics to construct a holistic understanding of various levels of the P2PU community. Then on this basis, a project-level dataset was created that characterized the nature, history, and activity associated with every project (e.g., study group, course, challenge) ever created on the P2PU platform. This analysis dataset included a number of variables for each project, which are described in turn below.

Project name: Name of the P2PU project. Language: What language the project uses (e.g., English, Spanish). Project category: What type of project is it (study group, course, or challenge)? Under development: A binary flag that designates whether a given project is under development or "live" and available to the public. Deleted: A binary flag designating whether a project has been deleted. Archived: A binary flag designating whether a project is no longer publicly available and has been archived. Project pages: A count of how many website pages have been created for a given project. Organizer count: P2PU project may have one or more creators or leaders (known as organizers). This variable indicates how many unique organizers are associated with a project. Participant count: P2PU Members can join study groups and courses as active "participants." This variable is a count of how many members have joined and self-designated as participants. Follower count: Members can also join study groups and courses as "followers," indicating that they will not actively participate but are interested in receiving notices about activity within the course. This variable is a count of how many members have joined and self-designated as followers. Total number of tasks: Each challenge consists of a series of tasks to be completed by learners. As they complete them, members record their progress by checking off each task. This variable indicates how many tasks are associated with each challenge. Adopter count: In challenges, members can join as active participants by working on tasks in order to complete the challenge. This variable indicates how many members begun working on the tasks associated with a challenge. Completers: In challenges, members can check off tasks as they work to achieve the challenge. When all the tasks are marked as completed, the challenge is said to have been completed. This variable is a count of how many members have completed a challenge. Average tasks completed: While some individuals complete all the tasks associated with a challenge, many do not. This variable captures the average number of tasks completed by the adopters of each challenge, providing a metric of the overall progress of the members participating in it. Total number of badges: Some challenges and courses are associated with badges that members can earn if they complete certain tasks. This variable is a count of how many badges were associated with a given project. Average badges earned: As with tasks, individuals can vary with respect to how far they have progressed with respect to the badges associated with a project. This variable captures the average number of badges completed (from those associated with the project) among the participating members.These variables were constructed based on an examination of the P2PU interface, consultation with P2PU developers, and detailed analysis of the P2PU platform code, which is available through an open source license. The constructed measures were also validated with SQL queries to multiple combinations of data tables of P2PU's backend database.

Findings

The P2PU Ecosystem

To explore how learners have participated and engaged with open online learning in P2PU several platform-level metrics that provided a glimpse into P2PU's history, its ecosystem of learning projects, and members in the community were examined. The primary component of the P2PU learning ecosystem is the project. Projects are bounded collections of dynamic and static content, organizers, participants, and infrastructure. There were 2,034 project records in the P2PU database that are broken down into three different learning environment types: 1,135 study groups, 506 courses, and 393 challenges.

Study groups bring together groups of informal learners together around educational tasks and discussions. Courses are similar in structure to study groups, but represent more formal learning projects that are open to anyone in P2PU. Courses organize around tasks that learners complete (and are typically structured as weekly tasks), in addition to threaded discussion pages where learners can participate in dialogue and share thoughts. In the recent iteration of P2PU, another learning environment called challenges was introduced. Challenges utilize a similar structure as courses but are asynchronous and persistent (whereas courses ran for a set period of time and learners dispersed). A challenge is a series of tasks and discussion spaces, where a learner can enter and exit at any point. Badges can also be associated with these challenges, which learners earn by completing various tasks. In approximately two years, P2PU has supported the creation of just over 2,000 distinct learning environments that combine a set of common elements to provide rich learning experiences for participants.

While there are numerous projects in P2PU, they vary significantly with respect to their level of development. The majority of the over 2,000 projects in P2PU were tagged as "under development," which means that members experimented with creating these learning environments, but never implemented them live in the community. Most projects appear to have started development but were never finished and released live, and others were deleted from the site over time. Of the total, there were 368 projects (18.09%) that went live and were not deleted from the P2PU platform. Of these projects 159 were challenges, 132 were study groups, and 77 were courses. About 85% of these projects were in English and 12% in Spanish (with less than 3% in other languages).

At the same time, it was also observed that in P2PU's nascent history, there has been a general trend of 10 to 25 new learning projects being developed and publicly released each month, with a sharp rise in July 2012 due to a special initiative that month (see Figure 2 below). Together, these statistics suggest that as a platform P2PU is able to successfully encourage individuals to experiment with creating a diverse array of participatory learning environments, but that a relatively small percentage of those projects are ever implemented as operational. Future research should consider the issues related to the development of learning projects in P2PU, strategies for fostering more complete learning project development over time, and analysis of whether this ratio of experimentation to "published" projects is optimal. In addition, there are rich opportunities to understand the barriers that individuals may face in the creation of projects within such a participatory platform. Factors such as a lack of understanding of the P2PU platform, prior knowledge in topic areas, motivation, or lack of community involvement may influence the systematic production and availability of learning projects over time.


Figure 2. Projects created by month ("live" courses only)

Along with this steady development of peer-created learning environments, there were 41,281 registered members in P2PU at the time of writing of this paper. Such membership represents a truly massive group of potential learners, and positions P2PU as an example of a large-scale participatory learning environment. P2PU does not describe itself as a cMOOC environment, but rather a platform to promote open, informal learning. However, taking a broad view of the entire platform, it is an online community of over 40,000 potential learners who can share and create their own participatory learning environments and experiences. This large pool of potential learners is notable as an experiment in participatory, peer-led learning; nonetheless, participation and engagement issues remain, just as in any other online community. Data in the P2PU database provides time-stamped fields for when a member created their P2PU account and the last date they were active in the community (when they logged on). Using these fields, it is possible to create various overall participation metrics, such as how many P2PU members ever returned to the site after first creating their account. It was found that 6,483 members returned to P2PU at least once after account creation (approximately 16% of all registered members). This implies that that the majority of members (approximately 85%) created accounts and never again engaged with the community. While this pattern of overall engagement (or relatively lack thereof) is a potential problem for P2PU, it is consistent with participation patterns observed in other online environments (Cummings, Butler, & Kraut, 2002). Nevertheless, having over 6,000 active members in this community still represents a substantial pool of learners and contributors in this participatory learning community. Furthermore, as P2PU has continued to establish itself as an online community, one can also observe that they have consistently attracted 300-400 new members per month (who return to the site) over the 2012 year (see Figure 3). Overall, these descriptive measures of P2PU's growth and history show an online community that is in an early stage of development, but steadily growing both in terms of individual users and different types of participatory learning environments.


Figure 3. Member accounts created by month (for 6,483 members who have returned at least once to P2PU)

To explore the relationship between these two interdependent elements of the P2PU ecology, metrics that describe how members participated and engaged within the context of study groups, courses, and challenges were examined. Members in P2PU can relate to projects in various ways. They can organize and create projects, or they can join in on projects as participants or followers. Of the 6,483 members of P2PU who logged in at least once more after creating an account, 4,730 (73%) have signed up to be an organizer, participant, or follower of at least one learning project. This suggests that of P2PU members who do return to the site, a substantial number attempt to become active members in projects by organizing a project or signing up to be a participant.

There was also significant variation in the number of members participating in different learning environments both within and across the different types (see Table 1). For example, study groups and courses in P2PU typically had 1-2 organizers who created the learning group and initiated activities. The majority of these learning groups were small. The median number of participants in study groups was three members and the median in courses was five members. However, some study groups and courses grew into quite large learning environments. Study groups reached up to 370 participants and 1,167 followers (individuals who followed along but did not actively participate). The largest courses grew to 173 members and 319 followers.

Table 1. Number of members who participated in P2PU study groups, courses, and challenges (live projects only)

Membership in P2PU challenges is organized a little differently. Learners choose to adopt the challenge and the platform also keeps track of who has completed a given learning challenge (completers). Again, a majority of challenges are small, with the median number of adopters being two members – although the distribution is skewed with some challenges successfully having up to 158 (current) adopters and 694 completers. These descriptive statistics highlight the wide diversity of participatory learning environments that arise in P2PU. Many study groups, courses, and challenges attract only a few like-minded individuals. However, some projects attract hundreds of learners to create larger-scale learning environments. Again, while the skewed distribution of participation potentially presents a challenge for P2PU and other open learning environments, it is consistent with the distribution of group and forum sizes that has been observed in other online contexts (Butler, 2001; Butler & Wang, 2012; Jones, Ravid, & Rafaeli, 2004). Future research that explores the relationships between support for development of focused learning environments, characteristics of the created learning environments, and the resulting patterns of participation would make substantial contributions to understanding how open learning communities such as P2PU can function more effectively.

Tasks and Badges: Opportune Measures of Engagement

As the P2PU platform has developed, support for different types of learning environments have been added. Initially, study groups (time-bounded, unstructured group interaction spaces) were supported, then courses (time-bounded, guided exploration of topics, with associated group interaction). Most recently challenges were introduced as ways to organize learning. Challenges are organized as an informal syllabus, consisting of a checklist of tasks. P2PU members join challenges and when they complete various learning tasks, they check off these tasks as completed. One unique affordance of this feature is that P2PU logs a record of challenges, tasks, adopters, and when these adopters checked off tasks as complete. Using these logs, measures of the persistence and engagement of participants in P2PU challenges as of October 2012 could be calculated.

The detailed logging of tasks provides some nuanced ways to conceptualize and observe persistence and engagement. For example, challenges in P2PU had an average of 4.67 learning tasks that members were asked to complete (Table 2). On average, members reported completing roughly half of the available tasks (2.24 average number of tasks completed per adopter). Interesting future research streams emerge from this descriptive data. For example, using the P2PU data it is possible to examine longitudinal trends of when P2PU members complete tasks and how other variables influence this task completion. In addition, future research might examine how characteristics of challenges, tasks, and P2PU members and their social networks relate to the completion of tasks over time.

Table 2. Tasks and badges associated with challenges in P2PU

Average number of tasks completed per adopterAverage number of badges earned per member

Note. N = 159 active challenges in P2PU.

In addition to tasks, P2PU had recently begun piloting an open badge system based on the Mozilla Open Badge Framework (The Mozilla Foundation & P2PU, 2012). The idea behind open badges is to provide publicly visible credentials for informal learning, and there is a rich area of future research to examine the varied social, cultural, and educational functions of online badges and informal credentialing (Abramovich, Schunn, & Higashi, 2013; Antin & Churchill, 2011). As with study groups, courses, and challenges, members of P2PU can create badges that other members can earn by completing different learning tasks and challenges. Badges are also logged in the P2PU system and provide a way to measure and understand learner achievements and their engagement with learning over time.

As of October 2012, the implementation of badges was in an early stage. As a result, many challenges did not have associated badges (i.e., challenges had an average of less than one badge associated with them). Some challenges had up to eight associated badges with some activity in members earning badges (Table 2). In future research, some interesting avenues could include examining learners' trajectories of badge earning and the different social dynamics at play in P2PU in relation to badge earning (e.g., to better understand who creates and earns badges, as well as what the predictors badge earning are).

Conclusion

One lesson from the recent rise of MOOCs (Pappano, 2012) is that new forms of online education will most likely endure, evolve, and become a vital part of how education is delivered in the future. Much of the initial excitement and attention to MOOCs have risen from a particular version of online learning, the xMOOC. This paper has highlighted how alternative, participatory forms of education production and delivery can develop within and be supported by social platforms such as P2PU. In such participatory, collective cMOOC contexts, participation and engagement take on varied forms. Courses must be developed, assessments created, fellow learners recruited, and learning environments sustained over time through engagement and participation. These different functions bring about new challenges to sustaining robust, large-scale, participatory learning environments. The challenges illuminate potentially high-impact research areas that could make a substantial contribution to improving and evolving large-scale, online learning that are alternatives to the high-profile examples that are popular today (Coursera, etc.).

Through the descriptive analysis of P2PU presented in this paper, some of these challenges have been highlighted and potential research directions for future work illuminated. P2PU is clearly an early-stage, but growing, online community. From the log data, it is evident that steady streams of members are joining the community over time, and that these members are actively experimenting with creating their own learning environments online. However, a substantial number of learning projects on P2PU are never released live and implemented. Interesting future questions remain about the experiences of learners who take on the role of course creators, including the following:

What are their motivations?What challenges do they face in creating participatory learning environments?What skills are needed to create effective courses, such as instructional design, and more importantly, how can the learning of these skills be "scaled out" to everyday individuals to foster more effective crowdsourced course development?What factors are related to courses actually being implemented in P2PU?Research in this stream would shed light on issues such as new forms of organization, instructional design, and production of education in a participatory, open, social computing context.

The analysis also highlights the wide variability present in the online learning environments that are present in P2PU. In a participatory setting such as P2PU, the majority of learning projects serve small groups of informal learners who come together around specific topics. These learning arrangements represent the long-tail of the ecosystem of learning contexts in P2PU with many courses that serve a few individuals around niche interests (Brown & Adler, 2008). A small number of P2PU projects successfully garnered the participation of hundreds and thousands of interested learners, and could be viewed as a form of cMOOC. Individual P2PU projects that garner thousands of students introduce new issues of educational practice at scale. However, the aggregation of hundreds and thousands of smaller learning arrangements in a platform such as P2PU also combines to create an alternative, and uniquely large-scale, ecology of education opportunities. Future questions remain concerning how to foster robust learning communities from this ecological perspective (Butler & Ahn, 2013). How many courses and members are needed to foster participation and learning activities? What factors within courses or learning projects are related to fostering engagement over time (e.g., Ahn, Weng, & Butler, 2013)? How can platforms such as P2PU foster the development of participatory learning behaviors such as social network formation, knowledge sharing, and active learning opportunities?

Finally, the P2PU platform also affords opportunities to examine concepts such as participation and engagement through new artifacts such as online learning tasks and open badges. It has been demonstrated here how log data of learning tasks in P2PU can show the variability of learning activity conducted by members. Future work is possible to examine longitudinal patterns of the learning tasks P2PU members undertake, which may provide indicators of sustained participation and engagement. In addition, research that can examine the relationship between factors such as course design, social interaction, and networks with the completion of learning tasks over time can make a substantial contribution to fundamental questions around supporting online learners to persist and not drop out of courses over time. P2PU's early piloting of open badges, and their logging of badge creation and earning behaviors, can also serve as an exciting context to explore the social, cultural, and educational functions of badges in online learning. These research areas promise to rise in importance as badges, and other artifacts of informal educational credentialing, become a major element of online learning in the future.

Overall, P2PU provides a rich environment for addressing many fundamental questions about the development, operation, and use of large scale, open, online learning platforms. While this work is still in its early stages, it provides an initial look at the way that focused learning environments, such as P2PU study groups, courses, and challenges, are developed (or not), are adopted (or not), and are engaged by learners (or not). Realizing the potential of MOOCs and other forms of large scale, technology-enabled learning environments will depend not only on understanding how to provide appropriate experiences for individual learners, but also on our ability to design platforms that provide the affordances necessary to support development of diverse populations of focused learning environments.

References

Abramovich, S., Schunn, C., & Higashi, R. M. (2013). Are badges useful in education? It depends upon the type of badge and expertise of learner. Educational Technology Research & Development, 61(2), 217-232. doi:10.1007/s11423-013-9289-2

Ahn, J., Weng, C., & Butler, B. S. (2013). The dynamics of open, peer-to-peer learning: What factors influence participation in the P2P University? In R. H. Sprague, Jr. (Ed.), Proceedings of the 46th Annual Hawaii International Conference on System Sciences (pp. 3098-3107). Los Alamitos, CA: IEEE Computer Society. doi:10.1109/HICSS.2013.515

Antin, J., & Churchill, E. F. (2011, May). Badges in social media: A social psychological perspective. Paper presented at the ACM SIGCHI Conference on Human Factors in Computing Systems, Vancouver, Canada. Retrieved from http://labs.yahoo.com/files/Antin%20&%20Churchill%20-%20Badges%20in%20Social%20Media.pdf

Bienkowski, M., Feng, M., & Means, B. (2012). Enhancing teaching and learning through educational data mining and learning analytics: An issue brief. Washington, DC: U.S. Department of Education, Office of Educational Technology. Retrieved from http://www.ed.gov/edblogs/technology/files/2012/03/edm-la-brief.pdf

Brown, J. S., & Adler, R. P. (2008). Minds on fire: Open education, the long tail, and Learning 2.0. EDUCAUSE Review, 43(1), 16-32. Retrieved from http://www-cdn.educause.edu/ir/library/pdf/ERM0811.pdf

Butler, B. S. (2001). Membership size, communication activity, and sustainability: A resource-based model of online social structures. Information Systems Research, 12(4), 346-362. doi:10.1287/isre.12.4.346.9703

Butler, B. S., & Ahn, J. (2013, February). Ecological perspectives on creating and sustaining open learning environments. Paper presented as part of the Workshop on CSCW and Education at the 16th ACM Conference on Computer Supported Cooperative Work, San Antonio, TX. Retrieved from http://www.ahnjune.com/wp-content/uploads/2012/12/CSCW-Position-Paper-_final.pdf

Butler, B. S., & Wang, X. (2012). The cross-purposes of cross-posting: Boundary reshaping behavior in online discussion communities. Information Systems Research, 23(3), 993-1010. doi:10.1287/isre.1110.0378

Cummings, J. N., Butler, B., & Kraut, R. (2002). The quality of online social relationships. Communications of the ACM, 45(7), 103-108. doi:10.1145/514236.514242

Howe, J. (June 2, 2006). Crowdsourcing: A definition [Web log post]. Retrieved from http://crowdsourcing.typepad.com/cs/2006/06/crowdsourcing_a.html

Jones, Q., Ravid, G., & Rafaeli, S. (2004). Information overload and the message dynamics of online interaction spaces: A theoretical model and empirical exploration. Information Systems Research, 15(2), 194-210. doi:10.1287/isre.1040.0023

Kafai, Y. B., & Peppler, K. A. (2011). Beyond small groups: New opportunities for research in computer-supported collective learning. In H. Spada, G. Stahl, N. Miyake, & N. Law (Eds.), Connecting computer-supported collaborative learning to policy and practice. Proceedings of the Ninth International Conference on Computer-Supported Collaborative Learning (CSCL 2011) (Vol. 3, pp. 17-24). Atlanta, GA: International Society of the Learning Sciences. Retrieved from http://www.gerrystahl.net/proceedings/cscl2011/cscl2011proceedingsIII.pdf

McAuley, A., Stewart, B., Siemens, G., & Cormier, D. (2010). The MOOC model for digital practice. Charlottetown, Canada: University of Prince Edward Island. Retrieved from http://www.elearnspace.org/Articles/MOOC_Final.pdf

The Mozilla Foundation, & Peer to Peer University (with The MacArthur Foundation). (2012). Open badges for lifelong learning. Retrieved from https://wiki.mozilla.org/images/b/b1/OpenBadges-Working-Paper_092011.pdf

Pappano, L. (2012, November 2). The year of the MOOC. The New York Times, ED26. Retrieved from http://www.nytimes.com/2012/11/04/education/edlife/massive-open-online-courses-are-multiplying-at-a-rapid-pace.html

Parameswaran, M., & Whinston, A. B. (2007). Social computing: An overview. Communications of the Association for Information Systems, 19, 762-780. Available from the AIS Electronic Library. (http://aisel.aisnet.org/cais/vol20/iss1/1)

Siemens, G. (2012, March 5). MOOCs for the win! [Web log post]. Retrieved from http://www.elearnspace.org/blog/2012/03/05/moocs-for-the-win/

Waters, A. (2012, July 23). Dropping out of MOOCs: Is it really ok? [Web log post]. Retrieved from http://www.insidehighered.com/blogs/hack-higher-education/dropping-out-moocs-it-really-okay


View the original article here

Online Student Support Services: A Case Based on Quality Frameworks

?

         Online Student Support Services:
A Case Based on Quality Frameworks


Barbara L. Stewart

Professor of Retailing and Consumer Science
Department of Human Development and Consumer Sciences
University of Houston
Houston, TX 77004 USA
bstewart@uh.edu

Carole E. Goodson
Professor of Technology
Department of Human Development and Consumer Sciences
University of Houston
Houston, TX 77004 USA
cgoodson@uh.edu

Susan L. Miertschin
Associate Professor of Information Systems Technology
Department of Information and Logistics Technology
University of Houston
Houston, TX 77004 USA
smiertsch@uh.edu

Marcella L. Norwood
Department Chair and Associate Professor of Retailing and Consumer Science
Department of Human Development and Consumer Sciences
University of Houston
Houston, TX 77004 USA
mnorwood@uh.edu

Shirley Ezell
Associate Professor of Retailing and Consumer Science
Department of Human Development and Consumer Sciences
University of Houston
Houston, TX 77004 USA
sezell@uh.edu

Abstract

Expansion of online teaching and learning has fostered the creation of numerous standards and frameworks to evaluate and promote quality in online instruction and learning. Review of these standards showed that there is consistency in indicators of quality for online programs. One indicator, student support, was selected for study. Quality frameworks were reviewed to determine if student support was included among the indicators. Then, to examine the inclusion of student support services within a real-world context, a case was selected for observation and illustration. Examination of the case showed that faculty and administrators had designed and were delivering critical student support services. Both new and existing support systems were needed and provided. This case indicated a primary need and provisions for student support in the following areas: admissions and registration, advising, orientation, learning support, scholarships and awards, library resources, computing and technology resources, articulation with other institutions, career placement, and communication.

Keywords: student services, online services, distance education services, quality standards, supporting student success

Introduction

Program and course evaluations provide a foundation for appropriate review and assessment of online educational pursuits. Expansion of online teaching and learning has fostered the creation of numerous standards, guides, and frameworks to evaluate and promote quality in online offerings. A selection of these frameworks provides indicators of the aspects of online delivery that are important for course quality.

While not always clearly differentiated, quality standards or frameworks can be applied to both programs and courses. Whether they are formatted into principles (Western Cooperative for Educational Telecommunications [WCET], 2001), scales (Walker & Fraser, 2005), categories (National Institute of Standards and Technology [NIST], 2009), clauses (Canadian Standards Association, 2009), benchmarks (Phipps & Merisotis, 2000), dimensions (Jung, 2011, 2012), elements (MarylandOnline, 2008), standards (MarylandOnline, 2008), or values (International Center for Academic Integrity, 1999), each can be used to capture and share a vision of the elements of online education that are critical to quality.

This paper is based on the authors' perceptions and experience that support is important for student success. Quality frameworks were studied to find whether support factors were included. A case was investigated to observe the application of student support services. Schematically, this paper proceeds in the following sequence: examination of quality frameworks, identification of student support, and case observation.

Quality Frameworks for Online Learning

Representative standards focus predominately on the evaluation of online programs. They include the following:

The Sloan Consortium's Quality Framework (Moore, 2005);WCET's Best Practices for Electronically Offered Degree and Certificate Programs (WCET, 2001);Distance Education Learning Environments Survey (Walker & Fraser, 2005);Quality Matters (QM) (MarylandOnline, 2008);American Distance Education Consortium (ADEC) Guiding Principles for Distance Teaching and Learning (ADEC, 2003);Quality Improvement Framework (Inglis, Ling, & Joosten, 2002);Institute for Higher Education Policy (IHEP) Benchmarks for Success in Internet-Based Distance Education (Phipps & Merisotis, 2000);Universitas 21 Global Quality Framework (Chua & Lam, 2007);Australasian Council on Open, Distance and E-learning (ACODE) Benchmarks (ACODE, 2010);Proactive Evaluation Framework (Sims, Dobb, & Hand, 2002);Quality Preference Framework (Ehlers, 2004);Model for Quality in Distance Education (Jung, 2012).Review of these standards by multiple authors has shown that there is consistency in indicators of quality components for online programs (Goodson, Miertschin, & Stewart, 2012; Inglis, 2008; Jung, 2011; Parker, 2008; Wang, 2006). Parker states, "Although the regulatory frameworks for quality assurance vary dramatically ... there is still enough common ground to establish some general characteristics for a scholarly approach to online teaching and learning" (p. 305). With this in mind, review of these quality standards, guides, and frameworks revealed great commonality. Specifically, student support was selected for study because the authors' experiences suggested it was important and because preliminary evidence in the literature indicated that support was perceived as an indicator of quality. Student support was consistently listed among the many important components of quality. Student support systems were found to be linked to quality factors in over half of the frameworks reviewed. For the frameworks that did not list student support services specifically, in nearly all cases, terminology either suggested or subsumed student support. For example, the 2009-2010 Malcolm Baldrige National Quality Award Education Criteria for Performance Excellence defined by NIST (2009) employ the term "customer focus," which implies inclusion of elements of student support. Examples of student support terms included "student support" (ACODE, 2010; McKinnon, Walker, & Davis, 2000; Phipps & Merisotis, 2000; Sims et al., 2002; Wang, 2006; WCET, 2001), "learner support" (MarylandOnline, 2008), "supporting the needs of learners" (Inglis et al., 2002), "tutor support" (Ehlers, 2004), "effective student services" (Goodson, Faulkenberry, Miertschin, & Stewart, 2004), and "support" (Phipps & Merisotis, 2000; Simelane, 2009).

Hence, the review of quality frameworks showed strong enough value for student services as a component of quality that the researchers judged the provision of these services as appropriate for further investigation. The excerpts from the literature that follow demonstrate both the perceived value shown for student support services and the position of student support among other indicators of quality.

Jung (2011) studied Korean students' perceptions of e-learning quality and included various aspects – technical, psychological, social, administrative, and complaint – of student support as one of her seven initial dimensions. By 2012, in a more comprehensive study of Korean and other Asian learners, she expanded her dimensions to 10, categorized these dimensions into three domains, and included the student support dimension within a domain she labels "supportive domain" (Jung, 2012, p. 5).

Three broadly recognized quality frameworks provide further illustration. The QM Program provides for assessment of 40 elements distributed across eight broad standards (MarylandOnline, 2008). "Learner support" and "accessibility" are two of the eight broad standards. IHEP (2010), striving to influence public policy on postsecondary education, published 24 benchmark criteria. The benchmarks are organized into seven standards. "Student support" is one of the seven standards. Similarly, the WCET (2001) published best practices including 27 principles across 51 activities. The five institutional level activities include "student support."

Methods

The literature review was designed to verify whether or not student support was included as an indicator of quality of online instruction. Since student support was found to be included as a factor of quality consistently in research studies and frameworks, it was retained as the focus of this investigation. Then, a method was sought to view the inclusion of student support and its impact on quality in a real-life context.

Operationally, student services were defined as the academic, administrative, social, and psychological policies and practices to enable and facilitate student success.

Case study as a qualitative empirical methodology was selected. Yin's (1994) recommendation of the case study method as an investigation of contemporary phenomenon within real-life contexts formed a foundational element of this study. This study was further undergirded by Lofland and Lofland's (1995) methodological recommendations to seek patterns in frequencies, magnitudes, structures, processes, causes, and consequences. From this list, this study emphasized Lofland and Lofland's recommendations to include frequencies, structures, and processes in investigations.

Yin (1994) proposes the formation of propositions from a research question as a link to what is examined in a study. He further suggests that these propositions define the units of analysis. This process guided the selection of a single case with which the researchers were very familiar and had access to germane details to answer the following research question:

What types of student support are needed and available to online students?

From this research question, following Yin's recommendation, four propositions were drawn:

Support services are available to online students.Existing support services can be used by online students.New support services are available for online students.Access to support services is needed by online students.Hence, as Yin (1994) recommends, these propositions guided investigation of the case as a study of student support in a real-world contextual online setting.

The researchers recognized the need in this study to foster validity and followed Yin's (1994) charge to use a prescribed protocol as a means of establishing reliability and validity for case studies. For this study, the protocol included: (1) formulating an overview of the project including objectives and readings; (2) evaluating the readings for the presence or absence of references to student services; (3) providing a sequence for the observation of student support services; and (4) adhering to the protocol in preparing the final report as outlined by Yin. Following Lofland and Lofland's (1995) orientations, both objective case details and the subjective observations of the researchers were recorded, tabulated, and categorized.

The following illustrative case is the basis for the discussion of supporting online students. While future research would benefit from the inclusion of additional cases, this work reflects Eisenhardt's (1989) value for theoretical rather than statistical sampling in case selection. The authors, in the roles of researchers-as-participants, were in ideal positions to review and reflect upon student support. Pettigrew (1990), for example, suggests that when the number of cases that could be studied is substantial, it makes sense to select cases for their particular value. Hence, this case was selected because of the experience of the authors.

More generally, case methodology was selected because of its capability to investigate contemporary phenomena in real-life contexts (Yin, 1994). Online education certainly is both contemporary and exists within a real-life context. Soy (1997) suggests that in addition to producing, disputing, and building upon theory, case study research design is useful in describing, explaining, and applying and evaluating solutions to objects, phenomena, and situations.

Identification of student services, both needed and offered, was enabled through student, faculty, and staff interviews as well as examination of University websites, catalogues, student orientation materials, and directories. Tracking and documenting student outcomes was facilitated by the familiarity of the researchers with the case due to their roles of researchers-as-participants. Tools used to observe student outcomes included course summative and formative evaluations, the college-wide instructional effectiveness instrument, online discussion boards and chat sessions, live and online conversations, feedback from a professional advisory board, a student survey of critical factors for online success, and elements of an intensive accreditation self-study and site visit process.

Case Description

The students of the Bachelor of Science in Retailing and Consumer Science (RCS) degree program at the University of Houston (UH) are part of a large urban institution located in the middle of an expansive metropolitan region. The program is well established, having been offered for more than 30 years. Ninety percent of the student population lives and works off campus. Specifically, more than 90% of RCS majors are employed, with the majority working full time. Commuting times to campus can be challenging. Increasingly, students who work and live in an electronic environment raise expectations and opportunities for the use of technology in their educational experiences. At the same time, faculty and administrators extend their knowledge and skills to applying technology to learning environments. Evidence of student use of technology provided early encouragement for the development of online classes for this program. Additionally, a statewide initiative toward a common general education core across public institutions of higher learning provided opportunity. Since students could complete their general education core in multiple locations and online, the RCS faculty could focus on developing major courses in an online format.

The first online course in the program was offered in 1997. By Fall 2002, six of the 12 courses in the RCS major were available to students online. Shortly after Spring 2003, all courses were taught online (Stewart, Norwood, Ezell, & Waight, 2006).

Case Outcomes

It is from this vantage point that this case offers experience that may be beneficial in considering best practices. Consistent cycles of assessment, change, and reassessment resulted in the emergence of multiple issues. Among the critical issues was the need for numerous and varied types of support for students.

The RCS faculty members, in their roles of researchers-as-participants, not only determined student needs and designed and executed systems to meet the support needs of students, but they also recorded and catalogued the systems and evaluated their impact for students. This process provided a record of student support initiatives and outcomes, which is described herein.

The following section contains a listing and description of the support services for students used in this case study. While it is somewhat lengthy, it is shared both to illustrate the scope of services needed and to demonstrate that in this case the collaborative efforts of university administrators with faculty as well as college, department, and program leaders provided the expansive range of services needed by online students.

Types of Student Support

Objective and subjective observations resulted in the identification of existing and needed student services. An analysis of these services enabled the creation of categories for them. In addition, the four propositions based on Yin (1994) outlined earlier were each confirmed. Furthermore, this case indicated a primary need for student support in the following areas::

admissions and registration;academic advising;orientations to the University and to online learning;academic support services;scholarships and awards;library resources;computing and technology resources;articulation and transfer from other institutions;career placement.Needs for communication and connectivity were found to be pervasive throughout the support systems.

Finally, from observations and analysis, categories of services emerged. These services were further categorized by whether they were provided at the (1) course; (2) Department/College; or (3) University levels. The resulting structure is presented in Figure 1.


Figure 1. Categories of student support services emerging from observations and analysis

Descriptions of Student Support and Student Support Outcomes

In some cases, systems or services were designed and implemented specifically to support online students. In other cases, accommodations were made to existing mainstream services to facilitate access for online students.

Course design elements that support students. The faculty involved in this case supported students through course design; they developed course elements and features to enable student success. Course structure and content delivery were the two major areas most carefully considered.

Course structure issues included: (1) systematic implementation of a learning management system (LMS); and (2) student information systems that were easy to access, easy to navigate, and easy to use without technical difficulty. To support course and content access, orientation materials and syllabi were created to provide students with a clear understanding of course objectives, materials, expectations, and assessments. These materials included information about objectives, content, contacts, texts and materials, technical requirements, policies and procedures, assignments and exams, grading, help, calendars, resources, and course features.

Course content organization within the online delivery format was the second major area of course support for students. For most courses, content modules were developed to facilitate delivery of content and to break the material into approachable units. Each module was set up so that students could easily access module objectives; content outlines and terms; content presentations in PowerPoint, streamed video, or other forms; module assignments; and module assessments. Students could easily "click through" the module sequence and resources to engage in the learning process. Enrichment links were also provided to enhance the learning experience.

Course design elements outcomes. Consistently, student reports from the multiple outcome measures indicated that well-designed course structures and delivery modes are important support components and are beneficial to student success.

Specifically, student success was enabled by the incorporation of a user-friendly course learning platform. In this case, the University adopted a standard commercial LMS to provide a uniform course format. The Department also employed a technical specialist to support faculty development of courses in a consistent format within the LMS. Students expressed appreciation for the consistency provided by a common format. A familiar working environment across courses freed them to focus on the course content rather than the operational system.

Paramount in students' perceptions of support through course design was appreciation of clear instructions. These facilitated easy access to course components, navigation within course sites, and use of instructional materials without technical difficulties. Course specific orientation materials and syllabi that clearly delineated course processes, expectations, and resources were highly valued.

Students responded positively when course delivery methods facilitated their success. Content modules, used to break the course into approachable units, provided students with the ability to progress through the course a step at a time, yet also allowed them to see the big picture of the course in its entirety. Completion of one unit at a time, for example, was reported as providing a sense of satisfaction and progression.

Department and College Support Services for Students and Outcomes

While course level design systems and features, created primarily by the instructors, were imperative for student success, additional needed services were provided by the Department and the College. These were in the areas of learning support, student organizations, academic advising, and instructional design. In some cases, divisions between Departmental/College services and University services were blurred since each level may have provided service.

Instructional Support Services Center. The primary mechanism for student learning support beyond the course level, yet within the Department, was the Instructional Support Services Center (ISS) Lab. The ISS Lab offered many services to aid with student learning, achievement, progression, and retention. Over the time period spanned by the case, these services evolved to accommodate the increase in online offerings. For example, individual course tutoring was initially offered on campus through the ISS Lab. As need arose, and over a two- to three-year period of iterative design–implement–evaluate cycles, online tutoring was incorporated (Miertschin, Goodson, & Schroeder, 2010). Currently, the ISS Lab offers the following student services: course software assistance;individual course tutoring (both face-to-face and online);Graduate Record Examinations (GRE) preparation;supplementary course materials;a meeting place for students working on group assignments;a worksite for teaching assistants to support online courses;distribution of student course materials;collection of student assignments, as requested by the instructor;provision for student supplies;proctoring of flexibly scheduled exams or makeup exams.The ISS Lab also provides faculty support for online courses.Student organizations. A professional student organization, Collegiate DECA (previously known as Delta Epsilon Chi and Distributive Education Clubs of America), supported students through membership and leadership opportunities. In addition to establishing connections to one another, students developed channels to the faculty, campus, and community. Student members engaged in diverse activities including sponsorship of speakers, student discussions, service projects, fundraising activities, career days, and campus events. In addition there were outreach opportunities where students applied professional skills to judge and organize local and regional Collegiate DECA events for high school students. Students judged competitive events in the areas of marketing, merchandising, and retailing.

In addition to the departmentally sponsored student organization, faculty members encouraged student participation in University-wide organizations and activities. This study found that affiliations with department and campus student organizations lessened the isolation that is sometimes experienced by online students.

Academic Services Center. Counseling through an Academic Services Center (ASC) provided support with academic counseling for students. Online students were served by academic advisors via e-mail, telephone, or on-campus visits. ASC advisors oriented students to academic processes and regulations, evaluated transfer credits, created degree plans, communicated with students regarding events and deadlines, monitored academic progress, provided referrals to other campus services, facilitated career planning, and evaluated readiness for graduation. Course design and technical support. The following areas of support for online instruction were targeted for faculty but are included here because a well-supported faculty is better equipped to help students: Course design. In addition to instructional design services provided by the University, the Department, and the College employed instructional design support staff members. These individuals trained faculty in the use of course software and online communication tools, trained and monitored graduate students who assisted with course design, and provided direct design services. Technical services. The Department and College employed technical support personnel to support the technology needs of faculty and staff. Their responsibilities included assisting faculty and staff with hardware and software needs and maintenance of computer labs for student use.Department and College Student Support Service Outcomes

Outcomes evaluation of support for students as provided by Department and College initiatives, including learning support, student organizations, academic advising, and instructional design, yielded positive effects for the students.

Learning support. Logs of student services, maintained by the ISS Lab, recorded individual contacts with students on a daily basis. Analysis of the logs provided evidence of extensive student use and problem resolution. Students used the ISS Lab as both a physical place of contact and as an online resource for assistance with challenges related to course content and software. Students used tutoring services in both face-to-face and online formats, which together met student needs for course support and flexibility. Mathematics tutoring and supplemental materials for courses were seen as particularly beneficial. For some online students, having a physical location on campus where they could receive face-to-face help was important. The ISS Lab supported students by providing a single highly available location where they could flexibly pick up or submit course materials and complete exams in a proctored setting. Flexible scheduling of exams and on-demand assistance accommodated students' busy lifestyles. Student organizations. Students reported multiple benefits from participation in Collegiate DECA. DECA activities generated excitement and connection, as evidenced by conversations in online forums and e-mails to faculty. Faculty members stated that they got to know students in a deeper way through connections made through DECA-sponsored events. Speakers' forums organized by DECA provided extracurricular, relevant content and triggered valuable links to the profession. Judging high school competitions and organizing fundraisers allowed students to apply and extend classroom knowledge, practice professional communication skills, and make social connections to peers. Affiliation with DECA reinforced a sense of community, especially for those studying remotely. These findings are consistent with Tinto's (1975) suggestion that social integration is important in student retention. Academic advising. As the site for academic advising, the ASC was used by almost all students in this case. ASC rosters and logs recorded student contact via e-mail, telephone, and on-campus appointments. Logs indicated that students benefited from support, having received advising about academic processes and regulations, evaluation of transfer credits, course sequencing, degree plans, and evaluation of progress toward graduation. Additionally, the ASC regularly made referrals to other campus services. Instructional design. Students responded favorably that online courses were well designed. Software training, professional course design support, and training of graduate students used by instructors to create effective courses yielded positive student outcomes. Students felt course structures were easy to navigate and content was readily accessible. Technical support. Technical support for faculty, including training, procurement services, applications, and troubleshooting of hardware and software, supported students by enhancing the capability of faculty members to teach. Added technical support in the form of assisted computer and learning laboratories for students, created an important additional direct support mechanism. University Support Services for Students and Outcomes

Online students benefited from numerous services provided by the University as well. Some of these services were designed and directed specifically for online students, while others were available to all potential and enrolled students. Existing services, in some cases, required careful attention in order to be made accessible for online students.

Orientation. Orientations were provided at a number of different levels: General University orientations. General University orientation programs were available for all new and transfer students in this case. Services included campus visits, registration assistance, placement testing and counseling, career guidance, and other support. Orientation programs also provided support and information for parents and families. Campus-wide online orientations. Beyond general campus orientations, the University developed an orientation system specifically directed to online students. Pathway to Distance Education was a mandatory online orientation designed for both prospective and current students to inform them about distance education at the University. It was separate from other mandatory freshman and transfer orientations.

Besides being provided with in-depth information about distance education at UH, students could access course materials posted by the instructor prior to the start of classes. In addition to basic information about courses, this online orientation system provided features and links with information on campus resources, University identification cards, the bookstore, tuition and fee payment, the libraries, career services, services for students with disabilities, campus student organizations, keys to success in online courses, study skills, and exam preparation strategies. Topics such as "What is distance education?," "Being a distance education student," "Getting started with Blackboard," "Keys to success in distance education classes," and "Getting ready for distance education" were included. Additionally, information about registration, buying books, technology, tuition, study skills, and online library access was available.

Course-specific online orientations. Each course instructor prepared and disseminated orientation materials specific to individual courses. These materials were provided in electronic format. For example, instructors posted items such as a "welcome" orientation letter to the course homepage or a visual presentation that was used to share course procedures, syllabus, calendar, tips for success, grading plans, tools, strategies, resources, etc. University student success and retention programs. Student success, satisfaction, and retention can be influenced by students' levels of connection and interactivity with University faculty, personnel, and systems. The University in this case supported several student success programs across campus that helped students make the transition to college and supported their academic pursuits once they arrived. These programs included the following and were accessible to online students: General University support services. Additionally, other critical support services were provided with attention to access for online students: Scholarships and financial aid. The University provided an array of financial aid options to assist online students in their educational pursuits. Additional information about available University undergraduate scholarships was available via the Office of Scholarships and Financial Aid website. The College also offered scholarships directed toward online students. Library resources. Off-campus access to library resources was a critical service for online students. Detailed procedures were developed to meet needs of these students. Resource guides and tutorials were available that outlined the procedures for requesting library materials to be delivered both electronically and/or physically to prescribed off-campus distance education sites. Librarians and electronic help desk services were available. Resources were available to assist students in learning how to remotely use research tools, identify and use databases, search for appropriate sources, request materials for use, and receive materials. Computing and technology help desks. The experiences of this case suggest that the technology support service most valued by online students was the help desk service. University Information Technology recently combined its Customer Services, Information Services, and University Media Services departments to form Technology Services and Support. For times when online students may have been on campus, the Department, College, and University also maintained numerous computer labs. University Student Support Services Outcomes

Orientations. While general University orientation provided support that was used by all students, additional orientation services prepared online students specifically. Help with registration, career decisions, and course selection are examples of the services that were used by all students. Beyond these services, specific electronically delivered orientations enabled online students to get an early start with the tools and strategies for distance learning. Students learned how to access and use campus resources such as the bookstore, libraries, advising, learning support services, and course tools from a distance. Students reported deriving value from these services in terms of helping them learn to use vital campus attributes.

At the course level, specific electronically delivered orientations provided roadmaps for students to succeed in classes. One instructor's use of a welcome letter to share course policies, procedures, and expectations was seen as a friendly way to start a learning relationship. Other students felt they benefitted from orientations provided in the form of PowerPoint or other visual presentations.

University student success and retention. Numerous support programs to facilitate students' transition to and success in college were used. While, in general, tracking of outcomes was difficult, two of the support programs provided for regular, systematic feedback among students, faculty, and program advisors. Both the Urban Experience and Challenger programs yielded strong evidence of positive impacts on students through interim grade reports and mentoring sessions.

Learning Support Services were found to support students by enhancing students' study skills, test-taking strategies, and learning style awareness. For specific content areas, such as Mexican American studies, English writing, natural sciences, and mathematics, additional services were proffered.

General University support services. Access for online students was facilitated for core University services. For example, the resources provided by University Career Services were popular, and, even in a tight job market, they yielded employment contacts and placements. Similarly, student use of the Center for Disabilities provided services and contacts with faculty that led to positive academic outcomes that would not otherwise have been possible.

The University's Counseling and Psychological Services, while providing critical services, could not share student-use data regarding outcomes due to the personal nature and confidentiality of psychological services. Faculty supported students by providing referrals for these services when need was indicated or interest was expressed.

A relatively recent University addition to the general services was Profs with Pride. Interested faculty members network with students through personal and electronic tools to encourage awareness and participation in campus events. For online students faculty used e-mails, links to campus updates, discussion board triggers and comments, and announcements posted to the course website to notify students of events and encourage school spirit. Event attendance showed that students used and valued this faculty-driven support system. An ancillary outcome of this service was that students not only learned about and attended events, but that attendance and communication fostered stronger connections including student–student, student–faculty, and student–campus relationships.

More traditionally, providing bookstore, admissions, administrative, registration, finance, and other basic services for online students ensured that all students could effectively adhere to university processes, requirements, and deadlines. Out-of-state students and those in international locations relied on access to these services. Such capability, for example, allowed several U.S. military personnel to continue their education while serving in remote areas with Internet access provided by the armed services.

Scholarships and awards. To support themselves financially, online students accessed websites that outlined scholarship and financial aid opportunities and application processes. The obvious outcome was that students were able to gain or retain capability to continue their studies. Library resources. Library services enabled students to electronically receive library materials. As an early participant in online student support, the library staff offered a well-developed selection of services that students used remotely. Computer and technology help desks. To meet the challenges of learning online, students made use of computing and technology help desks. Technical problems, while definitely unwanted, were a part of the student experience. Appropriately, the campus had designed a well-supported system of help desks to answer student's questions and service their needs. The help desk intercept logs and communications with course faculty showed that students needed and used these services to solve their technical problems.Discussion and Applications

In the present study, the authors performed a review of the literature, which they used as a means of verifying whether student services were identified as a component of quality in online education. This indicated value for further study of student services. The authors, through their subsequent data collection, categorization, tabulation, and analysis, found that many services were needed and offered. Thus, review of the literature regarding frameworks for quality in online learning systems and the experience of this case, including systems developed and student outcomes, suggests that student support is a recognized and valued component of quality in online programs. Further, it appears important that support be pervasive through all aspects of student engagement in the learning process. Support services are needed at the course, department/college, and university levels. Additionally, it is imperative that accommodation be made to existing student support services to guarantee their accessibility for online students.

Analysis and reflection on the findings of this study lead the researchers to conclude that student support services play a valued and integral part in quality online educational delivery. Directed by the research question "What types of student support services are needed and available to students?," the authors have found the methodological propositions to be confirmed.

Conclusion and The Future

Considerations of the future support needs for online students suggest that adaptations will be needed as students, faculty, and technology become more sophisticated with the supporting technologies and with the mode of instruction itself. Course design and emerging technology, as well as the instructional design imagined and created by faculty, will need to keep pace with increasing levels of sophistication. Students' expectations will expand through generalized exposure to greater technology in all aspects of life, including mobile information and communication technologies. Faculty training, development, and utilization must keep pace with and exceed that growth. Both students and faculty will have evolving needs.

While the needs of students and faculty change and evolve, so too, will the university environment. New technologies will emerge and become available to institutions of higher learning. The concept of "university" may also change, as more completely online institutions emerge. Competition from online-only programs may force changes in traditional universities and their support of students. Funding issues may both improve and present problems, based on new or expanded models of online education. For example, in Australia a lack of public funding has focused interest in online education as an alternate funding source (Gururajan, 2002).

Changes in online education will continue to stimulate changes in the roles of traditional student affairs and student services professionals as they adapt and develop services to meet evolving needs in conjunction with instructors, course designers, and personnel at all levels of the institution.

Traditionally, universities have played a primary role as caretakers of learning. In that context, as faculty continue to care about student learning, the view of support for students in online environments will continue to evolve, and it will require future investigation. Usun (2004), following review of online student support systems in Turkey, suggests that the institutions, not just the courses, must actually be designed or redesigned to enhance learner support. In sum, online education requires both the development of dedicated support services, specifically created for online students, as well as attention to or adaption of existing support to ensure access by online learners. The statement by Thompson and Hills (2005) is true: "Careful planning and adequate resourcing, especially long-term funding and staffing, are necessary to ensure the long-term viability of providing worthwhile learning support services" (p. 664). Similarly, Frieden (1999) states that "the creation and operation of a distance education support infrastructure requires the collaboration of virtually all administrative departments" (p. 48). The provision of these support services, in turn, contributes to overall quality as recognized by the inclusion of student support in the numerous frameworks and standards for online course and program quality.

References

American Distance Education Consortium. (2003). ADEC guiding principles for distance teaching and learning. Retrieved from http://www.adec.edu/admin/papers/distance-teaching_principles.html

Australasian Council on Open, Distance and E-learning. (2010). ACODE benchmarks. Retrieved from http://www.acode.edu.au/resources/ACODE_benchmarks.pdf

Canadian Standards Association. (2009). The ISO 9000 essentials: A practical handbook for implementing the ISO 9000 standards (4th ed.). Toronto, Canada: Author.

Chua, A., & Lam, W. (2007). Quality assurance in online education: The Universitas 21 global approach. British Journal of Educational Technology, 38(1), 133-152. doi:10.1111/j.1467-8535.2006.00652.x

Ehlers, U.-D. (2004). Quality in e-learning from a learner's perspective. European Journal of Open, Distance and E-learning, 2004(1). Retrieved from http://www.eurodl.org/materials/contrib/2004/Online_Master_COPs.html

Eisenhardt, K. M. (1989). Building theories from case study research. Academy of Management Review, 14(4), 532-550. doi:10.5465/AMR.1989.4308385

Frieden, S. (1999). Support services for distance education. Educational Technology & Society, 2(3), 48-54. Retrieved from http://www.ifets.info/journals/2_3/frieden.html

Goodson, C. E., Faulkenberry, L. M., Miertschin, S. L., & Stewart, B. L. (2004). Comprehensive program assessment: The whys and wherefores. In Proceedings of the 2004 ASEE Annual Conference and Exposition. Washington, DC: American Society for Engineering Education. Retrieved from http://search.asee.org/search/fetch?url=file%3A%2F%2Flocalhost%2FE%3A%2Fsearch%2Fconference%2F28%2FAC%25202004Paper492.pdf&index=conference_papers&space=129746797203605791716676178&type=application%2Fpdf&charset=

Goodson, C. E,  Miertschin, S. L., & Stewart, B. L. (2012). On-line delivery of courses: What components are important to students? Computers in Education Journal, 22(1), 47-58.

Gururajan, V. (2002, February). Knowledge management a new game in higher education: A discussion paper on the new trends of technology influencing the higher education landscape. Paper presented at the 11th Annual Teaching and Learning Forum, Perth, Australia. Abstract retrieved from http://otl.curtin.edu.au/professional_development/conferences/tlf/tlf2002/abstracts/gururajan-abs.html

Inglis, A. (2008). Approaches to the validation of quality frameworks for e-learning. Quality Assurance in Education, 16(4), 347-362. doi:10.1108/09684880810906490

Inglis, A., Ling, P., & Joosten, V. (2002). Delivering digitally: Managing the transition to the knowledge media (2nd ed.). London, UK: Kogan Page.

Institute for Higher Education Policy. (2010). About IHEP. Retrieved from http://www.ihep.org/About/about-IHEP.cfm

International Center for Academic Integrity. (1999). The fundamental values of academic integrity. Retrieved from http://www.academicintegrity.org/fundamental_Values_project/pdf/FVProject.pdf

Jung, I. (2011). The dimensions of e-learning quality: From the learner's perspective. Educational Technology Research & Development, 59(4), 445-464. doi:10.1007/s11423-010-9171-4

Jung, I. (2012). Asian learners' perception of quality in distance education and gender differences. The International Review of Research in Open and Distance Learning, 13(2), 1-25. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1159

Lofland, J., & Lofland, L. H. (1995). Analyzing social settings: A guide to qualitative observation and analysis (3rd ed.). Belmont, CA: Wadsworth.

MarylandOnline. (2008). The Quality Matters program rubric. Retrieved September 13, 2010, from http://www.qmprogram.org/rubric/ (archived at http://web.archive.org/web/20100911002421/http://www.qmprogram.org/rubric/)

McKinnon, K. R., Walker, S. H., & Davis, D. (2000, February). Benchmarking: A manual for Australian universities. Canberra, Australia: Department of Education, Training and Youth Affairs. Retrieved from http://foi.deewr.gov.au/system/files/doc/other/benchmarking_a_manual_for_australian_universities.pdf

Miertschin, S. L., Goodson, C. E., & Schroeder, S. (2010). Online tutoring support service for STEM. In Proceedings of the 2010 ASEE Annual Conference and Exposition. Washington, DC: American Society for Engineering Education. Retrieved from http://search.asee.org/search/fetch?url=file%3A%2F%2Flocalhost%2FE%3A%2Fsearch%2Fconference%2F32%2FAC%25202010Full1299.pdf&index=conference_papers&space=129746797203605791716676178&type=application%2Fpdf&charset=

Moore, J. C. (2005). The Sloan Consortium quality framework and the five pillars. Retrieved from http://www.sloanconsortium.org/publications/books/qualityframework.pdf

National Institute of Standards and Technology. (2009). 2009-2010 education criteria for performance excellence. Retrieved from http://www.nist.gov/baldrige/publications/upload/2009_2010_Education_Criteria.pdf

Parker, N. K. (2008). The quality dilemma in online education revisited. In T. Anderson (Ed.), The theory and practice of online learning (2nd ed., pp. 305-342). Edmonton, Canada: Athabasca University Press. Retrieved from http://www.aupress.ca/books/120146/ebook/13_Anderson_2008-Theory_and_Practice_of_Online_Learning.pdf

Pettigrew, A. M. (1990). Longitudinal field research on change: Theory and practice. Organizational Science, 1(3), 267-292. doi:10.1287/orsc.1.3.267

Phipps, R. A., & Merisotis, J. P. (2000). Quality on the line: Benchmarks for success in Internet-based distance education policy. Washington, DC: The Institute of Higher Education Policy. Retrieved from http://www.ihep.org/assets/files/publications/m-r/QualityOnTheLine.pdf

Simelane, S. (2009). Success indicators in implementing technology-enhanced courses during a professional development programme. In P. A. van Brakel (Ed.), Proceedings of the 11th Annual Conference on World Wide Web Applications. Cape Town, South Africa: Cape Peninsula University of Technology. Retrieved from http://www.zaw3.co.za/index.php/ZA-WWW/2009/paper/viewFile/25/20

Sims, R., Dobb, G., & Hand, T. (2002). Enhancing quality in online learning: Scaffolding planning and design through proactive evaluation. Distance Education, 23(2), 135-148. doi:10.1080/0158791022000009169

Soy, S. K. (1997). The case study as a research method. Unpublished manuscript, School of Information, The University of Texas at Austin, Austin, TX. Retrieved from http://www.ischool.utexas.edu/~ssoy/usesusers/l391d1b.htm

Stewart, B. L., Norwood, M., Ezell, S., & Waight, C. L. (2006). Case study: Collaborative creation of an online degree program. Innovations in Education and Teaching International, 43(3), 197-210. doi:10.1080/14703290600618399

Thompson, J., & Hills, J. (2005). Online learning support services for distance education students: Responding to and maintaining the momentum. In H. Goss (Ed.), Balance, fidelity, mobility: Maintaining the momentum? Proceedings of the 22nd ASCILITE Conference (pp. 661-665). Brisbane, Australia: Queensland University of Technology. Retrieved from http://www.ascilite.org.au/conferences/brisbane05/blogs/proceedings/76_Thompson.pdf

Tinto, V. (1975). Dropout from higher education: A theoretical synthesis of recent research. Review of Educational Research, 45(1), 89-25. doi:10.3102/00346543045001089

Usun, S. (2004). Learner support services in distance education system (A case study of Turkey). Turkish Online Journal of Distance Education, 5(4). Retrieved from http://tojde.anadolu.edu.tr/tojde16/articles/s_usun.htm

Walker, S. L., & Fraser, B. J. (2005). Development and validation of an instrument for assessing distance education learning environments in higher education: The Distance Education Learning Environments Survey (DELES). Learning Environments Research, 8(3), 289-308. doi:10.1007/s10984-005-1568-3

Wang, Q. (2006). Quality assurance – best practices for assessing online programs. International Journal on E-Learning, 5(2), 265-274. Retrieved from EdITLib Digital Library. (5601)

Western Cooperative for Educational Telecommunications. (2001). Best practices for electronically offered degree and certificate programs. Retrieved July 24, 2006 from http://www.wcet.info/Accrediting%20-%20Best%20Practices.pdf (archived at http://web.archive.org/web/20060724103942/http://www.wcet.info/Accrediting%20-%20Best%20Practices.pdf)

Yin, R. K. (1994). Case study research: Design and methods (2nd ed.). Thousand Oaks, CA: Sage.


View the original article here