Online or not ? A comparison of students ’ experiences of an online and an on-campus class

Copyright: © 2013. The Authors. Licensee: AOSIS OpenJournals. This work is licensed under the Creative Commons Attribution License. Educational discourse has long portrayed online, or e-based, learning and all non-campusbased learning options as second best to traditional face-to-face options. Critically much of the research and debate in this area of study has focused on evidence relating to student performance, attrition and retention with little consideration of the total learning experience, which values both the traditional learning outcome measures side-by-side with studentcentered factors, such as students’ satisfaction with their learning experience. The objective of this study was to present a synchronous head-to-head comparison between online and campus-based students’ experiences of an undergraduate course. This paper reports on a qualitative comparative cross-sectional study, which used multiple data collection approaches to assess student learning and student satisfaction of 61 students who completed a semester of an undergraduate course. Of the 61 students, 34 were enrolled purely as online students, whilst the remaining 27 students studied the same material entirely through the traditional face-to-face medium. Methods included a standardised student satisfaction survey and an ‘achievement of learning outcomes’ measurement tool. Students on the online cohort performed better in areas where ‘self-direction’ in learning was indicated, for example self-directed problem-based tasks within the course. Online students gave less positive self-assessments of their perceived content mastery than their campus-based counterparts, despite performing just as well in both summative and formative assignments. A multifactorial comparison shows online students to have comparable educational success and that, in terms of student satisfaction, online learners reported more satisfaction with their learning experience than their campus-based counterparts.


Introduction
Developments in computing, particularly with respect to the use of the Internet, have fuelled an unprecedented growth in the use of technology-based environments within education.Notably, both distance-learning institutions, as well as conventional academic institutions have integrated a range of electronic learning environments, such as virtual discussion rooms, podcasts, virtual simulations and twitter boards into their curricula.A number of reasons for these developments have been offered.Web-based strategies are seen as representing a revolutionary progression in learning through the flexibility of occurring anywhere, at anytime and at a lesser cost than faceto-face alternatives (Johnson & Aragon 2003;Mayne & Wu 2011).Societally, computing has been integrated into all areas of life and in this respect nursing's engagement is unsurprising and by all accounts a necessary development if the profession is to maintain social relevance.That being said, the belief that online learning is second best to traditional face-to-face modalities continues to be echoed within many literary sources aimed at and produced by students, educators and employers, so much so that employment research contributors, including Connolly et al. (2005) and Kirtman (2009), present studies that confirm the advantage that graduates of traditional campusbased programs have over their counterparts from online or distance education programs.Much of the criticism of online options is centered around the limited supervisory presence of the instructor.Even in the context of these challenges, early contributors to this discourse (Giles 1999;Rovai & Barnum 2003) identify e-learning environments as a vehicle for improving the quality of teaching and a range of other learning interactions.

Problem statement
Online alternatives present unmatched possibilities, particularly for groups that have traditionally found it difficult to attend campus-based courses.Notably, web-based education is seen as the panacea for the competing priorities that exist for those practicing nurses wishing to study whilst maintaining full-time employment.Some, including Bonk and Graham (2006) and Farrell (2006), have expressed scepticism against this wave of generalised optimism, particularly with the idea of relying on online modalities instead of traditional face-to-face approaches.The concerns centre on assertions that multi-media technologies have been prematurely adopted in place of traditional face-toface options -primarily for fiscal reasons with little regard for the way they might compromise important areas such as learners' satisfaction, learning, involvement and completion.The concerns are vocalised more specifically with regard to skills-based areas, such as nursing, where the tradition has been to teach skills through ward-based practice.A summative view of many studies has been to support online teaching methods as a complement rather than an alternative to face-to-face options.This 'middle of the road' response to the debate fails to clarify the core question that is being asked: can e-based or online learning be a replacement for traditional campus-based face-to-face modes of learning?Additionally, this study explores the extent to which e-learning ensures the fulfilment of students' learning outcomes when compared to traditional face-to-face modalities.

Background: Teaching nursing skills -is this a special case?
Within nursing education, observation and touch have historically defined many areas of learning and the absence of face-to-face contact presents unique challenges, particularly in instances where students have to be taught skills where touch and the repetition of observed tasks are key.Despite this, counter debates have been offered to support the introduction of new teaching approaches, particularly in the post-millennium era where learners' primary vade mecum is learning through technology.
It has been argued that learning through web-based simulation offers endless opportunities for learners to observe and repeat the skill being taught, in ways that traditional teaching methods cannot (Christie & Garrote Jurado 2009).Importantly, the choice given to students within the online alternatives with regard to the pace of learning aligns itself well with the wider goals of adult education: giving the student greater control over their learning environment (Knowles 1990).Furthermore others, including Forman, Nyatanga, and Rick (2002), suggest that introducing innovative and more student-centred strategies to learning, such as online discussion boards, encourages participants to engage in complex levels of knowledge application in varied contexts.This assertion suggests that particular web-based strategies, such as student-led discussion boards, enable functioning at the higher levels of Bloom's taxonomy of educational objectives (1956) by encouraging students to make their repertoire of knowledge representations known in active debate.The taxonomy (a widely accepted framework within education) identifies six levels: knowledge, comprehension, application, analysis, synthesis and evaluation.As learners gain more in-depth mastery of the chosen area, they develop skills in the latter higher-order levels, such as synthesis and evaluation (Summers, Waigandt & Whitaker 2005).

Trends
Despite showing clear benefits with regard to developing knowledge, notable concerns exist with respect to the extent to which web-based approaches offer the same safeguards to identify and support those learners whose education is later required to be the basis for practical skill delivery, such as is true of nursing and other health and social care professions.This concern is perhaps the most notable especially with respect to the nursing practice, which is tasked with 'protecting the public through safe practice' (Nursing & Midwifery Council 2008).The study conducted by Bambara et al. (2009) concluded that web-based learning approaches were suitable only as a complement to traditional face-to-face delivery approaches and their use in nursing as a sole teaching approach was seen as second best to traditional face-to-face learning.Similarly, in their assessment of quality issues pertaining to web-based learning, Yang and Cornelius (2004) focused on the potential difficulties that may arise from high attrition rates, concerns about plagiarism and the perception of social isolation faced by online students.These observations are echoed in a range of related studies and, to date, the advantages of purist web-based learning options have been limited to reasons relating to the flexibility they offer students.Parsad and Lewis (2008) and the Unites States Department of Education, Office of Planning, Evaluation and Policy Development ( 2009) have presented research evidence to the contrary, where web-based learning options have been shown to have some advantage over traditional face-to-face modes, particularly with regard to the flexibility of learning they offer.

Aims of the study
In an attempt to test some of these assertions, the study has two specific aims, (1) to describe the comparative experiences of online and campus-based students who took the same undergraduate nursing course in the Fall of 2010 -the only difference being their modes of learning and (2) to present insider insights with respect to how online and campusbased modalities compare on a range of criteria, including content mastery, attrition and student satisfaction.

Research objectives
The current study will compare students' academic performance on a traditional face-to-face course to those studying the same material via an online alternative.Student satisfaction with their learning experience will also be explored with regard to each mode of teaching and learning.

Definition of key concepts: 'Technology-based learning -is it about games?'
Before engaging in any exploration of the varied guises of online learning, it is important that its meaning and that of associated terms be clarified.In its purist form, online learning can be best described as a learning experience in which all aspects of teaching and learning, from course delivery to student group work and assessment, are carried out within a web-based medium (Atack 2003;Summers, Waigandt, & Whitaker 2005).All aspects of teaching and learning, including demonstrations of clinical tasks, are done via a range of web-based audio and visual tools and students do not have any traditional face-to-face contact with their teacher throughout the course of their studies.
Two specific modes of pure-form e-learning are generally described.Firstly, 'live' or 'synchronous' e-learning is characterized primarily by the unique feature that interactions between students and the facilitator occur instantaneously in 'real time' or 'live'.Video or audio-conferencing and 'live-chat' typify this mode of learning, which requires that students be online at the same time to simulate the scope for instant feedback offered by traditional face-to-face approaches (Mayne & Wu 2011).The requirement for all learners to be online at the same time limits flexibility for the learners in the way that they pace their learning and goes against a cardinal benefit (studying anytime) that is centrally identified as the trump card for online approaches over traditional face-to-face methods.
As an alternative, another mode of e-learning is 'asynchronous' or 'forward' e-learning in which participants do not communicate in real time and each can make their contribution at a time that suits them (Farrell 2006).Examples of this include posting messages within discussion groups and email communications with course tutors.Despite its convenience, this mode presents particular challenges to students, which include a lack of instant feedback and the increased risk of feeling isolated as a learner.Despite salient differences between the synchronous and asynchronous approaches, both share some commonalities with respect to being centred on learning primarily via web-based environments.In that respect they pose similar challenges to learners and education facilitators alike.

Contribution to field
The current study offers an important contribution to the field of Nursing Education by providing a rare opportunity for a head-to-head comparison of student experiences studying the same content over the same index period -the primary difference being the mode of delivery.As a result, rare insights into the utility of online or e-learning education as an alternative to traditional face-to-face modalities are offered.

Participants
Sixty-one second-year students on a four year preregistration baccalaureate nursing program, all enrolled on a psychiatric nursing course, were invited to take part in a short comparative study to explore and compare their experiences of the 14-week course they undertook.Students were either enrolled on the traditional campus-based face-to-face course (n = 27) or on the online distance education option (n = 34).Of the 61 students invited to participate, only 53 completed both the survey questionnaire and the course (23 from the campus-based group and 30 from the online class).Reasons for non-participation were primarily related to students dropping out of the course before they could complete the data collection exercises.
Both the students in the campus-based and online classes were taught the same material and both groups completed the same examinations throughout the course of study.The only differences related to the modes of delivery of the learning material.Campus-based students, for example, took part in in-class weekly discussion groups to discuss the course content matter.By contrast, the online class had a weekly discussion board called 'Blackboard' that they used via an online learning environment.The latter group posted written contributions to the discussion board and each week a new discussion thread was opened.With both classes, the instructor acted as a facilitator and responded if students had specific questions that needed tutor clarification or feedback.

Materials
This comparative study used two data collection questionnaires.A nine-item student performance data recording form collated information on the following:  2009) and focuses on a range of student satisfaction aspects, including students' satisfaction with the tutor-student and peer (student-tostudent) communications, quality of class discussions, access to instructor support and the convenience of learning options for the student.The second version has been modified to enable the evaluation of both the experiences of online and campus-based students.Both questionnaires were administered using Qualtrics, an online survey management database that enables anonymous single-time completion of questionnaires (Qualtrics Laboratories Incorporated 2012).

Data collection methods
A whole population sampling approach was used to select potential study participants on both the campus-based and online versions of the course during the Fall 2010 semester.One instructor taught both course delivery options.Out of the initial 61 students invited to participate in the postcourse study, 53 were ultimately eligible to take part in the study because they had completed the course and were able to comment on the total experience.Of the eight students who did not take part, six dropped the course in the first four weeks of the 14-week semester.None of the students' profiles suggested that they differed academically from those who completed the course.The two remaining students who did not take part in the study did not complete the end of semester class evaluations and their information was incomplete.

Data analysis
Krippendorf's content analysis (Polit & Beck 2008) and thematic analysis were used to analyse the data from the questionnaires.After the initial review of students' written responses, the researcher developed thematic codes in a series of incremental re-iterative steps that involved initial coding and repeated refinement of emergent thematic categories.

Results
Out of the 53 students who completed the two data collection questionnaires, 23 belonged to the campus-based course whilst 30 were registered on the online class student group.A diagrammatic summation of data collected via the nineitem student performance data recording form (Table 1).
The campus-based class were on average younger (23.5 years) compared with the online class (36.1 years).This was largely due to the fact that many campus-based students were traditional students who enrolled at the university upon completion of their high school education.By contrast, the online class comprised primarily working individuals who were studying to improve existing qualifications.Gender representation was similar in both groups, with male students representing 13% and 10% of the campus-based and online groups respectively.This gender representation typified many of the nursing cohorts research in comparable studies and is indicative of the under-representation of males within the profession (Roth & Coleman 2008).
The course had three unit examinations that both classes took.Each test was closed-book, computer-based, timed and proctored.Campus-based students took each unit examination during one of the classes and the online students each had verified proctors supervising their examination.After completion of the three unit examinations, the online class had an average of 87% (a B-plus on the class grading scale) compared to 81% (a B-minus) in the campus-based class.The institutional pass grade was 75% -two students in the campus class and four students in the online class were not able to achieve that and did not complete the course.No mitigating circumstances were reported by any of the failing students and failure was attributed to examination material difficulty.
The online class had a superior running GPA of 3.29 compared to 3.05 for the campus-based class (on a scale of 4.0) at the point of entering the course -this might help to explain possible differences between the groups in academic ability.Despite this advantage at the point of course entry, the online group reported spending 24% more time on course-related study per week (11.3 hours compared to 8.43 hours by the campus-based group).The campus-based class average included three hours utilised each week to attend the weekly-taught session and discussion seminar.Online students had access to an audio recording of each taught session and students could skip elements of the session during audio playback.
Perceived familiarity with the course content and declared content mastery was scored on a four-point scale.It ranged from those who assessed themselves as proficient with the course material (1), to those who felt confident (2), or moderately confident (3), to those who felt unsure and in need of further instruction and reading (4).Self-assessments by campus-based students more consistently reported perceived proficiency (72%) compared to the online class (64%).At the opposite end of the confidence and content mastery scale, online students felt less confident about their content mastery (12%) compared with 4% within the campus-based student group, despite scoring better on each of the unit examinations.Both groups had comparable prior knowledge of the taught subject matter (Table 2).

Results from the Student Instructional Reporting version 2 evaluations
The SIR-II evaluates a wide range of student satisfaction related aspects, which are represented as open-format essaytype responses on various aspects of the learning experience.Results were collated on the following areas of the student experience: • appropriateness of content material

Appropriateness of taught material
Both student groups had varied reports about the perceived appropriateness of the material taught.Across both classes, students felt that the new psychiatric terminology was difficult to understand at first -a view captured well by one of the students in the campus-based class: Despite initial difficulties with the comprehension of the taught material, online students more frequently reported feeling supported by peers by being able to post issues of difficulty on a discussion board for others to respond to.

Clarity of taught material
This aspect of the SIR-II reporting framework was reported on similarly to the question on appropriateness of courserelated material.Feedback from both groups centred around the difficulty of psychiatric terminology as a noteworthy factor in determining perceived clarity.Despite some reporting of difficulties with initial clarity of the material, online class students' feedback indicated that students replayed the audio sessions a number of times if the material could not be understood on initial play:

Level of tutorial support and immediacy of tutor response
Both student groups repeatedly reported on access to tutor support within the SIR evaluations.In some respects campus-based students' reports were more positive in that they felt they had the option of accessing the tutor before, during and after each taught session.However, some of the students were critical of the support received: '… [T]here's over 20 of us.Even when I speak to Dr. X, I know that others are listening and [I] may feel that my question is irrelevant'.(Respondent 12, campus-based class) 'The in-class support was good, but when reviewing material at home, there were issues that … would have been better enquiring about ASAP with the instructor'.(Respondent 6, campus-based class) The latter response spoke to the issue of the immediacy of access to tutor support.Online assessments of access to tutor support noted this as a specific area of strength: 'Dr.X responds almost immediately to questions and even when he doesn't, other students will post on the DB.Peer sharing is so important to me -more discussion activities are needed'.(Respondent 14, online class)

Level of engagement in class discussions
Feedback on participation and in-class discussions highlighted a particular difference between the organisation and facilitation of discussion between campus-based and online classes.Typically within the campus class discussions by students were perceived as voluntary, whilst in the online class students felt compelled to make a contribution because their contribution to discussions was assessed and formed part of the course grade.As a result, campus reviews of the level of engagement varied, with some students indicating that they had not participated in any discussions whilst others provided polar opposite feedback.The online student feedback, however, consistently identified the continued involvement in peer discussions.

Overall learning experience
The feedback from students with regard to their overall learning experience highlighted two separate, but related domains of interest.Firstly, feedback from campus-based

SIR-II Evaluation Item Campus-based Class Online Class
Appropriateness of content material Terminology was confusing and students found it hard to grasp at first.
Terminology was the most emphasised challenge for students.

Clarity of taught material
Terminology difficult to understand.Difficult material was played and replayed until understanding it better.

Level of tutor support
Tutor was available, but had to be shared with other students.
Students had access via discussion forum and could email for personalised support -tutor was easily accessible.

Immediacy of tutor responses to student questions and concerns
Tutor available once a week before and after the taught session.
Anytime, anywhere access, but problems with possible delay in responses.
Level of student engagement in discussions Only vocal students engaged in active discussion in the class.
All students had to take part in discussion group activities.
Overall learning experience Varied student feedback.Low identification of the importance of social interaction as a quality indicator.
Very positive recollection of learning experience.Value of social interaction highlighted.This difference between groups in their responses was noteworthy and stood out as an identifiable differentiator between distance and campus-based students.

Ethical considerations
Prior to the development of the study protocol a number of ethical considerations were specifically addressed to ensure the protection of student participants.Most notably, the fact that students would be commenting on instructional quality posed a potential risk.To protect them against a fear of fall-out from this, all student responses were anonymous.The questionnaire completion was via a confidential online Qualtrics survey and was timed after terminal grades were awarded to students.The University Institutional Review Board approved the study.

Discussion
This study presents the rare opportunity for a real-time headto-head comparison of online and campus-based classes and, in doing so, offers insights into issues on student's perceptions of their learning experiences.At first it might appear that the study is replicating the foci of previous research within the domain, but the contention is that the study is in fact notably different from predecessor work as its results offer new perspectives for nurse educators to consider.Results from this enquiry show that online courses are comparable to campus-based classes in the extent to which they develop student mastery of the taught content.In fact, it would seem that online students' perceptions of having to work harder because they might be missing something that they would have been taught in a face-to-face environment, served as a motivator that compelled students to spend relatively more time on course-related study than their campus-based counterparts.The discovery that the online class offered comparable content mastery to the campus-based class contradicts the generally held view that online teaching approaches are at best a complement to traditional face-toface delivery.
This study found that online students expressed greater satisfaction with their learning experience than their campusbased peers.This was unexpected, particularly if considered that in the latter medium students have much more direct access to tutor support and all of the other campus-based student support systems.What seems to hold the key is that fact that within online classes, each student has an equal opportunity to express their viewpoints and be heard more than in a campus-based class where only the more vocal students get opportunities to contribute.This feeling of being seen as an important contributor seems to positively impact students' perceptions of their experiences.
In this study, online students out-performed their campusbased peers in the formative and summative course examinations.This may be explained by a number of factors, including the fact that the online class had a higher average entry GPA and that they reported spending more time studying course-related materials than their campus-based counterparts.This observation was especially important in light of the fact that online students reported feeling less confident about their knowledge and familiarity with course content, even though their performance suggested otherwise.
The following three key findings from this study all provide important starting points for future research: • The comparable academic performance between campusbased and online students.• The better student satisfaction reported by online students compared to their campus-based peers.• The discrepancy between actual performance and students' self-assessments.

Limitations of the study
Despite this being a relatively large sample size for a qualitative study, the generalisability of its findings is limited.Even so, the results have transferrable relevance to similar study settings and they open up new avenues for future researchers to explore and test.As expected with any research endeavour, there were aspects of the study process that could have been improved on when assessed with the benefit of hindsight.This study had a number of situational strengths that emanate from the same centre of study over the same time in a natural study setting.The fact that both student groups were exposed to identical materials and both were simultaneously taught by the same instructor addresses many of the expected limitations that may have come from inconsistencies.The study focussed on a single course in a program with more than 15 courses and it would be reasonable to have some questions about whether or not student subject preferences had an impact on the outcomes.This may have been better managed by doing a similar assessment with other courses on the program so that a more holistic overview was provided.The current study did not exclude this from happening and may serve as a first step toward a more longitudinal analysis.Long-term retention of learned material is an important indicator and it would have been useful to test students' residual knowledge of the taught material some months after course completion.This observation is as much a limitation of the current study as it is an acknowledgement of future directions for research.The finding on students' satisfaction with their learning experience on the online course is worthy of further investigation in future studies, especially in the consumerist climate in which education is currently being delivered.The specific areas where online students report satisfaction may well be transposed to campus-based courses.

Conclusion
The findings from this study showed that online or e-based learning has comparable academic outcomes to traditional face-to-face alternatives.Within the groups of student participants in the current study, online students scored better than their face-to-face counterparts in the summative assessments, despite their reports of lower confidence about content mastery.Furthermore, online students reported spending more time on content mastery and peer discussions -which are both attributes associated with better student outcomes.Importantly, student satisfaction was higher for the online student group than for the campus-based class.
These findings are especially noteworthy in light of the fact that online students often study with an underlying belief that e-learning is the only option for them because they cannot study via the traditional face-to-face route.This article raises questions about this long-assumed superiority of campusbased learning over online study, and does so with regard to student performance on assessed tasks, student satisfaction and student engagement in class discussions.

'
Psych nursing is a new language for me.I spent most of my time looking up unfamiliar terms.[There was] little time [left] for much else'.(Respondent 4, campus-based class)

'
My first listen was a blur … A bunch of psych jargon.I don't play much music on my iPod these days.I played and replayed the lecture on the different causal factors for mental illnessnow I can sing [it] back to you and it makes more sense to me'.(Respondent 7, online class) Campus-based students offered a differing view as articulated by one the students in the class: 'The lectures were dependent on you understanding last week's session.Very little time was available for revisiting last week's material.The 10-minute recap at the beginning of each session was not enough … How do you recap a three-hour lecture in ten minutes?'.(Respondent 9, campus-based class) s mode of learning (campus or online and/or distance) • gender • ethnicity • level of prior education • current cumulative grade point average (GPA) • student's average grade on three of the module tests they had to take • time spent on study and/or course-related activities during one calendar week • self-assessed familiarity and content mastery • prior knowledge related to the taught subject.

TABLE 1 :
Summary of student data from the student performance-recording tool.
n, Given as number; GPA, Grade Point Average.

TABLE 2 :
Emergent themes for the Student Instructional Reporting questionnaire version 2 student evaluation.