From Face-To-Face to a Virtual Classroom in Three Days
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Paper ID #34056
From Face-To-Face to a Virtual Classroom in Three Days
Dr. Monica Palomo P.E., California State Polytechnic University, Pomona
Professor
B.S. Civil Engineering, University of Guanajuato, Gto, Mexico, December 1999.
M.S. Civil Engineering, Kansas State University, Manhattan, KS, May 2003.
PhD. Civil Engineering, Kansas State University, Manhattan, KS , May 2008.
Dr. Palomo is currently a Professor of the Civil Engineering Department in the College of Engineering
at California State Polytechnic University, Pomona (Cal Poly Pomona). In this position, Dr. Palomo
is responsible for teaching courses such as Introduction to Civil Engineering; Hydraulics; Water and
Wastewater Treatment; Groundwater Mechanics; Research Experience of Undergraduate Students; and
Engineering Outreach Service Learning courses, among others. She is also a faculty advisor for the Cal-
ifornia Water Environment Association (CWEA)- American Water Works Association (AWWA) student
chapter. Additionally, Dr. Palomo is the CE Water Analysis laboratory director and coordinates all teach-
ing, research and safety training activities in the engineering laboratory. Dr. Palomo conducts research
in surface water quality improvement via natural treatment systems, water and wastewater treatment pro-
cesses, and water education. She is involved in outreach programs for K-12 students to increase the
participation of Hispanic female students in STEM fields
Dr. Bettina Jeanine Casad, University of Missouri - St. Louis
I am a social psychologist with expertise and research interests in Broadening Participation in Engineering
and Engineering Technology. I work with scientists and engineers to develop and evaluate education and
traning programs to recruit and retain diverse students and faculty in STEM.
c American Society for Engineering Education, 2021
From a Face-To-Face to a Virtual Classroom in Three days This project investigates the effectiveness of the transition from a face-to-face (F2F) to a virtual classroom environment within a 3-day period as a response to the emergency stay-home order in March of 2020, due to the COVID-19 pandemic. The courses learning objectives were modified for two civil engineering courses (Environmental Engineering and Water & Wastewater Engineering). The pedagogic approach in the F2F environment used the six elements of the ExCEEd Teaching Model. The transition to the virtual classroom utilized the ExCEEd Teaching Model and emphasized synchronous interaction with students to build an effective online learning environment. A formal assessment of whether the approach was effective in a virtual format was conducted. The hypotheses evaluated included 1) the prediction that the learning objectives could be met regardless of the format in which the class was taught; 2) the prediction that students would engage in discussion, group work and project development in an effective way, regardless of the format in which they communicate; 3) the prediction that students would perform similarly to those that have completed the course F2F; 4) the prediction that the teaching method used in the F2F environment can be used and effective in the virtual environment; and 5) the prediction that the students in the virtual environment would be able to develop a positive rapport with peers and instructor. Course grade, assignment grades and two online questionnaires were used to test the hypotheses. The study results indicate that when comparing course grades the students in the virtual environment mostly obtained overall slightly higher grades, compared to students that took the course in the F2F format. In addition, results from student end-of-term course questionnaires indicated that over a third (37.9%) of the students agreed the learning objectives were critical to their success. About 97% (n = 64) of students reported the instructor used the learning objectives in each F2F lesson. In response to the virtual classroom experience, the average rating of the instructor’s use of the learning objectives was 3.21 (SD = .886), which was significantly greater than the scale midpoint of 2.0 (Moderately), t(65) = 11.114, p = .001. The positive relationship with the instructor was rated with a mean of 2.78 (SD = 1.071), and t(66) = 5.934, p = .001, which indicates that the actions taken by the professor in and outside of the virtual classroom supported the development of interpersonal rapport. In summary, the six elements of the ExCEEd model were critical for the emergency transition to the online environment. Introduction - On March 2020, due to the COVID-19 world pandemic (WHO, 2020), business, recreational and educational activities were shutdown with a stay-home order in the state of California (CA). Following this order, California State Polytechnic University, Pomona (Cal Poly Pomona) paused face-to-face (F2F) instruction on Friday, March 13, and resumed instruction in a virtual format on Wednesday, March 18. Cal Poly Pomona’s faculty had three business days to prepare materials, setup the learning management system (LMS), and creatively learn and use the technology and software at hand to launch the full synchronous virtual classroom and resume learning. This paper describes the transition process from the F2F classroom to the virtual classroom, which includes modification of the learning objectives and class activities, the approach used for virtual teaching, and the activities used to build an effective online learning environment. The results of the emergency transition efforts and the
subsequent learning experience were evaluated by collecting information via two online questionnaires given to students toward the end of the semester upon IRB approval (IRB-20-95), and analyzing spring 2020 and fall 2019 midterm, assignment and semester grades. Online Instruction- The shutdown in March of 2020 led higher education institutions to take an unforeseen emergency action of adapting the F2F instruction with some form of online instruction or a combination of both in a short period to sustain the education system (Bozkurt & Sharma, 2020; UNESCO, 2020a, 2020b). This study discusses the work done to launch emergency online courses and maintain the continuity of instruction. For a better understanding of the results, it is important to provide context and make a clear comparison between online asynchronous instruction, online synchronous instruction, and hybrid instruction, while contrasting them to emergency online instruction. Asynchronous online learning or instruction takes place on the learners’ schedule. The learner takes the responsibility to access all prepared materials, complete assignments and take exams within the course deadlines. This mode of learning is characterized by its flexibility, independence of location, and self-pacing. However, typically there is limited to no contact with the instructor or other learners, collaboration and discussion, or social interaction. Thus, asynchronous online learning requires the learner to be self-motivated to complete the experience (Bozkurt & Sharma, 2020). In addition, the time and resources required to design and build a well-developed asynchronous course are considerable (Bozkurt & Sharma, 2020). It should be noted that in response to the emergency actions taken in March of 2020, there was no time for such course development and planning. On the other hand, synchronous online learning or instruction takes place at a fixed schedule and in real-time where the instructor and learners interact in diverse and planned ways. Class engagement can happen via discussion (e.g., written chat, verbal communication, etc.), and with immediate feedback (e.g., via polls, verbal comments, etc.) from instructors or peers. It allows spontaneity and flexibility of the questions, answers, or content presented; and it promotes immediacy or community development. The rigidity of the schedule and potential technical difficulties are drawbacks of synchronous online learning environments. The time and resources required to design and build a well-developed synchronous course are considerable and such courses are significantly different from the synchronous courses offered at university as a response to the COVID-19 crisis (Hodges et al., 2020). Hybrid learning is defined as the use of online learning in combination with F2F learning activities. This learning environment has some flexibility in the time to review the materials, some degree of contact with instructor and peers, and the benefits of both asynchronous and synchronous tools that can be part of the instructional design (Garcia et al., 2014; Oh & Park, 2009). In contrast, emergency online instruction has been defined as the transferring of courses that were designed for F2F instruction to be taught in the online environment in a short period in response to a crisis. This form of instruction is characterized as temporary, where students do not have a choice, and that may be done with limited resources (Bozkurt & Sharma, 2020; Golden, 2020; Hodges et al., 2020). While for its delivery the emergency online instruction can include
the use of online synchronous and asynchronous tools (already existing or created individually to aid the rapid transition), it is important to note that it is different from established, well-planned online learning. This paper describes the results of the emergency online instruction transition that used synchronous meetings and a combination of synchronous and asynchronous tools to engage students in the virtual classroom. Online learning environment- The development of technology and the internet have promoted a rapid change in instructional and pedagogical approaches used to engage digital learners (Chyr et al., 2017). Online instruction (i.e., fully online or hybrid) has been used to effectively serve different types of learning needs (e.g., distance learning, active learning, collaborative learning, etc.), and to promote students’ continuous development, reflection, discovery, and innovation (Chyr et al., 2017; Tsa et al., 2015; Wei et al., 2015;). Nonetheless, while there is a lot of evidence of the advantages to online pedagogy, it has been reported that online learning can promote student segregation, which could negatively affect the learning experience and create frustration due to the lack of sense of community (McInnerney & Roberts, 2004). Successful online learning environments or online communities of learning have been characterized by appreciation of the social context and by the sense of self, that each of the virtual learners chooses to bring to the online interaction (McInnerney & Roberts, 2004). In online environments, projection of oneself and sense of belonging have been reported to help students develop a sense of community based on a mutual purpose that facilitates and makes the learning process more enjoyable (Garcia et al., 2014; Garrison, 2007; McInnerney & Roberts, 2004). Past studies have found that college students are not fully satisfied with online courses and prefer F2F interaction (Oh & Park, 2009; Sikora & Carroll, 2002). In addition, some literature indicates students’ attitudes toward learning are associated with their perception of the learning experience regardless of the teaching delivery method (Ginns & Ellis; 2007). Based on the existing literature this study focused efforts on using the learning environment developed during the F2F portion of the semester to counteract the potential negative attitude or fear towards the sudden and forced online instruction. The authors have defined an effective learning environment as the one that successfully implements the characteristics of the ASCE ExCEEd model described in later section (Estes et al., 2005). The institution- Cal Poly Pomona is primarily a teaching institution that values experiential learning and community engagement. The courses evaluated in this paper are taught in the College of Engineering, with a learn-by-doing mission founded in face-to-face, field, and laboratory activities. Faculty members have a load of 12 Weighted Teaching Units or four to five one to three-unit course load per semester, with face-to-face being the traditional teaching format. While Cal Poly Pomona has promoted the creation of hybrid courses and provided training opportunities to faculty members via the Engagement of Digital Student Initiative, the great majority of the courses were in-person until the 2020 world pandemic occurred. Before March 2020, Cal Poly Pomona’s faculty used technology to enhance their teaching or to deliver content, but with required face-to-face interaction. The College of Engineering with its Making Imagination Real motto has promoted active learning and hands-on projects. The civil engineering (CE) program, with its approximated 1,500 students, has had the culture of encouraging students to collaborate in academic and extra-
curricular projects and activities from freshmen to senior year. Before the global pandemic, all
the classrooms, computer laboratories and experimental laboratories were always in high demand
for teaching and student activities. The civil engineering curriculum includes hands-on
experience in surveying, materials, environmental, geotechnical, structural, geospatial
engineering and engineering graphic design and modeling (e.g., BIM, AutoCAD, HEC-RAS,
etc.). The transition to the virtual classroom stopped the high occurrence of peer-peer and
faculty-student interactions, the in-person collaboration, and prevented instruction based on
sensory and experimental learning experiences. The courses evaluated in this study are required
courses for the students in the BS CE environmental option, and elective courses for those in the
BS CE General option. The courses are described in detail in the section below.
The CE faculty members had an institutional issued laptop with the software needed for
preparing lecture/laboratory materials and to be used for F2F instruction. In addition, the CE
department had issued iPads or Microsoft Surfaces in spring of 2019 or in earlier academic years.
Once the campus was closed, faculty members had to stay home and figure out how to work with
the technology at hand to prepare and resume instruction in less than a week (three business
days). This paper discusses the resources of only one faculty member who was the instructor of
the courses described below.
Course Description- Two different courses were used in this study, Environmental Engineering
(EE) (2 sections, 72 students) and Water and Wastewater Engineering (WW) (1 section, 26
students), and both courses are senior level courses part of the CE curriculum. Out of the ninety-
eight students, ninety-seven students were seniors, and one student was a junior. While the
course content and learning objectives of each course are different, they have the following
commonalities:
• Taught by the same instructor.
• Project-based for a portion of the semester.
• Project teamwork in-class time.
• In-class group activities.
• Modeling of problem-solving process.
• Practice problems- highly recommended for preparing quizzes, midterms and final exam.
• Additional practice problems-for those who want to develop mastery of the topics.
The teaching approach in the F2F version of both courses was based on the ExCEEd Model that
includes six different elements that describe what constitutes good teaching (Estes et al., 2005).
The six elements are structured organization (SO), engaging presentation (EP), enthusiasm
(ENT), positive rapport with students (PRP), frequent assessment of student learning (FAL), and
appropriate use of technology (AUT). Literature indicates that the use of the six elements of the
ExCEEd Model has resulted in better student experiences in the classroom and that are reflected
in the teaching evaluations (Estes et al., 2006). Table 1 shows broad information on how the
ExCEEd model elements were implemented in the F2F and virtual classrooms.
Table 1- Elements of the ExCEEd model that were implemented in the F2F and virtual
classrooms.
ExCEEd F2F classroom elements Virtual classroom elements
Model
Elements
Structured 1. Learning objectives for the course and 1. Modified learning objectives.
organization each topic. 2. Expanded LMS content.
2. LMS organized per topic. 3. Existing printed course workbook.
3. Printed course workbook. 4. Prepared group activities in breakout
4. Prepared group activities. rooms.
Engaging 1. Learning student names. 1. Using student names.
presentation 2. Walk around classroom to have contact. 2. Zoom tiles with photo and preferred
3. Asking questions and building boards name.
with student input. 3. Asking questions and building e-White
4. Use of physical demos, photos, tours, 3D boards with student input.
demos, videos. 4. Use of virtual tours, photos, 3D demos,
and videos.
Enthusiasm 1. Relevance of topic with students’ and 1. Relevance to topic with students’ and
professor’s personal experiences, videos, professor’s personal experiences, videos,
photos and public reports. photos and online public reports.
2. Changing voice volume and tone and 2. Changing voice volume and tone and
smiling. smiling.
3. Showing passion for the profession 3. Sharing screen with zoom and playing
(personal experiences, voice volume, music and videos at the start of the class.
etc.). 4. Instructor’s camera on.
Positive 1. Playing music at the start of the class. 1. Zoom orientation session to demonstrate
rapport with 2. Asking for team photos. how the virtual classroom would work
students 3. Starting and ending class talking about and to give a LMS orientation.
their interests. 2. Playing music at the start of the class.
4. Inviting students to share their work on 3. Asking for virtual team photos.
the board. 4. Starting and ending class talking about
5. Inviting to visit office hours. their interests.
6. Giving extra credit for attending speaker 5. Inviting students to share their work on
presentations at student organizations the Zoom screen.
virtual meetings. 6. Inviting to Zoom office hours.
7. Giving extra credit for attending speaker
presentations at student organizations
virtual meetings.
8. Chat to have direct one-on-one
communication.
Frequent 1. Planned questions. 1. Planned and unplanned questioning.
assessment 2. Quizzes. 2. Quizzes.
of student 3. In-class competitions. 3. Use of virtual class breakout rooms for
learning competitions and polls.
Appropriate 1. Projector to show photos, videos, tables, 1. Zoom share to show photos, videos,
use of figures, 3D demos, etc. tables, etc.
technology 2. Whiteboard/projector used to deliver 2. OneNote on iPad as a white board.
pre-prepared content and to model 3. Use of prepared content and handouts.
problem solving.
The transition from F2F to the virtual classroom happened in three business days. Day 1 consisted of the modification of the learning objectives (LOs) to eliminate the ones that were associated with a field tour or that required the use of specific physical demos. One of the courses required the elimination of the second project, which provided the mastery of a specific skill, to give more time for completion of the first project. Thus, the corresponding learning objective was modified to reflect the change. The syllabus was updated to include the modified LOs, ways to access the online classroom, and other virtual classroom guidelines. Day 2 was used to set up the new classroom (like testing the pairing of the iPad and laptop). Days 1 through 3 were used to expand the content and organize the learning management system (LMS). A meeting with students was held on Day 2 to give them opportunity to login to Zoom (and troubleshoot their devices), an orientation of the LMS was given, a demonstration of how the class would be delivered was given, and opportunity for students to ask questions or to test their devices was provided. The Zoom orientation served two purposes: 1) giving the opportunity to check the technical aspects of the new virtual classroom and 2) checking-in with the students how they were feeling (opportunity to build rapport), and reassuring them that the learning environment experienced on campus could be transferred to the virtual classroom (Garcia et al., 2014; Garrison, 2007; McInnerney & Roberts, 2004). While the F2F contact was lost, engaging students in the virtual classroom was done similarly in both environments. Students were asked to place a professional photo in their Zoom profile and to add their preferred name. Breakout rooms were pre-assigned and used for group work. Upon completion of the activities students reported results in the main session. Students were asked to share their screens or annotate on the instructor’s board to participate in class. One-on-one communication increased significantly with the private chat capability of Zoom. The rapport built with students in the F2F classroom facilitated the transition and redevelopment of the learning community in the virtual environment. The F2F and virtual classrooms are shown in Figure 1. Students had the required printed version of the course workbook and that helped them in both the F2F and virtual classrooms for keeping their notes organized per topic, and in one place. Class assessment was conducted as frequently as in the F2F classroom, and the virtual room gave the opportunity to easily incorporate polls to enhance participation and to check-in. The biggest challenge could have been the delivery of the instruction; however, the use of an iPad paired to a laptop connected to a monitor, allowed for an easy way to provide instruction in the e-white board while displaying students’ Zoom tiles (Figure 2). The use of the e-white board allowed a written/graphic communication like the one used in the white board of the F2F environment. The positive rapport established with students in the F2F classroom was used to create a smooth transition, and to adapt the learning community established before the pandemic into the virtual environment. The technological resources provided by Cal Poly Pomona and the CE department along with the use of the six elements of the ExCEEd teaching model were the foundation for the adaptation of the F2F teaching materials for instruction in the virtual classroom.
A B
C D
E F
Figure 1- The F2F classroom is shown in Figures A and B. Figure A- students forming teams
and taking photos, and Figure B- classroom celebrating instructor’s birthday. The virtual
classroom is shown in figures C through F. Figures C and D show students’ presentations, and
figures E and F show the e-white board material.
Figure 2- Virtual classroom set up. Methods-This study investigated the effectiveness of the emergency transition from the F2F into the virtual learning environment. The courses studied had a high level of collaboration and hands-on learning components that were eliminated or modified due to the pandemic. The study compared assignments, project, and end-of-term course grades between F2F and virtual versions of the courses, and used the results of the online voluntary and confidential questionnaires. The questionnaires were made available to students as an optional extra credit activity and were made available once the project IRB protocol was approved. Questionnaires included four open-ended questions where students provided more information regarding: 1) things that students had done particularly well in this online/virtual class, 2) challenges faced by students during the online/virtual class, 3) things that enhanced their learning experience, and 4) things that could be changed in the future virtual terms (to enhance the learning experience). In addition, the Spring 2020 average rating for the courses instructional assessment (when the emergency transition occurred) and for past F2F courses were compared to see the impact on instruction assessment ratings. The specific aim of the study was to test the effectiveness of teaching engineering in F2F and virtual formats. The independent variable was the class format (online or F2F) and the dependent variables included students’ assignments, course grades, and end-of-term evaluations of their perceptions of instructor effectiveness and their learning experiences. Hypothesis 1: It is predicted that the learning objectives will be met regardless of the format in which the class was taught. To test the hypothesis students answered five questions about learning objectives in the F2F class and one question in the online class. The first question asked about the quality of the learning objectives, rated on a scale from 0 (Very Poor) to 4 (Very Good). Students were asked to rate their level of agreement on a scale from 1 (Strongly Disagree) to 4 (Strongly Agree) with three statements including: 1) using the learning objectives to prepare for class, test, and quizzes; 2) the effectives of the learning objectives for preparing for class, midterms, and quizzes; and 3) whether the learning objectives were critical to learning. Finally, students indicated with a Yes or No response whether the instructor used learning objectives for class and lessons. In response to the virtual class, students rated the question, “To what extent did your instructor organize the online/virtual class around the course learning objectives?” on a scale from 0 (Not at all) to 4 (Completely).
Hypothesis 2: It is predicted that students will engage in discussion, group work and project
development in an effective way regardless of the format in which they communicate. To test the
hypothesis, students were asked to rate seven items regarding their experiences with the design
project in the virtual environment, using a scale from 0 (Not at all) to 4 (Completely). The
questions led with the statement, “In the virtual environment, to what extent did the design
project allow you to…” followed by each statement including: 1) experience and practice a skill;
2) practice your technical skills; 3) practice communication skills; 4) experience effective team
dynamics; 5) observe a process (a series of units that together work as a whole); 6) execute data
analysis; and 7) collaborate with peers.
Hypothesis 3: It is predicted that students will perform similarly to those who have
completed the course F2F. To test the hypothesis, students’ grades on assignments and end of
course grades were compared. Students were also asked to answer five questions regarding their
preferences and their impressions of the virtual learning including: 1) After this experience in the
virtual classroom, what is your view of online/virtual learning in Engineering? 2) Thinking about
your performance in this online/virtual class, how do you think your course grade compares with
what you would have earned in a face-to-face version of this class? 3) Compared to a face-to-
face class, how do you think taking this class online/virtually affected your motivation to learn, if
at all? 4) Compared to a face-to-face class, how do you think taking this class online/virtual
affected the amount of time you spent in the class to be successful, if at all? 5) If you had to take
this class again, which format would you prefer?
Hypothesis 4: It is predicted that the teaching method used in the F2F environment can be
effective in the virtual environment. To test the hypothesis ten questions related to virtual
instruction were asked. The questions were linked to one or more of the six elements of the
ExCEEd teaching model [structured organization (SO), engaging presentation (EP), enthusiasm
(EN), positive rapport with students (PRS), frequent assessment of student learning (FAL), and
appropriate use of technology (AUT)]. The questions included:
1. Did your instructor ask questions during the virtual class meetings? (EP).
2. Where the questions asked by your instructor in the virtual classroom clear? (EP).
3. To what extent did your instructor organize the online/virtual class around the course
learning objectives? (SO).
4. To what extent did your instructor provide a variety of assignments to assess your
learning based on your learning style? (SO).
5. To what extent did your instructor show enthusiasm in the virtual classroom? (EN).
6. To what extent did your instructor provide engaging presentations (lectures) in the virtual
classroom? (EP).
7. To what extent did your instructor communicate clearly in the virtual classroom? (EP)
8. To what extent did your instructor communicate clearly via Blackboard (Bb) course
posted information, Bb announcements, and Bb emails? (SO, PRS).
9. To what extent did your instructor give timely and accurate feedback?(FAL).
10. The instructor used Blackboard tools or other technology to provide written feedback on
graded online assignments (quizzes, exams, project, etc) (FAL, AUT).Hypothesis 5: It is predicted that students learning in the virtual environment will be able to develop a positive rapport with their peers and instructor. To test this hypothesis, students were asked if the instructor had demonstrated positive expectations. Students answered five questions using a scale from 0 (Not at all) to 4 (Completely) including: 1) To what extent did you have an opportunity to interact with your instructor during the virtual teaching phase? 2) To what extent did you have an opportunity to interact with your peers during the virtual class meetings? 3) To what extent did you have an opportunity to interact with your peers outside of the virtual class meetings? 4) To what extent did you develop a positive relationship with your instructor? 5) To what extent did your instructor care about your success in this course? Results and Discussion- The results of the online questionnaires and of the grade distribution are shown and discussed below. A total of 66 students completed an online questionnaire that asked questions regarding their experience before the COVID-19 pandemic stay-home order, and 69 students completed an online questionnaire regarding their experience after the stay-home order. The analyses compare students’ responses related to their experiences in the F2F and online classrooms. A series of one-sample t-tests were computed to compare each average rating to the scale midpoint to show whether students rated the item/question/statement favorably or unfavorably. Hypothesis 1: It is predicted that the learning objectives will be met regardless of the format in which the class was taught. Table 2 shows the questions and results of the analysis. Most students in the F2F classes rated the class learning objectives as Very Good (50%) or Good (43.9%). The mean of 3.44 (SD = .611) was significantly higher than the scale midpoint of 2.0 (Satisfactory), t(65) = 12.493, p = .001, indicating the learning objectives were rated positively. Students reported using the learning objectives to prepare for class, exams, and quizzes (M = 4.00, SD = .945), t(65) = 8.60, p = .001, reflecting the usefulness of the learning objectives. Students also reported the learning objectives were effective, with an average rating of 3.93 (SD = 1.012), which was significantly higher than the scale midpoint of 3.0 (Unsure), t(65) = 8.600, p = .001. However, many students (43.9%) reported they were unsure whether the learning objectives were critical to their success in the class, with an average rating of 3.212 (SD = 1.016). Over a third (37.9%) of the students agreed the learning objectives were critical to their success. Lastly, 97% (n = 64) of students reported the instructor used the learning objectives in each F2F class or lesson. In response to the virtual classroom experience, the average rating of the instructor’s use of the learning objectives was 3.21 (SD = .886), which was significantly greater than the scale midpoint of 2.0 (Moderately), t(65) = 11.114, p = .001. In summary, Hypothesis 1 was supported.
Table 2-Questions asked to evaluate Hypothesis 1.
Questions Mean (Standard Deviation ), t-test
1. Rate the quality of the learning objectives, 3.44 (0.611), t(65) = 12.493, p = .001
rated.
2. Use of the learning objectives to prepare 4.0 (0.945), t(65) = 8,60, p = .001
for class, test, and quizzes.
3. Effectiveness of the learning objectives 3.93 (1.1012), t(65) = 8.60, p = .001
for preparing for class, midterms, and
quizzes.
4. Were learning objectives critical to 3.212 (1.016), with 43.9% students not sure,
learning? and 37.9% students agreed
5. Did the instructor used learning objectives 97% students reported yes
for class and lessons?
6. To what extent did your instructor 3.21 (SD = .886), t(65) = 11.114, p = .001
organize the online/virtual class around
the course learning objectives?
Hypothesis 2: It is predicted that students will engage in discussion, group work and project
development in an effective way regardless of the format in which they communicate. To test
this hypothesis, students were asked to rate their experiences with the design project in the
virtual environment. The results shown in Table 3 indicate that for all but one question, students’
responses were above the scale midpoint of 2.0 (Moderate).
Table 3-Questions asked to evaluate Hypothesis 2.
Questions Mean (Standard Deviation ), t-test
1. To what extent did the design project 2.24 (1.161), t(67) = 1.672, p = .099 (not
allow you to experience and practice a significantly different than midpoint)
skill?
2. To what extent did the design project 2.41 (1.162), t(67) = 2.922, p = .005
allow you to practice your technical skills?
3. To what extent did the design project 2.55 (1.034), t(66) = 4.372, p = .001
allow you to practice communication
skills?
4. To what extent did the design project 2.37 (1.021), t(67) = 2.970, p = .004
allow you to experience effective team
dynamics?
5. To what extent did the completion of the 2.60 (.986), t(67) = 4.958, p = .001
design project allowed you to observe a
process?
6. To what extent did the design project 2.43 (1.15), t(67) = 3.058, p = .003
allow you to execute data analysis?
7. To what extent did the design project 2.66 (1.087), t(67) = 5.019, p = .001
allow you to collaborate with peers?The average ratings for practicing technical skills (M = 2.41, SD = 1.162), t(67) = 2.922, p =
.005, communication skills (M = 2.55, SD = 1.034), t(66) = 4.372, p = .001, effective team
dynamics (M= 2.37, SD = 1.021), t(67) = 2.970, p = .004, observing a process (M = 2.60, SD =
.986), t(67) = 4.958, p = .001, executing data analysis (M = 2.43, SD = 1.15), t(67) = 3.058, p =
.003, and collaboration with peers (M = 2.66, SD = 1.087), t(67) = 5.019, p = .001, showed a
significant difference from the midpoint of 2.0. In summary, students engaged effectively in
discussion, group work and project development regardless of the class format, thus Hypothesis
2 was largely supported.
Hypothesis 3: It is predicted that students will perform similarly to those who have
completed the course F2F. The project grades for the F2F and virtual courses were compared
(See Table 4). The mean average grades were not significantly different for F2F-WW with a
mean of 81.87 (SD = 19.33) compared to Virtual-WW with a mean of 83.46 (SD = 18.15); and
for F2F-EE with a mean of 87.56 (SD = 6.60) compared to Virtual-EE with a mean of 87.78 (SD
= 10.84). This suggests that students completed the project with a satisfactory grade. The
completion of the project required engaging in discussion, group work and planning the project
development. Thus, students’ responses and project grades indicate that Hypothesis 3 was
supported.
The final grades for both F2F and virtual courses were compared (See Table 4). The mean
average grades were not significantly different for F2F-WW with a mean of 73.21 (SD =16.86)
compared to Virtual-WW with a mean of 76.94 (SD = 9.88). However, when comparing F2F-EE
with a mean of 75.32 (SD = 7.47) to the virtual-EE with a mean of 83.9 (SD =12.4), it suggests
that students in the virtual EE performed slightly better, which conflicts with the overall student
perception that the virtual environment would have a resulted in a “slightly worse” course grade.
For both courses the grades distributions (Table 4) were similar for the group project, while the
grades for individual tasks were overall slightly higher in the virtual environment when
compared to those in the F2F environment (standard deviations slightly overlapping). This could
be the result of the availability of online resources or other factors that are not identified by the
research questions in the study. Thus, in summary, the results from student work indicate that
students in the virtual environment performed as well as those in the F2F environment and
Hypothesis 3 is supported.
Table 4- Grade distribution from the virtual online term and from a F2F term.
Emergency Virtual Teaching F2F Teaching
WW (n = 26) EE (n = 72) WW (n = 26) EE (n = 38)
M SD M SD M SD M SD
Assignments 63.4* 11.96 83.63* 10.92 71.29 15.81 73.53 10.49
Project 83.46* 18.15 87.78* 10.84 81.87 19.33 87.56 6.60
Midterm 1 70.75 15.98 70.5 19.4 62.02 20.76 60.79 16.22
Midterm2 83.46* 18.15 92.2* 18.5 77.63 22.43 68.62 12.06
Final Exam 76.07* 12.25 83.7* 17.6 75.46 18.23 77.50 11.07
Final grade 76.94* 9.88 83.9* 12.4 73.21 16.86 75.32 7.47
*assignment or assessment completed during the virtual emergency teaching period.In addition, students rated their views of the virtual learning experience. Students’ responses to the question “What is your view of online/virtual learning in Engineering” was not significantly higher or lower than the Neutral (0) scale midpoint, t(66) = 1.229, p = .223. However, 32.8% of students indicated that they viewed the virtual learning as somewhat negative, 28.4% reported to have a neutral view, and about one third of the students rated their view as “Positive” to “Very positive.” Students’ responses to how their expected grade in the virtual class compared to the grade in the F2F class was not significantly higher or lower than the “About the Same” (0) scale midpoint, t(66) = 1.033, p = .305. About 46.3% of the students reported that they thought that the grade obtained was the same as if they would have had the F2F version of the class, while 20.9% reported the grade obtained in the virtual course to be slightly worse. In terms of motivation to learn, students’ average rating of -1.00 (SD = 1.015; Somewhat less motivated) was significantly lower than the scale midpoint of “About the Same (0)”, t(67) = 8.064, p = .001. Most students (41.8%) reported virtual learning had a “Very Negative” effect on their motivation to learn compared to the F2F class, followed by “Somewhat Negative” (25.4%) and “Neutral” (23.9%). Regarding the time spent in the course to be successful, the average rating of -0.44 (SD = .862; a little more time) was significantly lower (indicating more time spent) than the scale midpoint of “About the same (0)”, t(65) = 4.143, p = .001. 13.6% reported to have used “A lot more time,” while 27.3% and 48.5% indicated that they used “somewhat more time” and the “same” as in a F2F course, respectively. Finally, the study results indicate that if students had to retake the course again, 50% would select the F2F environment, 19.7% preferred the virtual environment, and 15.2% would prefer the course to be blended. This is consistent with past studies that report that college students are not fully satisfied with online courses and prefer F2F interaction (Oh & Park, 2009; Sikora & Carroll, 2002). Hypothesis 4: It is predicted that the teaching method used in the F2F environment can be effective in the virtual environment. To provide continuity to the course and seeking student engagement in the virtual environment, the instructor made efforts to intentionally use the ExCEEd teaching model elements as the transition from F2F was done. The elements that were prioritized were structured organization (SO), engaging presentation (EP), enthusiasm (ENT), and frequent assessment of student learning (FAL). With the organization of the course using the LMS, the class delivery and engagement of students as the main elements to drive the transition. Students’ responses (Table 5) regarding the organization around learning objectives had a mean of 3.12 (SD = 0.937) which is significantly higher than the scale midpoint of 2.0, and t(65) = 9.724, p = .001, this indicates that the organization was satisfactory. Similar results were observed for the assessment of different learning styles (M = 2.58, SD = 1.151), t(65) = 4.064, p = .001, and communications via email and LMS that supported the course organization (M =3.11, SD = 1.010), t(65) = 8.900, p = .001. Thus, the results indicated that the course organization was effective, supporting Hypothesis 4.
The engaging presentation element of the ExCEEd model was linked to the questions related to
written and verbal communication. Resulting mean values were higher than the scale midpoint of
2, indicating that the students perceived that the instructor asked questions (M = 3.26, SD =
0.997), t(65) = 10.246, p = .001, during the virtual classes and that the questions were clear (M =
3.12, SD = 0.937), t(65) = 9.724, p = .001. Students rated the presentations as engaging with a
mean of 2.86 (SD = 1.094), which is greater than the scale midpoint of 2.0, t(65) = 6.414, p =
.001, that indicated that students were involved in the synchronous session. The results support
Hypothesis 4 that states that the F2F teaching method can be effectively used in the virtual
environment.
Table 5- Questions asked to evaluate Hypothesis 4.
Questions Mean (Standard Deviation), t-Test
1. Did your instructor ask questions during 3.26 (0.997) , t(65) = 10.246, p = .001
the virtual class meetings? (EP).
2. Were the questions asked by your 3.12 (0.937), t(65) = 9.724, p = .001
instructor in the virtual classroom clear?
(EP).
3. To what extent did your instructor organize 3.21 (0.886), t(65) = 11.114, p = .001
the online/virtual class around the course
learning objectives? (SO).
4. To what extent did your instructor provide 2.58 (1.151), t(65) = 4.064, p = .001
a variety of assignments to assess your
learning based on your learning style?
(SO).
5. To what extent did your instructor show 3.38 (.780), t(65) = 14.364, p = .001
enthusiasm in the virtual classroom? (EN).
6. To what extent did your instructor provide 2.86 (1.094), t(65) = 6.414, p = .001
engaging presentations (lectures) in the
virtual classroom? (EP).
7. To what extent did your instructor 3.11 (0.930), t(65) = 9.659 p = .001
communicate clearly in the virtual
classroom? (EP).
8. To what extent did your instructor 3.11 (1.010), t(65) = 8.900, p = .001
communicate clearly via Blackboard (Bb)
course posted information, Bb
announcements, and Bb emails? (SO).
9. To what extent did your instructor give 2.85 (0.942), t(65) = 7.396, p = .001
timely and accurate feedback?(FAL).
10. The instructor used Blackboard tools or 2.98 (1.015), t(65) = 7.882, p = .001
other technology to provide written
feedback on graded online assignments
(quizzes, exams, project, etc.) (FAL, AUT).
The ratings for instructor enthusiasm (M = 3.38, SD = .780), t(65) = 14.364, p = .001, and timely
and accurate feedback (M = 2.85, SD = 0.942), t(65) = 7.396, p = .001, were also greater than the
scale midpoint of 2.0, indicating that students felt that the instructor brought energy to the virtualclassroom, and that the feedback provided supported their learning process. Lastly, students rated
the use of different technology to provide feedback well above the scale midpoint 2.0
(moderately) with a mean of 2.98 (SD = 1.015). Based on the findings, it can be concluded that
the instructor was able to incorporate some of the elements of the ExCEEd model used in the
F2F and applied them during the emergency transition to virtual teaching. Moreover, results
suggest that using the ExCEEd model positively influenced the virtual learning environment.
Thus, the results support Hypothesis 4.
Hypothesis 5: It is predicted that the students learning in the virtual environment will be
able to develop a positive rapport with peers and the instructor. The instructor demonstrated
positive expectations by keeping the camera on during instruction time and by asking students to
add their professional picture to the Zoom profile, and their preferred name in the Zoom tile. In
addition, the instructor did check-ins to support students with the confusion caused by the
pandemic. Instructor used confidential polls and private chat to encourage students’
participation. In addition, as a response to the lack of attention of some students during the
synchronous class, the instructor added, as extra credit, the submission of the notes taken during
the virtual class. The questions directed to measure the degree of interaction between students
and instructor were ranked well above the scale midpoint of 2.0 (Table 6). The rating of
interacting with the instructor during the virtual teaching phase had a mean of 3.84 (SD = 1.053),
t(66) = 6.497, p = .001; and the rating related to whether the instructor cared about student
success in the course had a mean of 3.23 (SD = 1.020), t(65) = 9.777, p = .001. Results suggest
that most students felt that that the professor was in touch with each student and wanted their
success in the course. The positive relationship with the instructor was rated with a mean of 2.78
(SD = 1.071), t(66) = 5.934, p = .001, which indicates that the actions taken by the professor in
and outside of the virtual classroom supported the development of interpersonal rapport
supporting Hypothesis 5.
Table 6- Questions asked to evaluate Hypothesis 5.
Questions Mean (Standard Deviation ), t-Test
1. To what extent did you have an 3.84 (1.053) , t(66) = 6.497, p = .001
opportunity to interact with your
instructor during the virtual teaching
phase?
2. To what extent did you have an 1.99 (1.121), t(66) = -.109, p = .914 (not
opportunity to interact with your peers significantly different than midpoint)
during the virtual class meetings?
3. To what extent did you have an 2.06 (1.192), t(66) = .410, p = .683 (not
opportunity to interact with your peers significantly different than midpoint)
outside of the virtual class meetings?
4. To what extent did you develop a positive 2.78 (1.071), t(66) = 5.934, p = .001
relationship with your instructor?
5. To what extent did your instructor care 3.23 (1.020), t(65) = 9.777, p = .001
about your success in this course?
The instructor used the group projects, collaboration documents and breakout rooms to promote
interaction with peers during and outside of the virtual class. Students rated them both at thelevel of the scale midpoint of 2.0 (moderately). In summary, the results indicate that the
instructor was able to develop positive rapport with students, thus supporting Hypothesis 5.
However, the students’ rapport with peers was moderate, and this could be associated with their
attitudes towards the virtual learning that were “somehow negative” (Ginns & Ellis. 2007).
Questionnaires’ open-ended questions- The open-ended questions were grouped in four
different themes: 1) What do you think you have done particularly well in this online/virtual
class? 2) What challenges did you face in this online/virtual class? 3) List three things that
enhanced your learning experience, and 4) Things that could be changed in future terms (to
enhance learning experience). This section includes a summary of the students’ responses.
Samples of students’ responses are shown in this section and they were directly extracted from
the survey results without editing.
1) What do you think you have done particularly well in this online/virtual class?
Some students reported challenges identifying a learning space at home, some others took this as
an opportunity to be creative and found ways to be part of all learning activities (connecting-
attending class, taking notes, reviewing notes, attending office hours, asking questions in class,
solving problems, doing extra credits, finding ways to manage time, etc.). Students’ comments
suggest that some students were proud to have found a way to be resilient during the virtual
learning period. Selected samples of students’ comments are listed below.
“for example even there was no quite place to study, but i tried to find my quit place and donot
give up.”
“ Trying to maintain class participation and review notes.”
“I attended every class and participated in the online learning process. I would participate in
activities the professor would set up for us. This allowed the students to speak to eachother and
interact with the professor. I also attended office hours which I felt I would have never done in
face-to-face classes. This ease of office hours access was very beneficial.”
“I didn't give up. Regardless of my own personal struggles during the crisis, I was still able to
complete my class and I'm happy that I did.”
“Given the current circumstances, I think I did as well as I could during this online class. I
completed more practice problems during the online learning portion of the semester,compared
to the in class portion. This helped enhance my overall learning of course in formation.”
“I believe I was able to better pay attention in class and I was able to actually understand
concepts because I was not sitting at the back anymore.”
“Keeping track of deadlines for assignments and tests.”, “Keeping myself engaged so I do not
miss out on learning.”“Using zoom to communicate with my team members and hold virtual study sessions was very
effective. I think i performed better on exams than I would have in a face-to-face setting.”
“I attended every class after switching to virtual and was able to stay with or slightly ahead of
the professor on handout problems. I also received higher quiz grades as I felt more confident
with the material and being in my own space without other peer distractions helped with my test
anxiety.”
“I was able to make it to class on-time because there is no need to look for parking.”
“I think the things I have done well are communicating with my group as a lot of collaboration
goes into the project. Meeting on Zoom and Google Docs really helped a lot because that was
one of the main ways the project could get done through this virtual period.”
“I think my group did a good job for our group project. It was harder to communicate and get
some point across, but thankfully other group members had the same ideas that I did. As a
group, we really pulled together to write an excellent report and presentation.”
“I have asked more questions during class since I have less anxiety during virtual class.”
“Time management has been even better”, “I have had to be more organized.”
“Communication with other classmates for the project.”
“that the format of the class remained the same.”
“I think I have made better connections with my classmates because I was forced to
communicate more.”
“I have had to be more independent and motivated.”
“I dont think I have done much particularly well in the online/virtual class.”
“Nothing, virtual sucks.”
“Uploading assignments to blackboard paperless.”
“I did a good job of attending every class and asking questions when i needed to.”
“Studied the topics and completed assignments on time and reviewed class handouts often.”“I did fairly well on exams/quizzes because I was able to prepare for them in a timely fashion. I
was unable to preform well during the transition to virtual because I was unable to have a
proper space and no Wi-Fi at home.”
“Emailing my professors more.”
“I have done a great job attending all of my classes despite it being online. I have heard of many
students who have stopped attending classes ever since we have moved online, however, I have
never skipped a class.”
2) What challenges did you face in this online/virtual class?
The majority of the students’ responses for this question were related to Wi-Fi and technology
issues, the lack of motivation, or the lack of a quiet space at home to learn. Selected samples of
students’ comments are listed below.
“At the beginning of quarantine, I found trouble felling motivated in all my classes, not just
this one, but once I got back into the motion of things it was back to normal. I also live away
from family during the semester, but since the quarantine, I have essentially moved back
home. It was a struggle trying to focus since I subconsciously associated being home as
being on vacation.”
“I faced the challenge of having a quiet place to complete the virtual class. Sometimes my
home can be noisy with my roommates.”
“The only challenged I face in this online class was being able to communicate in
classlectures and not having enough time to submit quizzes. It was difficult to communicate
dueto low connection and it was a challenge to turn in the quizzes considering that we had to
submit a PDF file.”
“Test online are not the same, Finding a quiet time to work on homework and study.”
“Being able to work in a collaborative environment with my peers on projects, Being
motivated to complete work.”
“Seperating school time with home time. It was hard to be motivated when I was at
homecofortable or stressed out but couldn't see my professor in person to ask questions.”
“the project is the biggest challenge because our group did not have the same schedule at all
(some are essential workers during covid19).”
“I could not keep my focus because there are too many distractions at home. Going to
classin-person forces me to pay attention better than virtual class.”3) List three things that enhanced your learning experience.
Many students reported that the learning experience was enhanced with the professor’s attitude,
with the different ways to engage them during the synchronous class, and with the information
provided during class and found in the LMS. In addition, students reported that the commuting
time became time for studying, for resting or working. Per students’ comments, the elimination
of the commuting to campus helped them to increase their contact with the class materials.
Samples of students’ comments are listed below.
“Zoom, solving example problems in real time, professor attitude.”
“Office hours being more flexible and accessible. Being at home having a bigger desk to work
on is better compared to the small desks on campus. 3) Teacher's engagement in the class helped
motivate me.”
“My learning experience was amazing. I learned so much in this class that the project completed
in this class can help in obtaining a full time job.I believe using more 3Danimations that we
students can interact with can be very beneficial. In addition, I believethe project should stay the
same, its a very good project that made me happy to know it completed.”
“Using slides on One-Note. It will make things easier for the instructor and provide better
delivery to the students.”
“frequent blackbaord announcements, gradebook, Zoom.”
“I think the way the lecture was conducted was very good since it was not really different from
how face to face lecture was conducted. I think it might have been more productive since we did
not have to wait for the white-board to be erased or the previous class or meeting to leave the
room.”
“My professor kept a positive attitude the entire class”.
“Three things that enhenced my learning was: workbook problems, project which as really
helpful, extra credits.”
“Some things that enhanced my learning in the virtual environment was the fact that our lectures
were recorded to go back if I had missed something, having access to the online textbook, and
being able to comfortably unmute myself if I did have a question just like how it was in face-to-
face lecture.”
“the ability to rewatch recorded lectures, the time saved from not having to commute, the ease
of "meeting" with group members via zoom.”
“The professor asking questions during class to enhance engagement, continuing with
presentations, group work during class.”
“Blackboard and sectioned modules. Being split into groups. Problems that the professor guided
us through.”“The break room sessions for accomplishing as a team the practice problems, interaction with
other students in break rooms once the practice problems were completed, and having camera
on for presentation.”
“Availability of handouts and PowerPoints online, Recording lectures for future reviewing,
Exams accurately reflecting in-class problems/exercises.”
“less time driving and commuting, more time studying, more time relaxing while studying.”
“Chat box with the professor to answer in-class questions, instructor attitude.”
“Having the professor work through problems with us on line like it was a white board inclass
made the transition to on line much easier.”
“Three things that enhanced my learning experience was being able to clearly see what the
professor was writing, being able to ask questions right away, and completing online quizes with
ease.”
“Screen sharing of handout problem solutions, professor taking student feedback during
lecture.”
“Lots of extra credit and other assignments to keep me engaged.”
4) Things that could be changed in the future terms to enhance the learning experience.
Overall, students indicated that to enhance the learning experience in future semesters the
professor should slow down the pace. Some students thought that it was difficult to communicate
via Zoom breakout rooms, and some others that the project should be completed in a shorter time
or should become an individual task. Few students commented on the importance of considering
the students mental health and well-being, and of providing mental health resources. Finally,
some students didn’t like the music played or that the professor asked questions during the class.
Samples of students’ comments are listed below.
“Calling out on students in the middle of lecture (it can be very stressful and can cause
anxiety).”
‘Student mental health is important to consider. I already had access to resources to help me but
some others may not.”
“ I do not find breakout rooms helpful at all because it often ends up in complete silence. I think
it would be better to use the class time to work through examples being led by the professor.”
“I do not think that quizzes/exams should be multiple choice but rather hand calculation
submittals to help act the same as doing it in person.”
“The group project put a lot of pressure on students at a time of already high stress (finals week
+ online classes + Covid lockdown), I felt like the project should have been due a bit earlier in
the semester, at a time when there was no midterms or finals.”You can also read
