As I have in previous blog articles, I will share here a small section of my doctoral research on the topic of satisfaction with distance learning (DL) experiences in the CAF. Specifically, I will share some quantitative data on CAF members’ satisfaction with various DL course quality variables. This research, which was defended in 2020, included a sample of 368 CAF members who had graduated from CAF Professional Military Education programs between the dates of January, 2015 and March, 2018.

So, to start – I am sure you can relate with some of the following lines, as we have all been there. You finish an online course by successfully passing a final quiz and happily close your laptop with a sign of relief. Sometimes you think about how the course was such a worthwhile and enjoyable learning experience. Other times, however, after many long and grueling pages of content and confusing quiz questions that you complete in total isolation “as a solo endeavor” (Jones, 2020, p. 149), your computer freezes, your final test score didn’t register, you search endlessly for the help line number to find out how to print a certificate, and you finally end your day by slamming down your laptop cover so hard that it makes a sad and somewhat concerning sound. Then, of course, you go on Twitter to complain about the whole experience with your virtual comrades… who can easily empathize 😉

So, what differentiates good e-learning from bad e-learning? Admittedly, I have described the two extremes of potential outcomes here. During my research on CAF member satisfaction with Distance Learning, I did heard a wide range of feedback that included similar scenarios that I have explained to you. Some feedback was positive and some was negative. As is often the case, the majority of experiences fell somewhere between these two extremes.

I started my research with a literature review to try to identify the different variables of course quality that could make a difference in the student’s experience, and in a larger sense, the success of the course.

During the literature review various questionnaires were explored, including Aman’s (2009) Learner Satisfaction Questionnaire that was later also used by Simpson and Benson (2013), in order to consider what should be included in the instruments for my research.  Many of the variables that I used for course quality were inspired by the quality factor questions used by Aman (2009).

Participants, in my survey, were asked to rate their level of satisfaction, on a scale of 1 to 5 (or N/A) where one (1) was “very dissatisfied” and five (5) was “very satisfied”, with the 13 following items related to course quality:

• clear learning objectives
• effective communications with instructor
• interactions with classmates
• feeling of being part of a learning community
• collaborative group work with classmates
• engaging course content
• easily accessible required course materials
• clearly described course assessment
• constructive feedback from instructors on assignments and assessments
• timely feedback from instructors on assignments and assessments
• effective course technology (e.g. in the CAF context – the Defence Learning Network (DLN))
• course materials provided that helped to reach course objectives
• course technology that helped to reach course objectives

Findings

Full results can be found in my dissertation, but here are some of the highlights regarding the course quality variables in relation to CAF member satisfaction.

In combining “somewhat satisfied” and “very satisfied” together, the three variables that ranked the highest for DL course quality satisfaction, in descending order, are as follows: 1) clear learning objectives (88.3%); 2) clearly described course assessments (83.1%); and 3) course materials provided that helped to reach course objectives (80.3%). This is positive and great to see!

On the other hand, combining responses of “somewhat dissatisfied” and “very dissatisfied” together, the three factors that ranked the highest for DL course quality dissatisfaction, in descending order, are as follows: 1) feelings of being part of a learning community (23.4%); 2) collaborative group work with classmates (23.0%); and 3) effective course technology (21.3%). With the fact that the CAF Junior Officer Development program has become self study with no peer or instructor interactions, the first two on this list are not surprising.

Interesting, in terms of correlations, all of these variables were found to be positively related to course satisfaction overall.

Correlations of Satisfaction with Course Quality Variables with Overall DL Satisfaction

Variables	n	Overall DL Satisfaction
Clear Learning Objectives	367	.523**
Effective Communications with Instructor	317	.441**
Interactions with Classmates	303	.546**
Feelings of Being Part of a Learning Community	342	.579**
Collaborative Group Work with Classmates	293	.553**
Engaging Course Content	364	.529**
Easily Accessible Required Course Materials	363	.409**
Clearly Described Course Assessments	360	.392**
Constructive Feedback from Instructors on Assignments and Assessments	320	.444**
Timely Feedback from Instructors on Assignments and Assessments	315	.439**
Effective Course Technology (e.g. DLN)	364	.549**
Course Materials Provided that Helped to Reach Course Objectives	364	.472**
Course Technology that Helped to Reach Course Objectives	365	.557**
** p < .01 (two-tailed); N/A answers have been treated as missing values.

After exploratory factor analyses and multiple regression analyses, I found that all three factors that were identified with these variables were significant predictors of overall DL satisfactions.  Specific variables within the three factors that were created were shown to have significant influence on overall DL satisfaction and included variables related to technology; feeling part of a learning community; and effective communications with instructors. (Note: details of these analyses can be found in the full dissertation at reference).

Student satisfaction is important in terms of important organizational outcomes such as readiness to transfer learning, levels of absenteeism, retention, and students’ intention to recommend the training to others (Jones, 2020, p. 11-16). As these variables have been been found to be positively related to student satisfaction and in some cases, even predictive of satisfaction, it is important to make sure that we are working to improve upon these in our online learning development and delivery.

One way to ensure quality in our e-learning would be to implement a course quality rubric that all courses must pass through in order to be launched in the organization. This could potentially be based on an existing and already validated rubric standard (such as is available by the Quality Matters Organization) or built taking into consideration the organization’s specific context and requirements. A standard minimum score could be adopted, and each course would be evaluated by a group of training/ Quality Assurance experts, prior to the DL course being used launched to the training audience.

On a somewhat positive note, 73.6% of participants (n=359) in my research responded that they agreed with the statement that the quality of DL in the CAF has increased over the past ten years. We must always strive to improve, though, and impress upon the other 26.4% that DL in the CAF is keeping up with the standards seen in academia and industry. For my Defence Team colleagues, who mainly include CAF Training Development Officers, CAF and civilian course designers, developers, and facilitators, as well as our e-learning technologists, we must take note of all of these important variables of course quality, continue to improve our skills and keep an eye open to potential technical and methodological advances, and always, always embrace continual improvement.

Reference

Jones, K.A. (2020). Satisfaction of Canadian Armed Forces Regular Forces Members with their Distance Learning Experiences [Dissertation]. Athabasca University.