Have you ever tried to complete a Distance Learning (DL) course when the technology became your focus, either because of the slowness or crashing of the system, the non user-friendly design of the interface, or the timing-out of a quiz where you lost all of your work? Me too! On the other hand, have you ever completed a learning experience at a distance where the technology was seamless and really seemed to enhance the learning experience? Me too! Let’s have a look at what some Canadian Armed Forces (CAF) members had to say about their satisfaction (or lack thereof) with technology as an aspect of course quality.
As I have in previous blog articles, I will share here a small portion of my doctoral research on the topic of satisfaction with DL experiences in the Canadian Armed Forces (CAF). Specifically, I will share some of my quantitative and qualitative findings related to technology, as a course quality consideration related to CAF member DL satisfaction. This research, which was defended in 2020, included a sample of 368 CAF members who had graduated from CAF Professional Military Education programs between the dates of January, 2015 and March, 2018. The data was gathered from surveys, as well as 12 follow-on interviews.
Technology was a subject that arose frequently in both the quantitative and qualitative research data, both as responses to direct questions as well as spontaneous comments in relation to satisfaction and ways to improve DL. For example, when asked about satisfaction with “effective course technology (e.g. DLN),” 66.4% of respondents (n=366) said that they were somewhat or very satisfied, while 21.3% of respondents said that they were somewhat or very dissatisfied. In response to the perception statement, “The CAF has good technical support systems in place to help should any technical problems arise during DL courses,” 41.3% of respondents (n=363) either agreed or strongly agreed with this statement, while 27.0% of respondents either disagreed or strongly disagreed with this statement. The responses to these two questions indicate that a portion of CAF members have negative perceptions regarding DL technologies and technical support in CAF DL. This supports earlier findings from the DND “Your Say” survey research (Budgell, Butler, & Eren, 2013), that found that only 45% of respondents, from a sample of 1730 CAF members, agreed that the CAF makes good use of technology in courses. This, of course, begs the question of what the other 55% of CAF members think we could be doing better in relation to technology use within DL courses.
In the correlation analyses for course quality variables, it is noteworthy that all course quality variables that were measured (e.g. timely instructor feedback, clear learning objectives, easily accessible course materials, etc.) had a positive significant correlations with overall satisfaction. Two of the strongest significant positive correlations with overall DL satisfaction involved the satisfaction with course technology, specifically: 1) course technology that helped to reach course objectives (rs (365) = .557, p < .01); and 2) effective course technology (e.g. DLN) (rs (364) = .557, p < .01). Both of these would be considered of moderate strength. With the multiple regression analyses that were completed, both “Effective course technology” and “Course technology that helped to reach objectives” were shown to be significant predictors of DL overall satisfaction.
The qualitative data supported the quantitative findings that the topic of technology was relevant to CAF DL satisfaction. Technolgy was so prevalent in the qualitative data that it emerged as a theme unto itself. The technology theme included the following four categories: 1) accessibility; 2) usability of technologies supporting DL; 3) learning management systems (LMS); and 4) perceptions regarding DL technology in the CAF. Although some members indicated that the technology to support CAF DL had improved over the years, fewer positive sentiments and experiences concerning CAF DL technologies were shared.
Within the category of accessibility, there were many comments regarding difficulties experienced with DL technology including issues of connectivity and bandwidth. Connectivity was brought up for both office and home settings, but also in operational settings such as on ships and on overseas deployments. Members did mention that they liked the fact that they could access their DL from their homes, outside of their workplace computer. One member said, “the system is very user-friendly because it exists outside of the DWAN [Defence Wide Area Network] system, very easy to use, home computer, home-based internet.” One liked that the technology exists so that they can do DL, “anywhere, anytime.” Another member, however, stated, “I do not have access to reliable internet from my home and must conduct the course at work.” Indeed, Internet bandwidth and reliability in the more rural areas can still be an issue. Other members stated that they had issues with connectivity while trying to do their DL in the office. One member said that “the servers themselves need desperately to be updated. The system struggles greatly with large courses.” Another member commented that the “intranet at work is dead slow.” These issues, one member stated, “often result in complete loss of connection” and that sometimes the system, “does not save the work that was already completed.”
Within the category of usability, items such as the following were brought up: members’ comfort level with the DL technology, DL technology support available (including from a help desk), firewall issues, members’ requirement to use external technology to support their course, and issues encountered such as with the DND search engine, inactivity time-outs, the inability to print courseware, and a vast array of “technical hiccups.” In relation to comfort levels, one member stated that “although I am older, my computer skills and comfort level with software systems are good. I never had any issues with that part of the DL.” On the other hand, one stated that, “regardless of age, not everyone is computer savvy (i.e. a digital native). There needs to be training available that is aimed at every type of learner.” In terms of ease of searching for references on the DWAN Intranet, one member stated that it “was of no use when trying to find reference material.” Another suggested that the DND/CAF should, “invest in upgrading the DND/DWAN to have better browsers and access for research.”
Regarding the Learning Management Systems (LMS), participants discussed the Defence Learning Network 2.0 and/or Moodle, which is being used by the Royal Military Colleges of Canada (RMCC) and the Canadian Forces College (CFC), dependent on the program they had completed. There was a range of satisfaction with these tools. One Junior Officer who was interviewed stated that the DLN is “easy to use,” “very user-friendly,” and that “anybody could do it.” Other members felt that the DLN, however, left some things to be desired. One Senior Non-Commissioned Member (NCM) who I interviewed about their Senior Leadership Program (SLP) experience, for example, stated that the LMS affected the interactions between students in their forum discussions. They stated that “because of the software of DLN… it wasn’t a flow,” adding that “it wasn’t very intuitive or well laid out… it was very cumbersome.”
Another member stated that the DLN, “is a very difficult program to work with… navigating the DLN is terrible and something needs to be changed…. It… creates needless frustration.” Another stated that it creates “stress for no reason trying to navigate it.” Another member commented that the “DLN is very clunky and difficult to find courses and is not very user friendly.” A Senior Officer, who had completed the Junior Command and Staff Programme (JCSP)- DL version, said of the LMS used by CFC, “it was okay, I guess. I’ve seen better, but I’ve seen worse.” Another member said, “I would complete more DL courses if the system was easier to work with.” One Senior NCM suggested, “improving the platform to enable students who are working on that theory portion of the DL so they can actually collaboratively work together if that’s what’s required. So be it from smartphones, from tablets, from work, traveling on the train, traveling in a car, whatever.”
Perceptions related to DL in the CAF, as the fourth category, were quite varied and included members’ general levels of satisfaction and expressions of frustration. Related to satisfaction, one member said the “technology was decent,” and another stated that “as a whole I think that it is getting better.” Some expressions of frustration with the technology, however, were also shared. One Intermediate Leadership Program (ILP) graduate stated, for example, that “the technical hiccups were very distracting and at times infuriating.”
As illustrated in the comments from these CAF members, the various technology components related to DL, as a course quality consideration, can have a positive or negative effect on the DL satisfaction and learning experience of students. To optimize the learning experience of CAF members, we must always strive to improve upon what we have and trial courses on various platforms and browsers prior to launch. Issues found, as well as student feedback, must be addressed. It should be noted that the technology of DL is always evolving and we must strive to keep pace. The DND/CAF is currently in the process of upgrading to a newer version of our Saba Learning Management System in the cloud, known as the Defence Learning Network (DLN) 3.0. I look forward to learning more about the benefits of this new DLN iteration, including its new functionalities. This is a great step forward, as is the availability of the new DLN 3.0 virtual classroom and MS Teams for synchronous group discussions. I look forward to see what future technologies in this space will provide in order to further improve upon the student learning experience.
I would like to thank the CAF members who took part in the surveys and/or interviews in support of this research.
Reference
Budgell, G., Butler, A., & Eren, E. (2013). Task # 138: Regular Force Your-Say Survey: Spring 2012 Focus Selection Results. DRDC-RDDC-2015-C102.
Government of Canada. (2018). Canadian Armed Forces professional development framework
Government of Canada. (2022). The Maple Leaf, The Defence Learning Network 3.0 is here!
Jones, K.A. (2021). Satisfaction of Canadian Armed Forces Regular Fores Members with their Distance Learning Experiences [Dissertation]. Athabasca University.