Category: Research

The Flipped Classroom: What is it? What Does the Research Have to Say?

On By kimjones12In Education, Research, Training1 Comment

What is a Flipped Classroom Anyways?

Have you heard of, participated in, or facilitated a “flipped classroom”? A flipped classroom, sometimes called an “inverted classroom”, is a relatively new instructional model that has been gaining traction. In a flipped classroom, we flip the traditional “sage on the stage” model on its head. Traditionally, the common model of instruction saw a professor/teacher/instructor at the front of the class lecturing to a classroom/amphitheater full of students listening and taking notes. This model can often yield a passive type of learning and there is a good chance that students’ minds will wander and there may be very little interaction between the teacher and the students, and between student peers. We’ve all experienced this type of class.

In the traditional model, students most often listen to a lecture and then leave class to complete problems, practice skills, and do various types of projects on their own. In the flipped classroom, focus is shifted from the instructor to the students during the classroom time. Students do their homework up front, prior to class, with this pre-work focusing on what would have traditionally been the lecture. The lecture content is presented to the students prior to coming to the classroom. Students normally watch a video of the lecture and/or interact with other digital content. All students must complete this work to be prepared for what will take place in the classroom afterwards.

The classroom time is then devoted to active learning, most often targeting the higher cognitive levels of learning objectives such as applying, analyzing or synthesizing. Activities can include collaborative group work based on the new content learned, instructors mentoring students working through problems or scenarios together, group discussions, and hands-on practice for skills. In short, instructors/professors turn into facilitators and, for the students, passive listeners turn into active learners during the scheduled class time.

Research Related to the Flipped Classroom Model

There has been much research done on the effectiveness of this model, across many subject domains. Beyond effectiveness, research has also looked into aspects such as student and teacher satisfaction with this model, the advantages and challenges of this model, and the variety of activities and digital tools being used to support this model.

A meta-analysis of 71 research articles, dated from 2000 to 2016, related to the flipped classroom was completed by Akçayır & Akçayır (2018). In their overview of the findings of this meta-analysis, they found that “In general, the flipped model in education yields positive results” (p. 343) and that the most frequently reported advantage was an improvement in student learning performance.

Common advantages and challenges reviewed in their 71 research studies were listed. In terms of advantages, the areas highlighted included: 1) learner outcomes e.g., improved performance, satisfaction, engagement, and motivation; 2) pedagogical contributions e.g., flexible learning, individualized learning, enhanced enjoyment, better preparation, and fostered autonomy; 3) time efficiency e.g., more efficient class time & more time for practice; 4) dispositions (i.e. attitudes towards) e.g., positive feedback, attitudes, perceptions from students and teachers; and 5) interactions e.g., student to instructor & student to peers.

In terms of challenges, the areas highlighted included: 1) pedagogical e.g., limited student preparation prior to class, students need guidelines while at home, students were unable to get help while on their own, & the inability of teachers to know if the students completed their pre-class work; 2) students’ perspectives e.g., time-consuming & increased workload; 3) teachers’ perspectives e.g., time consuming, higher workload; and 4) technological e.g., quality of videos, inequality of technological accessibility/competency, the requirement for infrastructure such as Internet in remote areas.

They also listed the main activities that they saw being used prior to class and in-class in the flipped classroom model. Some of the most common activities prior to class included: videos, readings, quizzes, asynchronous discussions, and PowerPoint presentations. Some of the most common activities in-class included: group discussions, small group activities, feedback, problem solving, questions & answers, case studies, hands-on experiments, and learning games.

In a more recent meta-analysis, completed by Roehling & Bredow (2021), they found that students in flipped classrooms reported greater satisfaction than those in lecture-based courses. They also found that flipped classrooms produced the greatest academic benefits in courses such as language, technology, and health-science, over, for example, mathematics and engineering courses. Their research also suggested that partially flipped courses can be more effective if, for example, the instructor/developer considers which content/activities lend themselves well to the flipped model and which don’t.

One particle study (Nouri, 2016), noted some additional interesting findings. The study (n = 240 students) showed that 75% of the students left the course with a positive attitude towards flipped classrooms. The research also showed that there was a general appreciation by students on the use of video lectures prior to class. On a Likert scale of 1-5, students found it useful to be able to: 1) pause the video lectures (mean of 4.52); 2) rewind the video lectures (mean of 4.48), 3) fast-forward the video lectures (mean of 4.04); and 4) watch video lectures in a mobile way (mean of 3.98) such as on the bus. All of this facilitates the student’s ability to reflect and learn at their own pace. The study also showed that there was a perceived increased learning greater amongst the lower achieving students than the higher achieving students. Nouri posited that lower achievers may appreciate more the ability to learn at their own pace through the use of lecture videos, rather than keeping up with a fast-paced classroom lecture.

Reflections

Since COVID lock-downs, we have seen schoolhouse closings and the many benefits of distance learning. Could you imagine integrating the flipped classroom solely at a distance. Students, for example, could complete their pre-class work of watching video lectures and interacting with digital content prior to attending a virtual classroom for group discussions, break-out group work, working through scenarios together, and playing group online quizzes such as Kahoot. Assuming all students and the instructor have the required technology, reliable Internet, and sufficient technological competencies, this could increase distance learning student engagement, improve interpersonal & collaborative skills amongst students, and also make effective use of the instructor and student’s time during the scheduled virtual class.

As an instructor or learning developer, could you imagine giving this model of delivery a try? Although there is work up-front (e.g., to develop video lectures), there could be a big pay-off in terms of student learning performance and satisfaction. As a student, how would you feel about participating in this model? I would be happy to hear your thoughts and experiences related to the flipped classroom.

References

Akçayır, G., & Akçayır, M. (2018). The flipped classroom: A review of its advantages and challenges. Computers & Education, 126, 334–345. https://doi.org/10.1016/j.compedu.2018.07.021

Campillo-Ferrer, J.M., & Miralles-Martínez, P. (2021). Effectiveness of the flipped classroom model on students’ self-reported motivation and learning during the COVID-19 pandemic. Humanities & Social Sciences Communications, 8(1), 1–9. https://doi.org/10.1057/s41599-021-00860-4

Nouri, J. (2016). The Flipped Classroom: For Active, Effective and Increased Learning – Especially for Low Achievers. International Journal of Educational Technology in Higher Education, 13, 1–10.

Roehling, P.,Bredow C. (2021). Flipped learning: What is it, and when is it effective?

Paradoxes of Inclusion: An Article Summary from a CAF Member Point of View

On By kimjones12In Inclusion, Military, ResearchLeave a comment

As part of the culture change efforts in the Canadian Armed Forces (CAF) and the Department of National Defence (DND), I have appreciated the opportunity to act as host during a series of Inclusive Leadership workshops for our DND/CAF executive cadre. The discussions have been rich and I have been moved by the conviction of our leaders to help enact culture change in our department. Our facilitator, Dr Lise Hebabi, discussed a particularly eye-opening article with us, which was written by Dr. Bernardo Ferdman. The article is called “Paradoxes of Inclusion: Understanding and Managing the Tensions of Diversity and Multiculturalism” (2017). As the paradoxes of inclusion spurred such great discussion in the groups and my own reflections, I share here a summary of the article, along with some CAF context examples to consider.

So, what is inclusion? DND/CAF has used the following definition of an inclusive workplace: “a collective culture in which people feel valued, respected, connected, psychologically safe, involved in decision-making, recognized as having unique characteristics that contribute to organizational success, and empowered to bring their authentic selves to the workplace.” (CPCC, 2021).

According to Ferdman (2017), there are three main paradoxes related to inclusion. He likens a paradox to a tension or challenge between two seemingly contradictory components. Ferdman argues that inclusion is inherently paradoxical. The challenge is not moving from one sole perspective to another, but, instead, managing the tensions that will inevitably arise between two paradoxical perspectives as an organization aims to become more inclusive.

Ferdman’s three paradoxes of inclusion, as I will describe, are: 1) Self-Expression & Identity; 2) Boundaries & Norms; and 3) Safety & Comfort. Note that the images below are taken directly from the Ferdman (2017) article.

The Paradox of Self-Expression & Identity

On the left side of this first paradox, we see the perspective of belonging and absorption, and on the right side, we see distinctiveness and uniqueness. Ferdman states that this paradox is often one of the most salient tensions. On one side, we value assimilation. Members must be absorbed into the greater group and may have to give up or hide some aspects of who they are to be similar to the others and to fit in. The perceived benefit of this is that each member then becomes an equal within the group and receives equal benefits.

On the other side of the coin, everyone can present themselves as unique in their own distinctiveness, without losing any of the benefits of being part of the assimilated whole. Within this perspective, members can present their true authentic selves at work and differing opinions and even dissent should be welcomed.

It is easy, in the CAF context, to identify the left side perspective in how we have historically welcomed our new soldiers during basic military training. An individual walks through the [green] door and exits as part of the collective. This is done through, for example, fresh new short hair cuts, identical uniforms, a new ethos, lingo, and culture learned and integrated. I can recall being told by my military husband, at the outset of my military training, to be “gray”, to keep my head down, and to not become the “lightning rod.” The message was clear: fit in or there would be problems. Being distinct, too different, or *gasp* dissenting (as mentioned in the right box) would surely not have been in my best interest.

The CAF is currently making efforts to create a more inclusive environment. Inclusion, according to Shore et al. (2011), requires a sense of belonging and an ability to show one’s own uniqueness. The tension between these two perspectives will need to continually be managed. Are we a melting pot, where all are assimilated, or are we a mosiac, where we can come together as one while displaying our own uniqueness to the benefit of all? In other words, “To fully belong, I need to be able to keep my separate and distinct identity; for that identity to matter and make a difference to the whole [such as in terms of more diverse perspectives and skills], I need to fully belong” (Ferdman, 2017, p. 248). A great quote to reflect on!

The Paradox of Boundaries & Norms

On the left side of this paradox, related to boundaries and norms, we see that these are stable and well-defined, and on the right side, we see that these as shifting and open. On one side, our norms and boundaries are well-defined, persistent, consistent across contexts, and passed on to newcomers. There is certainly a value in stable and well-defined norms and boundaries as they help us define ourselves as an organization. Such examples in the CAF include the common standards of accepting unlimited liability, respecting the dignity of all persons, submitting to the code of service discipline, and wearing a uniform representing our country.

On the other side, we can see our norms and boundaries as shifting and open to change. What is “normal” (e.g. rules & “the way we’ve always done it”) within the organization can and should change as the population and its requirements change. An example of this in the CAF could be religious accommodations for meals, holidays, and prayer time. In the past, it may not have been the norm to accommodate such things during training. Today, boundaries and norms are shifting in order to meet the religious requirements of members.

Within the CAF, a recent change shows an example of how the organization is working to manage inclusion-related tensions. An update to the CAF Dress Instructions came out in September, 2022 after a re-examination of dress-related norms and boundaries. A FAQ page (GC, 2022) stated that “the appearance of the Canadian Armed Forces (CAF) has not kept pace with the Canadian society which it serves.” While the CAF is an organization that values stable & well-defined norms and boundaries, this example shows an understanding that as the population of the CAF and of Canada shifts, it is necessary to periodically re-evaluate our norms and boundaries to make sure that they are still representative. New changes allow for more authentic self-expression while in uniform including examples such as allowing for varied hair length, colours, and styles, more freedom with relation to tattoos and jewelry, and providing the options of both male and female uniform pieces to all members or for these to be intermixed.

A common tension arises from opening up the long established boundaries or, one could say, “the way we’ve always done things.” Some may feel that it creates a slippery slope. What will come next? How far will we go? Often, unfortunately, these kinds of fears lead to no changes, no evolution – thus the tension. To summarize the challenge, in terms of norms and boundaries,: “both overboundedness and underboundedness can be a problem for social systems; the challenge is to get it just right” (Ferdman, 2011, p. 253). An enduring and timeless challenge, no doubt!

The Paradox of Safety & Comfort

On the left side of this paradox, related to an environment of safety & comfort, we have comfort & “my way,” and on the right side, we see discomfort & openness to change. On one side, there is the perspective that everyone should be comfortable and accepted exactly as they are, full authenticity should be encouraged, and nobody should ever need to change.

On the other side of this tension, the perspective says that all must accept some discomfort and have an openness to change. Old comfortable patterns of behaviours may need to adjust in order to create an inclusive and safe environment for all.

I can imagine a CAF scenario where a member moves from a certain historical culture where swearing and off-collar jokes have been the norm and where that had become a comfortable way for colleagues to interact. This, of course, may not be comfortable to all. In some situations, one’s “comfort-zone” may offend those around them, potentially in new environments. As Ferdman (2017) points out, our “familiar behaviour and styles may not have the intended results” (p. 257) in other situations or contexts. As we often talk about “bringing your authentic self to the workplace,” in relation to inclusion and psychological safety, we can now see the tension inherent in this paradox. Bringing your “authentic self” to the workplace could have the potential to make others feel unsafe or uneasy. To build inclusion, all members must adapt themselves, as required, and always remember to “incorporate a good dose of mutual responsibility and sensitivity” (p. 257).

Reflections

According to Ferdman (2017), engaging in inclusion within a diverse population will always involve some level of discomfort. He states that we must learn to be “more comfortable with discomfort” (p. 258) as we open ourselves up to let go of our “certainties” and endeavor to view situations from a new diverse set of vantage points. “When we can recognize, hold, and even welcome the contradictions inherent in inclusion [i.e. the three paradoxes], we should be better equipped to engage and address diversity dynamics” (p. 259).

As an organization, I would suggest, it is not the goal to change perspectives from one side of the paradox to the other. It is to better understand the tensions found in these three paradoxes. We should consider our current state and how we should evolve to better serve and represent our diverse population. We should consider collectively, how far are we willing to evolve and to what end. Do certain contexts, in headquarters versus on operations, affect our responses to these tensions?

I hope I have done justice in reviewing this valuable article on the paradoxes of inclusion. I highly recommend reading this Ferdman (2017) article in full if you are looking to further grow your understanding in this important area.

References

CPCC Initiating Directive on the Integration of the Measurement of Inclusive Behaviours in the Defence Team, September 2021

Ferdman, B. M. (2017). Paradoxes of Inclusion: Understanding and Managing the Tensions of Diversity and Multiculturalism. The Journal of Applied Behavioral Science, 53(2), 235–263.

Government of Canada. (2022). Changes to the Canadian Forces Dress Instructions: Frequently Asked Questions Regarding Changes to the Canadian Forces Dress Instructions.

Shore, L. M., Randel, A. E., Chung, B. G., Dean, M. A., and Ehrhart, & Singh, G. (2011). Inclusion and diversity in work groups: A review and model for future research. Journal of Management, 37(4), 1262–1289.

Reflections on my Doctoral Journey

On By kimjones12In Education, ResearchLeave a comment

Since defending my dissertation and graduating with my Doctor of Education degree in 2020, a lot of people have asked me about the experience. Some people consider whether that would like to follow a similar path but, often, I find that the path seems fuzzy and people don’t really know what to expect. I know this for sure because I also did not know. As John Dewey states in the picture above, the reflection upon any experience is an integral part of learning. After two year and a half years since my graduation, I have finally decided to share my reflections on the experience, along with some lessons learned and random pointers that may potentially help someone else along the path.

First, attaining the degree was much more of a huge experience than I had ever imagined, both in terms of effort and the time required. When I started out, I assumed I would be completed within fours years, which was, frankly, already a huge commitment to make. In the end, it took me almost six years and I was still one of the first few in my cohort to graduate. I was working full-time and mothering a young child throughout those years. I question myself that if I would have really understood the level of commitment required and the emotional toll that it would all take, would I have reconsidered. Hindsight, of course, is always 20/20 and now that I have achieved my very own Mount Everest, I am very proud of the accomplishment.

A Long, Winding Path

This journey, unfortunately, did not go from point A to point B or “as the crow flies.” There were many changes of plans and many redirects. At one point, I received a huge amount of critique from my committee. I remember that it was so overwhelming that I could not look at my work for 6 months. I eventually crawled back out of my cave and started to deal with the feedback. I would like to think that pushing forward over and over and over again, especially when really I did not want to, grew some extra strength in my persistence muscles.

Some Lessons Learned & Pointers to Those Starting Out

Although everyone’s experiences are different and the context of their research projects are different, I share with you some of the lessons I picked up along the way and some random unsolicited advice.

  1. The proposal defence was harder than the final dissertation defence. I had always heard this to be true but, in fact, it was. By the time you get to your defence, you are the expert in the room on the topic and this makes the questions and commentary much easier to react to.
  2. Before I started, I had no idea about the steps required to achieve a doctorate degree e.g., that I would need to defend a proposal, that I would have a committee, etc. I would recommend to others to learn more about the process and what to expect before stepping out.
  3. Unwittingly, I collected too much data and I could not possible use it all. I could have decreased my scope and still had a fulsome research project. (Perhaps I still have a book inside me!)
  4. Mixed methods seemed like a great idea at the start but as the years went by, it started to feel like I was doing double the work of my program mates. Now that it is completed, I do feel that it really added to the richness of the finding. The rich quotes from service members that I was able to add to the discussion really made the narrative come alive. It cost me time, though, so it is worth weighing out the advantages of quantitative, qualitative, or mixed methods, depending on your research goals.
  5. Since my research used military members as participants, there were extra approvals that were required. Any time you interview or survey Canadian Armed Forces (CAF) members or public servants, in this context, a Public Opinion Research request must be submitted. In the Department of National Defence, this is ultimately coordinated through ADM(Public Affairs).
  6. All research projects within the department, using DND employees, CAF members or their families as participants, must go through the Social Science Research Review Board (SSRRB), as detailed in the defence directive: DAOD 5062-1, Conduct of Social Science Research
  7. It’s OK to hire help for some things e.g. transcriptions and advanced statistics. If you do receive help for complex statistics, make sure that you can sufficiently explain them during your defence.
  8. At least in my research context, I had a much higher survey response rates from higher ranks. Make sure to have sufficient numbers of participants from the outset for the range of demographics you are targeting.
  9. The data analysis tools ended up being really fun to learn and use (i.e. NVivo for qualitative data and SPSS for quantitative data). I won’t say I’ve mastered every aspect of them but they were essential to mining the gold out of the large data set I collected. Incidentally, there are many great YouTube videos that helped along the way.
  10. Build a group of academic colleagues. I stayed close to my cohort and we helped and encouraged each other along the way. This ended up being invaluable to my morale.
  11. A good reminder on the difficult days: The best dissertation is the finished dissertation! If it is not everything you ever imagined, chances are you will have more opportunities to shine in the future.
  12. After I graduated, I felt a big, hollow: “Now what?” I had put so much effort and time into the program, that I felt somewhat empty when I graduated. Also, let’s be honest, who is truly going to read a 316-page dissertation?! I wanted to share and talk about it! I did find some gratification from publishing a short version of my research as a Scientific Letter for Director Research & Development Canada (DRDC). I have also been enjoying sharing small slices of my research findings in my blog articles.
  13. Try to take opportunities to enjoy the unique experience. After I had my proposal completed, I entered a 3-minute thesis speech competition. I won first place at the university level and that allowed me to travel to the University of Regina to compete against other winners. Although I did not win the next level, I thoroughly enjoyed the experience and it is now a wonderful memory for me. If you’re curious: here is the video of my 3-minute thesis – a simplified version of my dissertation proposal. Memorizing the speech was the toughest part of it all!

Defending During COVID Lockdown

Although I was studying at a distance, the defence of my dissertation at my kitchen table was even more significant as COVID lock-down had just begun (end-March, 2020). As you can see below, I had all my notes on hand and was more than ready to go. Thankfully the internet connection was strong that day! Unfortunately, I was never able to cross the university auditorium stage in a robe and cap with all the pomp and circumstance to receive my diploma. I attended a virtual event and my parchment was mailed to me. A bit anti-climatic, I know, but we did drink some bubbly the evening of the virtual graduation and I got a fancy frame for my diploma.

Thanks for sharing in my reflections. As it was such a big experience for me, I really enjoy sharing with others who are thinking about potentially starting a similar program or who are in the midst of it and need some encouragement to keep going.

A special thank-you to my Doctor of Education (EdD) cohort members (#7!) at Athabasca University who were so supportive along the way. For those who are still working their way to the finish line, keep going. There is light at the end of the long and winding tunnel.

Reference

Jones, K.A. (2020). Satisfaction of Canadian Armed Forces Regular Fores Members with their Distance Learning Experiences [Dissertation]. Athabasca University.

Distance Learning Course Quality Variables: from a Canadian Armed Forces Member’s Research Perspective

On By kimjones12In Education, Military, Research, TrainingLeave a comment
This Photo by Unknown Author is licensed under CC BY-NC

As I have in previous blog articles, I will share here a small section of my doctoral research on the topic of satisfaction with distance learning (DL) experiences in the CAF. Specifically, I will share some quantitative data on CAF members’ satisfaction with various DL course quality variables. This research, which was defended in 2020, included a sample of 368 CAF members who had graduated from CAF Professional Military Education programs between the dates of January, 2015 and March, 2018.

So, to start – I am sure you can relate with some of the following lines, as we have all been there. You finish an online course by successfully passing a final quiz and happily close your laptop with a sign of relief. Sometimes you think about how the course was such a worthwhile and enjoyable learning experience. Other times, however, after many long and grueling pages of content and confusing quiz questions that you complete in total isolation “as a solo endeavor” (Jones, 2020, p. 149), your computer freezes, your final test score didn’t register, you search endlessly for the help line number to find out how to print a certificate, and you finally end your day by slamming down your laptop cover so hard that it makes a sad and somewhat concerning sound. Then, of course, you go on Twitter to complain about the whole experience with your virtual comrades… who can easily empathize 😉

So, what differentiates good e-learning from bad e-learning? Admittedly, I have described the two extremes of potential outcomes here. During my research on CAF member satisfaction with Distance Learning, I did heard a wide range of feedback that included similar scenarios that I have explained to you. Some feedback was positive and some was negative. As is often the case, the majority of experiences fell somewhere between these two extremes.

I started my research with a literature review to try to identify the different variables of course quality that could make a difference in the student’s experience, and in a larger sense, the success of the course.

During the literature review various questionnaires were explored, including Aman’s (2009) Learner Satisfaction Questionnaire that was later also used by Simpson and Benson (2013), in order to consider what should be included in the instruments for my research.  Many of the variables that I used for course quality were inspired by the quality factor questions used by Aman (2009).

Participants, in my survey, were asked to rate their level of satisfaction, on a scale of 1 to 5 (or N/A) where one (1) was “very dissatisfied” and five (5) was “very satisfied”, with the 13 following items related to course quality:

  • clear learning objectives
  • effective communications with instructor
  • interactions with classmates
  • feeling of being part of a learning community
  • collaborative group work with classmates
  • engaging course content
  • easily accessible required course materials
  • clearly described course assessment
  • constructive feedback from instructors on assignments and assessments
  • timely feedback from instructors on assignments and assessments
  • effective course technology (e.g. in the CAF context – the Defence Learning Network (DLN))
  • course materials provided that helped to reach course objectives
  • course technology that helped to reach course objectives

Findings

Full results can be found in my dissertation, but here are some of the highlights regarding the course quality variables in relation to CAF member satisfaction.

In combining “somewhat satisfied” and “very satisfied” together, the three variables that ranked the highest for DL course quality satisfaction, in descending order, are as follows: 1) clear learning objectives (88.3%); 2) clearly described course assessments (83.1%); and 3) course materials provided that helped to reach course objectives (80.3%). This is positive and great to see!

On the other hand, combining responses of “somewhat dissatisfied” and “very dissatisfied” together, the three factors that ranked the highest for DL course quality dissatisfaction, in descending order, are as follows: 1) feelings of being part of a learning community (23.4%); 2) collaborative group work with classmates (23.0%); and 3) effective course technology (21.3%). With the fact that the CAF Junior Officer Development program has become self study with no peer or instructor interactions, the first two on this list are not surprising.

Interesting, in terms of correlations, all of these variables were found to be positively related to course satisfaction overall.

Correlations of Satisfaction with Course Quality Variables with Overall DL Satisfaction

VariablesnOverall DL Satisfaction
Clear Learning Objectives367.523**
Effective Communications with Instructor317.441**
Interactions with Classmates303.546**
Feelings of Being Part of a Learning Community342.579**
Collaborative Group Work with Classmates293.553**
Engaging Course Content364.529**
Easily Accessible Required Course Materials363.409**
Clearly Described Course Assessments360.392**
Constructive Feedback from Instructors on Assignments and Assessments320.444**
Timely Feedback from Instructors on Assignments and Assessments315.439**
Effective Course Technology (e.g. DLN)364.549**
Course Materials Provided that Helped to Reach Course Objectives364.472**
Course Technology that Helped to Reach Course Objectives365.557**
** p < .01 (two-tailed); N/A answers have been treated as missing values.

After exploratory factor analyses and multiple regression analyses, I found that all three factors that were identified with these variables were significant predictors of overall DL satisfactions.  Specific variables within the three factors that were created were shown to have significant influence on overall DL satisfaction and included variables related to technology; feeling part of a learning community; and effective communications with instructors. (Note: details of these analyses can be found in the full dissertation at reference).

Student satisfaction is important in terms of important organizational outcomes such as readiness to transfer learning, levels of absenteeism, retention, and students’ intention to recommend the training to others (Jones, 2020, p. 11-16). As these variables have been been found to be positively related to student satisfaction and in some cases, even predictive of satisfaction, it is important to make sure that we are working to improve upon these in our online learning development and delivery.

One way to ensure quality in our e-learning would be to implement a course quality rubric that all courses must pass through in order to be launched in the organization. This could potentially be based on an existing and already validated rubric standard (such as is available by the Quality Matters Organization) or built taking into consideration the organization’s specific context and requirements. A standard minimum score could be adopted, and each course would be evaluated by a group of training/ Quality Assurance experts, prior to the DL course being used launched to the training audience.

On a somewhat positive note, 73.6% of participants (n=359) in my research responded that they agreed with the statement that the quality of DL in the CAF has increased over the past ten years. We must always strive to improve, though, and impress upon the other 26.4% that DL in the CAF is keeping up with the standards seen in academia and industry. For my Defence Team colleagues, who mainly include CAF Training Development Officers, CAF and civilian course designers, developers, and facilitators, as well as our e-learning technologists, we must take note of all of these important variables of course quality, continue to improve our skills and keep an eye open to potential technical and methodological advances, and always, always embrace continual improvement.

Reference

Jones, K.A. (2020). Satisfaction of Canadian Armed Forces Regular Forces Members with their Distance Learning Experiences [Dissertation]. Athabasca University.

Inclusion: a review of the literature from a Canadian Armed Forces member point of view

On By kimjones12In Inclusion, Leadership, Military, Research, UncategorizedLeave a comment
This Photo by Unknown Author is licensed under CC BY-SA-NC

I have been reading a lot about inclusion and inclusive leadership lately so I thought I would share some of the definitions, concepts, and models that I have reviewed. As it is a vast area of research, this blog article explores merely a select set of resources that I have explored to date.

So, what is inclusion? The Department of National Defence (DND) and the Canadian Armed Forces (CAF) have used the following definition of an inclusive workplace within the CDS/DM Initiating Directive for Professional Conduct and Culture. It is “a collective culture in which people feel valued, respected, connected, psychologically safe, involved in decision-making, recognized as having unique characteristics that contribute to organizational success, and empowered to bring their authentic selves to the workplace” (CPCC, 2021).

In the newly published CAF Ethos: Trusted to Serve (DND, 2022a), inclusion is identified as one of six military values. It states that inclusion is “essential to creating a sense of belonging and cohesion,” and that it makes, “our military teams stronger” (p. 29). The CAF, “values the knowledge, skills and life experience that each individual brings to the team and needs to maximize this diverse potential through an inclusive culture” (p. 29). It also explicitly states that those who are inclusive will reject and take a “proactive approach to prevent, stop and report” actions related to “racism, sexism, heteronormativity, homophobia, xenophobia or any other form of hateful, discriminatory or hurtful behaviour, conduct or association” (p. 30).

There has been much research on the concepts of inclusion, inclusive leadership, and inclusive climates including the conceptualization of various models, the elaboration of proposed measures of inclusion, and the exploration of variables related to inclusion in various different contexts (e.g. healthcare, military).

Carmeli et al. (2010), for example, measured inclusive leadership based on three facets of leadership: 1) open-mindedness, availability, and 3) accessibility. Various researchers (Shore et al., 2011, Randel et al., 2016, Perry et. al, 2020, Chung et al. 2020) have based their research on a model where belongingness + uniqueness = a sense of inclusion. i.e. I feel like I’m a part of the team + I am able to bring my authentic, unique self to work = feeling included.

“Experiencing inclusion in a group or organization involves being fully part of the whole while retaining a sense of authenticity and uniqueness.” -Ferdman (2010, p. 14)

The US Army did research on inclusion (Brown et al., 2020) and devised a set of measures based on two dimensions: 1) Horizontal Social Inclusion; and 2) Vertical Informational Inclusion. The first dimension included social items such as feeling valued and respected within the team and helping each other out when needed. The second dimension seems to point to the importance of information flow within the military context. It included things such as communications effectively going up and down the Chain of Command and soldiers actively being listened to.

Bernardo Ferdman (2010) is a well-known researcher in the area of inclusion. He, with his research partners, proposed four key elements of inclusion: 1) feeling safe; 2) feeling valued; 3) being involved and engaged in the work group; 4) being embraced for one’s own authentic self. Although more empirical research is still required in these areas, he proposed that a high level of collective environment of inclusion (EOI) + high diversity in the work group + low dispersion of EOI (i.e. that there is a low difference amongst the group in individual’s feeling of inclusion) = the greatest levels of benefits in terms of work performance. In short, if Ferdman and his colleagues are correct: A highly diverse workforce where all employees equally feel a high sense of inclusion will yield the most benefits in terms of increased performance. That sounds like a worthwhile goal to me!

Other research has shown other positive relationships between an inclusion climate and/or inclusive leadership, such as with 1) creativity (innovation) (Carmeli et al., 2010), and 2) workplace/organisational improvements through sharing mistakes (due to the resulting psychological safety) and learning from them together (Nembhand & Edmondson, 2006).

Perry et al. (2020) proposed a conceptual model, based on extensive previous research, that stated that inclusive work climates would positively impact workplace outcomes, at the employee and unit levels, and negatively impact incidences of sexual harassment and other forms of mistreatment. Their conceptual model also proposed that inclusive leaders foster inclusive work environments that experience lower levels of sexual harassment at the individual and work unit levels. Of interest in the military hierarchical context, they state that, “inclusive leadership is a particular type of leadership that is important when status differences exist in the teams” (p. 438). “In order to eliminate sexual harassment, leaders must address both the overt reification of status differentials that harassment seeks to maintain” (p. 438). Leaders can do this through positive role modelling of inclusive behaviours, such as encouraging and valuing everyone’s unique perspectives.

While we work to increase diversity numbers in the CAF, it is important to understand that diversity, in and of itself, is not sufficient to achieve the potential benefits. Inclusion must be a key component in the equation. Diverse opinions, perspectives, skills, talents, and backgrounds are only capitalized upon when members feel safe and welcomed to contribute. Having a seat at the decision-making table is only beneficial if you also have a voice. Research linking diversity to increased workplace performance has been inconclusive on its own because the key mediating effect of inclusion must be taken into consideration.

That is to say, if employees, in all their diversity, 1) feel safe; 2) feel valued; 3) are being involved and engaged in the work group; and are 4) being embraced for their own authentic selves, the benefits of the diverse work force are more likely to materialize (Ferdman, 2010). If a diverse workplace is not led by inclusive leaders and if there is not an inclusive environment, the benefits from diversity will not be realized to the extent that they could be otherwise. Leaders at all levels have to play a role in ensuring our workplaces are, indeed, inclusive.

Members of November Company, 3rd Battalion, The Royal Canadian Regiment (3 RCR) make their way to the live-fire range during the Joint Readiness Training Center (JRTC) exercise in Fort Polk, Louisiana on February 28, 2022, Please credit: Corporal Sarah Morley, Canadian Armed Forces photo ~ Des membres de la Compagnie November du 3e Bataillon du Royal Canadian Regiment (3 RCR) se dirigent vers le champ de tir réel au cours de l’exercice Joint Readiness Training Center (JRTC), à Fort Polk, en Louisiane, le 28 février 2022, Photo : Caporale Sarah Morley, Forces armées canadiennes
Photo credit: Corporal Sarah Morley, Canadian Armed Forces photo

So, how can you be more inclusive in your own workplace context? Luckily, the CAF and DND have already made efforts to describe inclusive behaviours, in line with our already-existing CAF competencies, outlined in the CAF Competency Dictionary, and in line with the DND/ public service core competencies. A working group, co-chaired by leaders within Chief of Military Personnel (CMP), chief, Professional Conduct & Development (CPCC), and Assistant Deputy Minister (Human Resources – Civilian) (ADM(HR-Civ)), and with input from Level 1 reps, developed two aide-mémoires, one for CAF members (CAF, 2022) and one for DND civilian employees (DND, 2022b). These list the behaviours relevant to inclusion, and provide fulsome lists of tangible things that you can do to increase your inclusive behaviours in your own workplace context.

Some of these great examples include:

  • Become an ally; learn about the challenges that your colleagues face, speak up in your own social circles, and amplify the voices of those who may not feel heard.
  • Ask: Which gender pronoun do you prefer? Asking is a sign of care for the person you are talking to and a way to give them the space to feel comfortable with their identity.(CAF, 2022)
  • Capitalize on the diverse talents that each individual brings to the team.
  • Do not make assumptions about someone’s abilities or preferences based on factors such as age, sex, gender, current job. When in doubt, ask.
  • Be a role model, and remembering that little things matter. Supervisors/leaders need to be seen by subordinates as demonstrating integrity in their day-to-day interactions.
  • Show a genuine interest in the personal life of your colleagues; get to know them, and acknowledge key events (e.g., birthdays, family, accomplishments) in their lives.
  • Avoid giving your own opinion first, so that you don’t sway responses.
  • Be open to feedback and ensure that your subordinates and colleagues feel confident to express themselves without fear of reprisals.
  • Whether chairing a meeting or participating in one, don’t dismiss or disrespect other people’s contributions. Make sure to be welcoming and open to what they say, regardless of whether or not you are in agreement. Instead of dismissing them, use phrases like I see your point, That’s a new perspective for me, or I’ve never thought about it that way.

Working towards building a more inclusive workplace is a collective responsibility and the whole Defence Team will benefit. As stated in the Chief of Defence Staff/Deputy Minister Initiating Directive for Professional Conduct and Culture (DND, 2021), “diversity, inclusion, and equity must be fostered if we are to maximize the Defence Team’s operational effectiveness.” (para. 7). Indeed!

What actions will you take today to be more inclusive and be a good role model exhibiting these behaviours within your team?

References

Brown, T.A., Ratwani, K.L.., Key-Roberts, M.J., Simmons, M.J., Toumbeva, T.H., & Nishii, L.H. (2020). Contextualizing inclusion: Developing a framework and measure for a military context. Military Psychology, 32:4, 313-328.

Canadian Armed Forces (CAF). (2022). Aide-mémoire on Inclusive Behaviours and the CAF Competency Dictionary. [accessible only on DND internal DWAN]

Carmeli, A., Reiter-Palmon, R., & Ziv, E. (2010). Inclusive leadership and employee involvement in
creative tasks in the workplace: the mediating role of psychological safety. Creativity Research
Journal, 22, 250–260.

Chief, Professional Conduct and Culture (CPCC). (2021). Initiating Directive on the Integration of the Measurement of Inclusive Behaviours in the Defence Team. [accessible only on DND internal DWAN]

Chung, B.G., Ehrhart, K.H., Shore, L.M., Randel, A.E., Dean, M.A& Kedharnath, U. (2020). Work
group inclusion: Test of a scale and model. Group & Organization Management, 45(1), 75-102.

Ferdman, B. M., Avigdor, A., Braun, D., Konkin, J., and Kuzmycz, D. (2010). “Collective experience of inclusion, diversity, and performance in work groups”. RAM, Revista de Administração Mackenzie, Vol. 11 No. 3, pp. 6–26.

National Defence (DND) (2021) CDS/DM Initiating Directive for Professional Conduct and Culture

National Defence (2022a). The CAF Ethos: Trusted to Serve.

National Defence. (2022b). ADM(HR-Civ) Initiating Directive on Inclusion and Performance. Annex A: Inclusive Behaviours. [accessible only on DND internal DWAN]

Nembhard, I. M., & Edmondson, A. C. (2006). Making it safe: The effects of leader inclusiveness andprofessional status on psychological safety and improvement efforts in health care teams. Journalof Organizational Behavior, 27(7), 941–966.

Perry, E.L., Block, C.J., & Noumair, D.A. (2020). Leading in: inclusive leadership, inclusive climates and sexual harassment. Equality, Diversity and Inclusion: An International Journal, 40(4), 430-447.

Randel, A. E., Galvin, B. M., Shore, L. M., Ehrhart, K. H., Chung, B. G., Dean, M. A., & Kedharnath, U.
(2018). Inclusive leadership: Realizing positive outcomes through belongingness and being
valued for uniqueness. Human Resource Management Review, 28(2), 190–203.

Shore, L. M., Randel, A. E., Chung, B. G., Dean, M. A., and Ehrhart, & Singh, G. (2011). Inclusion and
diversity in work groups: A review and model for future research. Journal of Management, 37(4),
1262–1289.

Work/Life Balance and Chain of Command Support Related to Canadian Armed Forces Members’ Distance Learning Satisfaction

On By kimjones12In Education, Military, Research, Training, UncategorizedLeave a comment
Major Kim Jones, a learner in #SecondLife

As I have in some of my previous blog articles, I will share here a small section of my doctoral research on the topic of satisfaction with distance learning (DL) experiences in the Canadian Armed Forces (CAF). Specifically, I will share some quantitative and qualitative findings on CAF members’ perceptions related to work/life balance and their Chain of Command [employer] support in relation to their DL efforts. This research, which was defended in 2020, surveyed a sample of 368 CAF members, with 12 follow-on interviews. These participants had graduated from CAF Professional Military Education programs, both for Officers and Non-Commissioned Members, between the dates of January, 2015 and March, 2018. While CAF members represent a unique population within a unique employment context, I would venture to guess that some of these findings may be relatable in other fields where employees are either obliged or choose to shoulder the burden of continuing with their professional development while being employed full-time.

One issue related to DL satisfaction that emerged strongly in the research data was work/life balance, including DL’s effect on family and personal time, support from the Chain of Command and, specifically, the amount of time that was provided by the Chain of Command for DL studies. For example, when asked about members’ satisfaction with the support they received from the Chain of Command, 71.7% of respondents answered that they were either somewhat or very satisfied, which is quite positive. It must be noted, however, that another 15.8% reported that they were either somewhat or very dissatisfied (n = 358). In response to the following statement: “CAF members who are DL learners are often required to complete their studies while continuing to be responsible for their normal position workload” (n = 368), responses showed high levels of agreement (92.1% agreed, 72.3% strongly agreed).

Further, some members reported not being permitted to use working hours at all for DL or, in other cases, not personally being able to divorce themselves from their heavy workloads to focus on their DL. In response to a question that asked members to comment on the amount of time they were given during working hours for their DL program/course, the top three responses, based on a coding frequency analysis were: 1) time as available; 2) one day per week; and 3) no time at all. This shows that there was a range of realities for members in terms of time provided, but the concerns of those who received “no time” or not enough time, were very pronounced in the qualitative findings. These members who had to, or in some cases, chose to complete their DL on their personal time, sometimes faced difficulties that included physical or mental health issues and distress, and issues with balancing their family responsibilities. They shared with me, as responses to open-ended survey questions and interviews, their various challenges in juggling their workload, their DL studies, and their personal and family life.

This issue was illustrated by a code frequency analysis in response to a question asking members to identify their greatest dissatisfiers with DL. The 3rd most frequent response was balancing their job with DL, and the 5th most frequent response was work/life balance, including family issues. (Of additional interest, other top dissatisfiers identified included: lack of meaningful interactions, technological issues, and issues with the quality of the course design). Further, 36.9% of respondents (n = 363) either agreed or strongly agreed with the statement, “DL increases the chance of burn-out for CAF members.” This may indicate that some members perceive that DL can cause work/life balance issues, potentially through the difficulties that arise from juggling their work, professional development, and other life and family responsibilities. These findings were corroborated by the qualitative data, in that the phrase “burnout” and related discussions arose numerous times.

Correlation analyses between support from the Chain of Command, family, and coworkers with overall DL satisfaction indicated that support from the Chain of Command was significantly correlated with overall DL satisfaction (rs (358) = .294, p < .01). Multiple regression analysis of the support factor, which included support from the Chain of Command, family, and co-workers combined, was shown to have a significant association with overall DL satisfaction. When these three variables were separated out (i.e. support from Chain of Command, family, and coworkers), support from the Chain of Command was found to be the most significant support predictor of overall DL satisfaction.

Presently, some members make agreements with their Chain of Command prior to starting their courses regarding the time they will use during working hours to complete their DL. This could be a helpful strategy, given that 68.5% of respondents (n = 368) agreed or strongly agreed with the statement: “Establishing Learning Contracts to be signed by CAF members and their supervisors assigning permitted hours per week for the DL course should be a requirement for all learners of DL courses.”

The qualitative data supported the quantitative findings in that some members shared their stories of working long hours between their heavy workplace commitments and DL course loads. Others shared their stories of trying to juggle their work and DL commitments and how this caused strain on their family situations. Still others suggested that perhaps time away from work duties should be a mandatory requirement to allow members to have a more focused and valuable learning experience. Further, some members suggested that if the CAF were to ensure further availability of quiet work-spaces or computer labs on all bases, away from the regular workplace, it could be beneficial and allow members to better concentrate on DL courses with fewer interruptions.

Recommendations

Mandating an amount of time to CAF students, outside of the normal workplace and in line with the time required for effective learning to take place during DL, could be considered for all mandatory training and education. Ensuring that the Chain of Command is made aware that a certain amount of time is required, that regular tasks may need to be delayed or be reassigned, and that it is their responsibility to encourage members to take the time required and prioritize their learning appropriately could increase student satisfaction and positive learning outcomes within DL experiences.

Reflections

Balancing a full workload with various training and educational pursuits can be challenging, both in the CAF and, I suspect, in any workplace. Frank, open discussions between employee and employers and re-prioritization of time and tasks can sometimes help alleviate issues related to a heavy workload. As one research participant stated, “You can’t burn the candle at both ends.” Indeed! You may try, for a time (as I have!), but it tends not to be a sustainable way of living in the long run.

If you would like to see further details on my research, such as research methodology and full findings, please see the link below.

Once again, thank you to the survey and interview participants who took part in this research.

Reference

Jones, K.A. (2020). Satisfaction of Canadian Armed Forces Regular Fores Members with their Distance Learning Experiences [Dissertation]. Athabasca University.

Leading Truthfully: A Reflection on “Lying to Ourselves: Dishonesty in the [US] Army Profession”

On By kimjones12In Education, Leadership, Military, Research, Training1 Comment
Canadian and American military members exchange their flags at 5 Wing Goose Bay, Newfoundland and Labrador during Exercise VIGILANT SHIELD 17 on October 17, 2016.
Photo: MCpl Krista Blizzard, 5 Wing Public Affairs GB2016-10-215

Some years back, I read a report out of the Strategic Studies Institute and U.S. Army War College Press that really impacted me and has lingered in my mind ever since. Lying to Ourselves: Dishonesty in the Army Profession, published in 2015 and written by Dr Leonard Wong and Dr Stephen Gerras, is a frank account of how members can become ethically numb, and, therefore, react dishonestly in the face of ever-growing and cumulative loads placed on the forces. These overwhelming workloads can include things like mandatory training, reporting requirements, data requests, compliance checks, personnel evaluation reporting, and the list can go on and on.

Although this report focused on the United States Army and gathered qualitative data from its members, I suggest that my Canadian colleagues reflect on the points presented in this blog article, and then perhaps read the full report, to see if any of these points are relatable in the Canadian Armed Forces (CAF) context. If so, what should/could be done about a culture of dishonesty?

Truth as an Important Aspect of our CAF Ethos

Military professionals will most often consider themselves to be truthful and honest. Our military ethos calls for it. In the summary of Duty with Honour: The Profession of Arms in Canada (2003), it states that integrity, “calls for honesty, truthfulness, uprightness, the avoidance of deception…” (p. 17). In the new Canadian Armed Forces Ethos: Trusted to Serve (2022), it points out that “a person with integrity is truthful, strong of character and reliable,” (p. 25) and that integrity requires, “pursuing truth regardless of personal consequences” (p. 24). Our professed value in the truth is even shown in the motto of the Royal Military College: “Truth– Duty- Valour.”

The reality that was shown through this qualitative research with the US Army, which can be hard to hear or accept, is that impossible deadlines, the deluge of reporting requirements, the inundation of directives from above, and so much mandatory training is often difficult, if not impossible, to fit into the schedule. These examples can sometimes lead to, what members may tell ourselves, are white lies for the greater good. Add in a culture of “Yes, Sir!”, “Yes, Ma’am!” and “no-fail” and it can be the perfect set-up for dishonesty to become the norm and, therefore, for members to constantly have to choose between lying or standing out from the crowd to be truthful. This, in the military culture, can lead to scorn from their colleagues and supervisors and potentially hurt their advancement (e.g. being the only one who is unable to report 100% compliance in X,Y,or Z).

Examples

Some applicable key words found in the report, that may or may not ring a bell, include: “hand-waving”, “fudging the numbers”, “massaging the truth”, “checking the box’, “pencil-whipping it,” “bending the truth, “giving them [leaders] what they want”. One member stated that “You gotta make priorities, we met the intent, or we got creative” (Wong & Gerras, 2015, p. 8). In “feeding the beast” with inaccurate statistics, in reporting 100% compliance when 85% would be more accurate, when signing that a personnel briefing took place when it did not, when reporting that unit members have completed mandatory training when, actually, time did not allow, when filling in colorful PowerPoint slides with questionable numbers, many lead to members experiencing “ethical fading.” Ethical fading occurs when the “moral colors of an ethical decision fade into bleached hues that are void of moral implications. Ethical fading allows us to convince ourselves that considerations of right or wrong are not applicable to decisions that in any other circumstances would be ethical dilemmas” (Wong & Gerras, 2015, p. 17). This can lead to ethical numbness and then, at that point, we must question whether dishonesty will grow beyond trivial small “numbers fudging” to more monumentally dishonest acts.

The report lists various examples related to training, compliance, finances, and actions taken during operations related to reporting. Being a Training Development Officer (TDO), I was reminded of the dishonesty of acquiring a course certificate from merely flipping through e-learning pages and not applying oneself to actually learning the content. The report shared a situation where one of the “smart” members sat down at a computer and quickly completed the course and printed the certificate for all nine section members. Another example included a Sergeant printing off course completion certificates for the whole team, knowing full well that the training had not been given.

As I am presently working at the Chief, Professional Conduct & Culture, I am especially disturbed by the following example:

“One captain spoke of trying to complete mandatory Sexual Assault Prevention and Response Program (SHARP) training:

We needed to get SHARP training done and reported to higher headquarters, so we called the platoons and
told them to gather the boys around the radio and we said, ‘Don’t touch girls.’ That was our quarterly
SHARP training.” (Wong & Gerras, 2015, p. 13)

Again, this example took place in the US Army, but it is worth reflecting on. In a situation where mandatory training requirements are so heavy that there is physically not enough time in the day to complete them, along with the avalanche of other administrative responsibilities that are continually passed down from the highest levels, is this example a common result? “It [the US Army] is excessively permissive in allowing the creation of new requirements, but it is also amazingly reluctant to discard old demands” (Wong & Gerras, 2015, p. 18). This may or not be relatable to the CAF context but, in my opinion, it is worth some consideration.

What Can Be Done?

So, how do the authors of this report suggest moving beyond dishonesty in the profession?

1) Acknowledge the Problem– We should discuss these things openly and honestly without fear of reprisal. Leaders should lead the discussion. They should admit that they know these things happen at all levels from their own experiences.

2) Exercise Restraint– Restraint must be given towards the number of no-fail tasks and #1 priorities. Workload must actually be accomplish-able. Mandatory training and new directives can come from all directions at all different levels and leaders must “shoulder the burden of prioritizing” (Wong & Gerras, 2015, p. 30). Leaders must also consider what is actually required and valuable in terms of reporting and, then, prioritize appropriately. If everything is vital, then nothing is. Perhaps 100% compliance, for example, is not realistic in a given context and 85% compliance could actually be an acceptable risk. If a legacy requirement is no longer important, consider getting rid of it. Also, if the requirement is important, ensure that the member providing information, completing the training, or checking whatever box, understands why it is important. If the importance is understood, it should decrease dishonest reporting or the fudging of numbers.

3) Lead Truthfully – Leading truthfully could include “speaking truth to power” while insisting that training module X,Y, or Z is not worth being mandatory training for the whole organization. Leading truthfully informs subordinates that accurate reporting is more important that achieving 100%.

Reflections

In terms of reflections, do we, in the CAF, condone dishonesty or perhaps even expect dishonesty in some circumstances/situations? Are we generally overwhelmed with the deluge of requirements that seem impossible to meet? Do our members and leaders sometimes face the feeling of dissonance that comes with needing to “feed the beast” bogus &/or inaccurate information in a time crunch, all the while feeling the need to maintain a self-identity of “a person with integrity [who] is truthful, strong of character and reliable,” (DND, 2022, p. 25) as required by our CAF Ethos. Does this report describe merely a problem within the US Army, or are there aspects of this report that ring true and relatable for you within the CAF context?

I hope I have given you some good food for thought and that I have done justice in summarizing this excellent report. If you have found this topic interesting, I highly recommend that you take the time to read and reflect on the full report: Lying to Ourselves: Dishonesty in the Army Profession

Reference

Leonard Wong Dr. and Stephen J. Gerras Dr., Lying to Ourselves: Dishonesty in the Army Profession ( US Army War College Press, 2015), https://press.armywarcollege.edu/monographs/466

National Defence. (2003). Summary of Duty with Honour: the Profession of Arms in Canada.

National Defence. (2022). Canadian Armed Forces Ethos: Trusted to Serve.

Technology: a Course Quality Consideration for Canadian Armed Forces Members’ Distance Learning Satisfaction

On By kimjones12In Education, Military, Research, Training, UncategorizedLeave a comment
This Photo by Unknown Author is licensed under CC BY-SA-NC

Have you ever tried to complete a Distance Learning (DL) course when the technology became your focus, either because of the slowness or crashing of the system, the non user-friendly design of the interface, or the timing-out of a quiz where you lost all of your work? Me too! On the other hand, have you ever completed a learning experience at a distance where the technology was seamless and really seemed to enhance the learning experience? Me too! Let’s have a look at what some Canadian Armed Forces (CAF) members had to say about their satisfaction (or lack thereof) with technology as an aspect of course quality.

As I have in previous blog articles, I will share here a small portion of my doctoral research on the topic of satisfaction with DL experiences in the Canadian Armed Forces (CAF). Specifically, I will share some of my quantitative and qualitative findings related to technology, as a course quality consideration related to CAF member DL satisfaction. This research, which was defended in 2020, included a sample of 368 CAF members who had graduated from CAF Professional Military Education programs between the dates of January, 2015 and March, 2018. The data was gathered from surveys, as well as 12 follow-on interviews.

Technology was a subject that arose frequently in both the quantitative and qualitative research data, both as responses to direct questions as well as spontaneous comments in relation to satisfaction and ways to improve DL. For example, when asked about satisfaction with “effective course technology (e.g. DLN),” 66.4% of respondents (n=366) said that they were somewhat or very satisfied, while 21.3% of respondents said that they were somewhat or very dissatisfied. In response to the perception statement, “The CAF has good technical support systems in place to help should any technical problems arise during DL courses,” 41.3% of respondents (n=363) either agreed or strongly agreed with this statement, while 27.0% of respondents either disagreed or strongly disagreed with this statement. The responses to these two questions indicate that a portion of CAF members have negative perceptions regarding DL technologies and technical support in CAF DL. This supports earlier findings from the DND “Your Say” survey research (Budgell, Butler, & Eren, 2013), that found that only 45% of respondents, from a sample of 1730 CAF members, agreed that the CAF makes good use of technology in courses. This, of course, begs the question of what the other 55% of CAF members think we could be doing better in relation to technology use within DL courses.

In the correlation analyses for course quality variables, it is noteworthy that all course quality variables that were measured (e.g. timely instructor feedback, clear learning objectives, easily accessible course materials, etc.) had a positive significant correlations with overall satisfaction. Two of the strongest significant positive correlations with overall DL satisfaction involved the satisfaction with course technology, specifically: 1) course technology that helped to reach course objectives (rs (365) = .557, p < .01); and 2) effective course technology (e.g. DLN) (rs (364) = .557, p < .01). Both of these would be considered of moderate strength. With the multiple regression analyses that were completed, both “Effective course technology” and “Course technology that helped to reach objectives” were shown to be significant predictors of DL overall satisfaction.

Government of Canada. (2018). Canadian Armed Forces professional development framework

The qualitative data supported the quantitative findings that the topic of technology was relevant to CAF DL satisfaction. Technolgy was so prevalent in the qualitative data that it emerged as a theme unto itself. The technology theme included the following four categories: 1) accessibility; 2) usability of technologies supporting DL; 3) learning management systems (LMS); and 4) perceptions regarding DL technology in the CAF. Although some members indicated that the technology to support CAF DL had improved over the years, fewer positive sentiments and experiences concerning CAF DL technologies were shared.

Within the category of accessibility, there were many comments regarding difficulties experienced with DL technology including issues of connectivity and bandwidth. Connectivity was brought up for both office and home settings, but also in operational settings such as on ships and on overseas deployments. Members did mention that they liked the fact that they could access their DL from their homes, outside of their workplace computer. One member said, “the system is very user-friendly because it exists outside of the DWAN [Defence Wide Area Network] system, very easy to use, home computer, home-based internet.” One liked that the technology exists so that they can do DL, “anywhere, anytime.” Another member, however, stated, “I do not have access to reliable internet from my home and must conduct the course at work.” Indeed, Internet bandwidth and reliability in the more rural areas can still be an issue. Other members stated that they had issues with connectivity while trying to do their DL in the office. One member said that “the servers themselves need desperately to be updated. The system struggles greatly with large courses.” Another member commented that the “intranet at work is dead slow.” These issues, one member stated, “often result in complete loss of connection” and that sometimes the system, “does not save the work that was already completed.”

Within the category of usability, items such as the following were brought up: members’ comfort level with the DL technology, DL technology support available (including from a help desk), firewall issues, members’ requirement to use external technology to support their course, and issues encountered such as with the DND search engine, inactivity time-outs, the inability to print courseware, and a vast array of “technical hiccups.” In relation to comfort levels, one member stated that “although I am older, my computer skills and comfort level with software systems are good. I never had any issues with that part of the DL.” On the other hand, one stated that, “regardless of age, not everyone is computer savvy (i.e. a digital native). There needs to be training available that is aimed at every type of learner.” In terms of ease of searching for references on the DWAN Intranet, one member stated that it “was of no use when trying to find reference material.” Another suggested that the DND/CAF should, “invest in upgrading the DND/DWAN to have better browsers and access for research.”

Regarding the Learning Management Systems (LMS), participants discussed the Defence Learning Network 2.0 and/or Moodle, which is being used by the Royal Military Colleges of Canada (RMCC) and the Canadian Forces College (CFC), dependent on the program they had completed. There was a range of satisfaction with these tools. One Junior Officer who was interviewed stated that the DLN is “easy to use,” “very user-friendly,” and that “anybody could do it.” Other members felt that the DLN, however, left some things to be desired. One Senior Non-Commissioned Member (NCM) who I interviewed about their Senior Leadership Program (SLP) experience, for example, stated that the LMS affected the interactions between students in their forum discussions. They stated that “because of the software of DLN… it wasn’t a flow,” adding that “it wasn’t very intuitive or well laid out… it was very cumbersome.”

Another member stated that the DLN, “is a very difficult program to work with… navigating the DLN is terrible and something needs to be changed…. It… creates needless frustration.” Another stated that it creates “stress for no reason trying to navigate it.” Another member commented that the “DLN is very clunky and difficult to find courses and is not very user friendly.” A Senior Officer, who had completed the Junior Command and Staff Programme (JCSP)- DL version, said of the LMS used by CFC, “it was okay, I guess. I’ve seen better, but I’ve seen worse.” Another member said, “I would complete more DL courses if the system was easier to work with.” One Senior NCM suggested, “improving the platform to enable students who are working on that theory portion of the DL so they can actually collaboratively work together if that’s what’s required. So be it from smartphones, from tablets, from work, traveling on the train, traveling in a car, whatever.”

Perceptions related to DL in the CAF, as the fourth category, were quite varied and included members’ general levels of satisfaction and expressions of frustration. Related to satisfaction, one member said the “technology was decent,” and another stated that “as a whole I think that it is getting better.” Some expressions of frustration with the technology, however, were also shared. One Intermediate Leadership Program (ILP) graduate stated, for example, that “the technical hiccups were very distracting and at times infuriating.”

The Maple Leaf, The Defence Learning Network 3.0 is here!, 8 March 2022

As illustrated in the comments from these CAF members, the various technology components related to DL, as a course quality consideration, can have a positive or negative effect on the DL satisfaction and learning experience of students. To optimize the learning experience of CAF members, we must always strive to improve upon what we have and trial courses on various platforms and browsers prior to launch. Issues found, as well as student feedback, must be addressed. It should be noted that the technology of DL is always evolving and we must strive to keep pace. The DND/CAF is currently in the process of upgrading to a newer version of our Saba Learning Management System in the cloud, known as the Defence Learning Network (DLN) 3.0. I look forward to learning more about the benefits of this new DLN iteration, including its new functionalities. This is a great step forward, as is the availability of the new DLN 3.0 virtual classroom and MS Teams for synchronous group discussions. I look forward to see what future technologies in this space will provide in order to further improve upon the student learning experience.

I would like to thank the CAF members who took part in the surveys and/or interviews in support of this research.

Reference

Budgell, G., Butler, A., & Eren, E. (2013). Task # 138: Regular Force Your-Say Survey: Spring 2012 Focus Selection Results. DRDC-RDDC-2015-C102. 

Government of Canada. (2018). Canadian Armed Forces professional development framework

Government of Canada. (2022). The Maple Leaf, The Defence Learning Network 3.0 is here!

Jones, K.A. (2021). Satisfaction of Canadian Armed Forces Regular Fores Members with their Distance Learning Experiences [Dissertation]. Athabasca University.

Distance Learning vs Traditional Classroom-Based Learning

On By kimjones12In Education, Military, ResearchLeave a comment
Myself, Major Kim Jones, at my previous rank in 2012. MCpl Kathryn Poudrier, Defence Public Affairs Learning Centre, Copyright 2012 DND-MDN Canada

Throughout my time as a Training Development Officer in the Canadian Armed Forces (CAF), as well as during my studies in the domain of education, I have been asked many times which is better, more effective, and/or more satisfying: distance learning (DL) or traditional classroom-based learning? What does the research say?

The answer is “It depends!” I have experienced excellent examples of DL, as well as poor examples, and I have experienced excellent examples of classroom learning, as well as poor examples. Chances are that you have too! Variables such as the quality of the course design, the effectiveness of the technology and the learning environment, as well as the incorporation of (or lack thereof) quality interactions with instructors and peers all affect the quality of the course, whether it be delivered via DL, face-to-face in the classroom, or in some blended format. So, let’s have a look at what the research has to say.

There have been many studies over the years comparing DL with traditional classroom-based courses, both for effectiveness and learner satisfaction.  A well-known historical theoretical debate took place during the 1980s and 1990s between Richard Clark and Robert Kozma concerning training delivery systems.  On one hand, Clark (1983, 1994) stated that the medium was a neutral carrier of the course content and method, and that the important aspect that could affect the outcome of learning, positively or negatively, was the instructional method.  Kozma (1994), on the other hand, argued that as DL technologies evolved, Clark’s (1983) earlier assertion was just not enough as, “a medium’s capabilities enable methods” (Kozma, 1994, p. 20).  Newer interactive technologies, for example, that enable such things as collaborative learning, can, indeed, influence learner outcomes.

Generally speaking, though, research has found that there is no significant difference in learner outcomes or satisfaction between DL and classroom-based instruction. Indeed, an entire website is dedicated to supporting this conclusion (nosignificantdifference.org). Numerous meta-analyses over nearly two decades have led to this conclusion. Russell (1999) reported, using a meta-analysis of 355 studies, that there was no significant difference in learning outcomes (i.e., effectiveness) based on the mode of education delivery (traditional classroom versus DL) alone.  Several meta-analyses have also looked at learner satisfaction comparing distance education and the traditional classroom. For example, Allen, Bourhis, Burrell, and Mabry (2002) conducted a meta-analysis of 22 studies with a total sample of 3822 learners (after removing three outlier studies).  These 22 studies compared learner satisfaction for distance education to the traditional classroom in higher education.  The meta-analysis concluded that learners found distance education as satisfying as traditional classroom study.

Within my own doctoral research (2020), I delved into CAF members’ DL satisfaction with a sample of 368 CAF members who had graduated from CAF Professional Military Education programs between the dates of January, 2015 and March, 2018. While I didn’t compare DL to classroom learning, it is interesting to note that 78% of survey participants reported that they were somewhat or very satisfied with their DL experiences and 16% reported that they were somewhat or very dissatisfied with their DL experiences.

Although this sounds rather positive for CAF DL, it is also very interesting to consider that when asked, with all thing being equal (e.g. time required), which mode of delivery members prefer, classroom or DL, 71.7% said that they would choose classroom learning and 28.3% said that they would choose DL. There are multiple reasons why this could be and some could be CAF-specific, such as high operational tempo, technology issues, etc. I did find, however, that course quality and design variables, such as instructional methods, course materials, technology effectiveness, and interactions with peers and instructors were all predictive of DL satisfaction. Additionally, as some members reported having difficulty juggling their work and home life with their DL, support from the Chain of Command (often in terms of hours provided to devote to DL) was also a significant predictor of DL satisfaction.

As DL tools continue to evolve, we now have new ways to interact with learners at a distance. While we continue to improve upon our DL offerings, one thing is shown over and over in the research. There is no significant difference in the effectiveness of learning, depending solely on whether the learning is in the classroom or via DL. The differences are most often found in the quality of the design, development, and delivery of training & education. Effective technology, fulsome and valued interactions by a skilled facilitator (either synchronously in a virtual classroom or asynchronously in a discussion forum), and high quality design should equate to effective learning with satisfied learners regardless of delivery method.

I propose that we put this decades-old debate to sleep and focus our energy on making all of our training & educational offerings the best learning experiences possible, regardless if the delivery is through distance learning, within a traditional classroom, or some mixture of both.

References

Allen, M., Bourhis, J., Burrell, N., & Mabry, E. (2002). Comparing learner satisfaction with distance education to traditional classrooms in higher education: a meta-analysis. American Journal of Distance Education, 16(2), 83-97.

Clark, R.E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53(4), 445-459.

Clark, R.E. (1994). Media will never influence learning. Educational Technology Research and Development, (2), 21-29.

Jones, K.A. (2021). Satisfaction of Canadian Armed Forces Regular Fores Members with their Distance Learning Experiences [Dissertation]. Athabasca University.

Kozma, R. B. (1994). Will media influence learning? Reframing the debate. Educational Technology Research and Development, 42(2), 7–19.

Russell, T. (1999). The no significant difference phenomenon: as reported in 355 research reports, summaries and papers: A comparative research annotated bibliography on technology for distance education. Raleigh, NC: North Carolina State University.

Perceptions about Distance Learning from CAF Members’ Points of View

On By kimjones12In Education, Military, Research, TrainingLeave a comment

As I have in previous blog articles, I will share here a small section of my doctoral research on the topic of satisfaction with distance learning (DL) experiences in the CAF. Specifically, I will share some qualitative data on CAF members’ general perceptions, both positive and negative, related to DL, as well as perceptions related to the direction the CAF is going with DL use, CAF cultural aspects related to DL, and considerations related to generational differences and individual learning preferences. This research, which was defended in 2020, included a sample of 368 CAF members who had graduated from CAF Professional Military Education programs between the dates of January, 2015 and March, 2018.

There was a wide variety of responses regarding overall satisfaction with DL. One Senior Non-Commissioned Member (NCM), for example, spoke positively about his experience and, in rating his overall experience from 1 to 10, stated, “I would say about 9.” An Officer Cadet, who had completed the Primary Leadership Qualification (PLQ) as a Junior NCM, rated his experiences as “6, 7 out of 10.” A Senior Officer shared that “on a scale of 1 to 10, 5.” General comments regarding CAF DL satisfaction, ranging widely from positive comments to more negative sentiments included, “best learning experience,” “various degrees of positive,” “globally it’s great,” “pretty good,” “pleased,” “quite favourable,” “satisfied,” “fine,” “mixed emotions,” “strongly dislike,” and “painful to get through.” One member even stated, “I cannot convey the depth of my dissatisfaction.” As we can see, generally speaking, the range of perceptions was vast.

Many members shared their perceptions concerning the direction the CAF was taking with the use of DL. Generally, I found that those members who were positive about their DL experiences felt that the CAF was moving in a good direction in increasing the use of DL and improving upon current offerings. Some of the comments included that the CAF was making “incredible strides,” and that “it’s a great way to go.” One member said that he was “impressed that we are going this way,” and another stated that “the CAF should continue to move in the way we are.” Additional positive comments included that “DL is a vital tool in contemporary learning and the CAF should continue to embrace it,” and that “DL is a great capability that should be explored and leveraged as much as possible.”

On the other hand, members who were more negative about their DL experiences shared that they believed the CAF was relying too heavily on the use of DL and should minimize its use. Often members made the comment that we would need to ensure that we are choosing the right balance between the use of DL and classroom. One Senior NCM stated that “we’re starting to do too much by DL, simply because there’s a cost savings factor.” An Advanced Leadership Programme (ALP) graduate suggested that the CAF should, “arrêter d’augmenter le nombre de cours donner en AD [stop increasing the number of courses given by DL].” Another member stated, “I’m just concerned that we put too many eggs in the same basket there for the DL,” and yet another stated that “the CAF needs to stop ‘pushing the easy button’ on DL courses in general”. Again, these perceptions ranged from very positive to very negative on the use of DL in the CAF.

Some comments seemed to be CAF- or military-specific cultural perceptions regarding DL. Although these quotes represent individual beliefs and could potentially be just one person’s view, they seemed valuable to consider. One Canadian Armed Forces Junior Officer Development (CAFJOD) graduate stated that operations and workload must take priority and that professional development courses are, “known as selfish career climbing initiatives.” A Senior Appointment Programme (SAP) graduate stated that “if a course is taught by a person or in class it is assumed that the lesson is important. When course materials are covered by DL, the mentality is that ‘it’s the less important stuff.’” One Primary Leadership Qualification (PLQ) graduate shared his opinion that, “In my mind, there’s no room for DL in the Army. It’s been my experience that most folks didn’t join the army because they were academics. DL courses are designed for academic oriented people, not for the typical blue collar.”

Regarding members using work time for DL, some members shared their perceptions that it would be seen negatively if they were to take this time. One member stated that they did all the DL on their own time due to, “not wanting to be judged for taking time off.” A CAFJOD graduate said that supervisors say that “professional development through courses, especially DL, are a “personal responsibility” and should “not take work time” to complete them.” These are all noteworthy individual perspectives/opinions to contemplate.

Some members shared a perception that DL was less favourable for older members and more favourable for younger members. Some members shared their opinion that there are no real differences between the generations in their affinity and satisfaction with the use of DL. One member, for example, stated that “as an older member of the CAF the greatest beginning difficulty was the technology of the computers and navigation of a DL course.” Another stated that “when it comes to the younger generation, they are probably more comfortable doing stuff online…. we older [members] have to get used to it.” Another member stated that “some of the senior NCMs… may not be as comfortable with computers…. so maybe they wouldn’t be as positive… using such a tool.” Another shared, “maybe I’m becoming one of those old guys I don’t know, but I’m reluctant, or hesitant to invest myself too much into DL.” On the other hand, one member stated that “particularly for the younger generation that’s starting to come through now, they’re so used to technology and so used to the resources and being able to find things online and that kind of thing.” Another member stated that “maybe the new generation responds better to DLN as they are less likely to want to leave “home” for a course and are more reliant on networking with “friends” they’ve never met.”

Not everyone, however, saw generational considerations having a real effect on DL satisfaction. One Senior Officer shared his opinion that we tend to think of DL as, “generational, like all the young folks like it, the older people don’t. I think that might be a bit of a misnomer or a fallacy because it just depends.” A Senior NCM stated, “I certainly think that across-the-board of generations -so whether you’re 19 years old or whether you’re 55 years old… DL is a very good mechanism.” As one 52-year-old member with 35 years of service in the CAF stated, “even us old guys can do it!” One 53-year-old member with 28 years of service made the point that even the older CAF members have been in a “technology powered workplace” for a long time now. “We may not be Digital Natives,” he stated, “but we should be just about out of web-illiterates [in the CAF].” Personally, I would tend to agree with this point. As someone who advances beyond middle-age, even I had a Commodore VIC-20 growing up and I would not necessarily consider myself a “digital immigrant”.

One CAFJOD graduate shared that “everyone learns differently, DL may work for some but it does not work for me.” One member suggested that the CAF should, “have some options for people. Some like DL… many, like myself, hate it. Basically stop looking for “one” solution because it will never work for everyone.” Another member stated that “not everyone is computer savvy (i.e. a digital native). There needs to be training available that is aimed at every type of learner.” Another stated that “we’re getting better with identifying people with… how they learn, and just try[ing] to adapt to it whenever we can…. otherwise we’ll always be leaving someone behind.” A CAFJOD graduate stated that “many people learn in different ways, some prefer classroom instruction and some prefer DL and some prefer hands-on courses. We should be helping people learn according to their strengths and not forcing everyone to supposedly “learn” in exactly the same way.” Some good food for thought regarding members’ perception related to learning preferences and the value of providing options.

For further details related to this research, the methodology used and fulsome findings, please feel free to refer to the link below. There have been some exciting advancements in CAF DL in the years since this research took place including the introduction of the new DLN 3.0 and more widely used videoconferencing/virtual classrooms for synchronous DL, which may influence CAF members’ perceptions of DL today.

Thanks so much to the CAF members who offered their time to answer the open-ended survey questions and who participated in interviews for this research. Their voices have added so much to the quantitative numerical data collected.

Reference:

Jones, K.A. (2020). Satisfaction of Canadian Armed Forces Regular Fores Members with their Distance Learning Experiences [Dissertation]. Athabasca University.