eLearning Evaluation for Continuous Improvement:
A great deal of time and budget will be spent when developing your eLearning course. So, how do you ensure it is effective, engaging, relevant, and has a long shelf life? By evaluating an eLearning course, you can assess the effectiveness, engagement, relevance of the course and continue to improve upon it to provide your learners a better and long-lasting eLearning experience. In this article, I will share why, when, who, and how to use eLearning course evaluations to create a long-lasting and successful eLearning product.
Why use an eLearning Course Evaluation
As an eLearning, training, or HR professional, you need to know if your eLearning course achieves its goals and meets its objectives to be considered successful. The best way to measure an eLearning course's success is through evaluation. This is essential as it enables you to assess a course’s quality, effectiveness, engagement, relevance, and identify and understand what did or didn’t work to keep, remove, or improve content. To understand how evaluations work in regards to eLearning continuous we simplify it to a three-step process.
Evaluation Three-step Process: Data Collection + Data Analytics + Course Improvement = eLearning Course Continuous Improvement
When to use an eLearning Course Evaluation
For a long-lasting eLearning based on continuous improvement evaluations are a requirement and should be a top priority. So, when do you start evaluating your course, before it launches or after it is completed? The best answer is throughout. You need feedback to ensure the course is effective, correct, engaging, and able to meet the current needs. Whether you are still developing the course or the course has launched and learners are evaluating it the evaluation process should be an ongoing process and ritual process; you need to constantly evaluate your eLearning course regularly to improve it and make it current. The basis for evaluation data collection and analysis should be determined by your staff’s availability, tools, and flexibility of the content matter. In an ideal state, all eLearning content should at least be reviewed annually to ensure effectiveness, engagement, and content relevance.
Who should evaluate the content?
The participants for the evaluation will depend on the phase in which it is. We breakdown eLearning evaluation into 3 phases: Phase 1 Development, Phase 2 During, Phase 3 After. Each phases’ evaluation data will guarantee you end with a more effective, engaging, and relevant eLearning course. Let’s look at each phase and the different participant groups.
1. eLearning Course Development Phase
During the when to use an eLearning Course Evaluation we address the need to use evaluations through the how to process including the development phase. This is the phase in which the content is being discussed, designed, and built. An evaluation during this phase requires feedback from everyone who has a role in the creation from designers, subject matter experts, and even some participants in your eLearning course.
o Ask your eLearning design team
To access how well your eLearning course meets its objectives and goals you need to have regular meetings with the eLearning design team. These meetings should cover course objectives, goals, content, technical and user limitations (these are course limitations due to your users such as internet speeds, language barriers, and access to computers), budget, timeline, and dedicated resources. By addressing these topics, you can ensure all members of the team are aligned to the eLearning goal with a realistic understanding of budget, time, resources, and limitations. This meeting should organize regularly either online or face to face. These meetings will also need to cover key issues, a defined agenda should be created before any meeting to ensure all aspects are properly covered. Take careful notes to effectively analyze the real-time evaluation data during this meeting.
o Ask your subject matter experts
The relevance of the eLearning content should be directed by the subject matter expert. They should evaluate the content through a lens to ensure the content provided is correct, relevant, and pertinent to the training’s objectives and goals. Subject matter experts should provide feedback 3 times during the development phase. First feedback should be during the content discovery step helping to construct the goals, objectives, and content resource material. The second feedback should be during the eLearning construction and design step to check the course content is correct, relevant, and pertinent to the training’s objectives and goals. The last feedback should be a seal of approval to say all the content is present and taught correctly.
o Ask your participants.
Lastly, you can ask key participants by running focus groups from a sample audience, to evaluate the eLearning experience that your eLearning course should offer. Evaluation from a sample audience in the early phase of your eLearning course’s development could identify greater needs and lead to greater improvements. An effective way to construct your questions for the evaluation is to align them with your eLearning course’s objectives. Additionally, in the later stages of the course development, you can ask your potential learners if your online content is easy to understand, relevant, engaging, easy to navigate, if the images, graphics, videos, and language are appropriate, and if they have any issues. Ask them everything you may have in mind, record their responses, analyze the data, and then revise and modify your eLearning course accordingly. By doing this, you will have the chance to prevent errors and resolve several issues before your eLearning course goes live.
2. During the eLearning Course Phase.
These evaluations are done after the launch of the content and while the learners are actively engaged in the course. The most effective mid-course evaluation is done by tracking learner metrics. By tracking your learners during a course, you gain insight into the course. The most effective and easiest way to do this is by using an LMS. Most LMS will provide you with your learners’ course data in either real-time or daily. Using this data, you can track time in course, number of launches, completions percentage, along with user data such as location and device. This type of data can help you identify course issues before they become a larger issue such as a large number of launches with no completions after an extended period of time or certain locations or devices unable to launch the content.
3. After the eLearning Course Completion Phase.
Once a learner has completed a course is a great time to evaluate the participants’ satisfaction with your course using a level 1 of Kirkpatrick’s Levels of Evaluation. The easiest way to conduct this type of evaluation is either to include the survey at the end inside your eLearning course or if your LMS has the capability to build and pack an evaluation at the end of the eLearning course. Ask your learners about the course’s effectiveness, engagement, relevance, and its effect on their skills, knowledge, work performance. This will allow you to assess if the eLearning course was successful, or if any parts of the course were unclear or incorrect. The post-course evaluation will help to guide you to see what improvements you need to make to deliver an even better eLearning course next time. After a course run, it is important to evaluate your eLearning course’s financial aspects. What was the return of investment, ROI, for the course? It is essential to know whether there was a financial benefit that outweighs the cost of your eLearning course after its completion. You may need to revise your financial and design strategies or keep them the same based on the ROI.
How to evaluate the content at all phases?
To successfully evaluate your eLearning course at all phase, you should have to checklist:
1. eLearning goals, objectives, and needs.
What is the need for this training? A training regardless of modality needs to have relevant reasons for the development. Is this course for compliance, skill/ knowledge gap closure, or skill/ knowledge growth? A successful course should have clear and realistic goals and objectives that align with a specific business need. Is the learning goal of the course supported by the objectives? Your course should always be built on a foundation of the learning objectives supporting the learning goal with ties to a business need. If the answer to all these questions is yes then process to the next step.
2. Modality, design, and interactivity.
With the rise in the popularity of eLearning in this digital world, eLearning is starting to be divided into smaller modalities that focus on delivery types such as video, interactive, audio, and roleplay sicarios. Additionally, eLearning complexity is defined by the level of interactivity. Interactivity has been tied to increase engagement within eLearning but is also tied to increased cost, time to develop, and course complexity. Each graphic and course design layout for an eLearning course is not only affected by the relevance to the content but also the effect on engagement of the learner. The modality, design, and interactivity should be decided not only with the content relevance in mind but also with the learners, technical limits, timeline, and cost as deciding factors.
A. Users Location, Culture, and Language.
Location and Language are key elements of your eLearning course and its design. You will need to identify if there are locational governances or cultural needs required for the training. Are those needs specific to a certain group or can be applied globally? Additionally, is this something that affects a small piece of a course or changes large sections requiring the development of another eLearning course? You will also want to determine if your learners are able to completely comprehend what they are reading or listening to. Is your content well written and free from jargon some learners would not understand? Are the examples you give in your stories or scenarios relevant or excluded from their job, situations, or culture? If you have localized the content of your eLearning course, is the translation effective? Finally, if using humor or idioms to make a point, is it appropriate, or are there some groups of your learners that might find it offensive or not understand the context? You will want to make sure the language you use in your eLearning course is understandable, respectful, and deliberate.
B. Technical Limitations and Aspects.
Your eLearning course is only as successful as the limitations it must work within. A video-heavy eLearning course is worthless if your learners have limited access to computers and slow internet speeds. Learners will become frustrated, disengaged, and the message of the training can become lost and blurred. It is important for you to identify any technical limitations that may affect your eLearning deliverables. You should find out how the course with be hosted, what is the preferred device and browser/ application of launching your eLearning course, are there any firewalls or whitelist that need to be addressed if in an LMS are there inactivity time limits, do the learners use a shared computer, do the computers allow for sound, and other technical limitations that could affect the design, modality, and interactivity of your course. Additionally, you should ensure all technical aspects of the course work within your technical limitations. Check videos, links, interactions, pass/failing scores, layouts, look and work properly in all sicarios you have identified as a common way for your learners to access and complete the training. You should be very detailed and meticulous in this step.
C. Time.
When thinking of the time of an eLearning, it is broken into two parts. The first being the time a user takes to complete training, and the second is the time it takes to develop the training. The longer a learner must sit to complete the training, the greater the loss of knowledge retention and gain. Training should be developed with a chunking model in mind making training easy to digest and allow for learners to complete one part before starting on another piece. Ideally, no training as a whole should require a learner to sit for longer than 20-30 minutes. The ideal time for an eLearning is less than 15 minutes. The time to develop an eLearning is determined by the number of resources and the complexity of the modality, design, and interactions. When developing an eLearning course time frame should be taken into consideration to make sure the appropriate modality, design, and interactions to reach a successful eLearning course.
D. Cost.
Lastly, you should evaluate the cost of your eLearning course. Even if you have successfully met all the other criteria, if your eLearning course costs too much you run the risk of the eLearning not being created or affecting other or future projects. You will need to identify the cost of designing and developing your eLearning course (internal resources, external resources, team members’ time, purchasing cost, etc.), then identify the cost to running it and maintain it (licenses, hosting, updating, maintenance, etc.) and then calculate the cost per learner. Most Importantly, you should include in your cost approximation of reusability, as you are designing a long-lasting eLearning course, which will considerably increase the ROI.
Evaluating during all three phases with a defined checklist is critical to running an eLearning development built on contains improvement.
