Welcome back to Designing Learning Sunday, the place you can go to for insight and instructions on how to create learning for your employees. Here we take an everyman’s approach and are breaking down learning and development into small bites that anyone can do. My name is Instructor Dave, and here we go.
If you need to get caught up;
DLS Blog 1 – Is This Training Or Coaching?
DLS Blog 2 – Needs-Based Assessment?!
DLS Blog 3 – The Design Document
DLS Blog 4 – Choosing a Delivery Method
DLS Blog 5 – The Learning Objective
DLS Blog 6 – The Storyboard
DLS Blog 7 – Information, Activities, and Engagement
DLS Blog 8 – Supporting Documents Last time we talked about Supporting Documents and how they need to support your learning that you are creating. This week in Designing Learning Sunday #9 - Assessments!? we are talking about Assessments. We have all taken assessments or tests at some point in our lives, in school, or without.
Assessments are used by many to gauge learning, “understanding of a subject”, or proficiency of a skill. Not too different from education where assessments are given at the end of a lesson, and you are tested over your knowledge of the subject, employers and educational institutions use assessments to assess a benchmark of performance or “understanding”, or a skill. When designed and delivered appropriately, assessments can be a valuable asset in determining where proficiency exists with a learner, or where gaps lie.
More than just “fill out this 10 question assessment”, assessments hold more power than many realize. Taking tests and assessments is easy for some, and challenging for the next. Think back to the last time you had to take an assessment. Did you study for it? Were there questions on the assessment that did not make any sense? Was it easy? Was it challenging? Did you just breeze through it, or are you like so many adult learners, and have test anxiety? Yes, test anxiety is a real thing. Having taught at the corporate level for over a decade and a half, I can say with certainty that test anxiety exists. But, why? Why is this important and relevant? Well, let's scroll back a paragraph or two and take a look at what assessments are “supposed to do”.
We established that they are to measure performance, understanding, or a skill, right? This is true. However, in the business world, especially when you are in class for a new job, or job skill, assessments hold an entirely separate level of built-in stress. For many companies, if someone fails an assessment (as some with test anxiety might) for whatever reason, they may lose their employment. This is serious, and for the instructor, designer, and team behind them, employee attrition for performance during training can be a big deal. I mean, you would not want to lose someone who can do the job simply because they’re a bad test-taker, would you? No. Yet, many do. But, that is not why we are here, this is L&D made easy, so let’s focus on that.
When designing an assessment I want to you think about this question; “What are we trying to assess?” In the past, I have been asked to include an assessment as part of the training request, and when I follow-up and ask what they would like to include in the assessment, the requestor didn’t know. This was pretty normal. Including an assessment in learning is an expectation, but as I learned, few people know what that should be. So, I’ll give it to you straight on it.
Remember that question from a short time ago? What are we trying to assess? The learning objective. That’s right. The assessment should be written to support the learning objective and measure it. Because, at the end of it all, the learner should be able to complete what the objective was, and we should be able to measure that.
With me so far? The learning objective tells us our goal, and the assessment measures how well we have accomplished that goal.
As with everything L&D, assessments come in a variety, each with a different purpose, and depending on your company their terminology may be used interchangeably. Within our organization, we use the following;
Written Assessment – Assessments where the learners have to answer or respond to questions by writing down or recording their responses. Similar to testing in school or at university where a paper test or online test is taken. This is the most common form of assessment, easy to grade/score, and is used for tracking performance.
Spoken Assessment – A call and response type of activity that is often used during discussion or lectures where the instructor asks a question and calls upon learners for an answer. This also includes teach-back activities and presentations made by learners. More challenging to grade/score, and require a rubric or predetermined
measurement criteria.
Skill Demonstration – Exactly as it sounds. This activity requires the learner to demonstrate their level of proficiency with the skills they have just learned. Essentially, Skill Demonstration is “can the learner effectively do what they have just learned how to do?” Measurement for Skill Demonstration can be challenging if you are looking to include the learner's proficiency, however, please remember that proficiency is built through practice.
While being a very high-level overview of the assessments that we use, written evaluations, spoken evaluations, and skill demonstrations have more components to them, and also come in different variety. For example, there are;
Knowledge Checks (small topics – reinforcement)
Short assessments added during the lesson to reinforce the learning, often used as a follow-up to build upon what was recently covered. Quizzes (In the middle of lessons – reinforcement)
An assessment that is given in the middle of, or at the end of a short topic that is part of a larger topic.
Assessments/Tests (Complete topic – recall on prior learning)
The term is used interchangeably and is the larger assessment at the end of a lesson
On a personal level, I am not a fan of assessments. At all. Let me explain...
Here is why; Assessments are only half of the measurement. Can the score a learner receives on an assessment tell you how much of the learning material they learned or understood? No. It cannot. There is a generic benchmark score (that we all use) of 80% that is required to pass almost any assessment. With all of the assessments that you took while you were in school, at university, or training for advanced job skills, did you always do well? And with that, did you learn anything? More importantly, did the assessments do anything to enhance, or reinforce your learning?! No, not really. Some will argue with me about the effectiveness of assessments, and I will gladly take that discussion, but as I stated above, assessments are only half of the measurement. They are only half of the pie. Assessments and tests, primarily, only prove that people can research (if they are allowed notes), can retain information, or can take a test. When given alone assessments, do not determine that any learning occurred!
For assessments to truly be effective, we have to include some type of evaluation. Assessments only tell us the yes/no, right/wrong. They do not tell us how well, how much, or if it was understood. This is where adding evaluation mechanics to our assessments comes in. Now, we are not here to discuss performance metrics, that will be in a future blog, it is imperative to understand that there has to be an evaluative component to our assessment.
We need our assessments to truly be impactful, we must include in our assessment design, some way for the learner to demonstrate their learning. If we can do this, we can better determine where any deficiencies may exist and address them as needed.
We can do this through our question writing and supporting the learning objective.
Assessments have different types of questions, and each one should be used for a specific purpose. Let's talk about question types for a minute.
True or False questions are the most common form of an assessment question. It is a statement that results in a yes or no, this or that, A or B answer.
Multiple Choice questions provide the learner with a question that has several options to choose from, where only one specific answer is correct.
Multiple Answer questions have the learner provide information for a specific question that lists several options available with multiple options being correct, and some being incorrect.
Situational questions have a specific situation written for the learner, with a bank of available answers to choose from with the best response being the correct one. This can be a True/False, Multiple Choice, or Multiple Answer question. This question type is the most effective to promote learning.
Essay-based questions require the learner to either write down or describe their response to a situation.
So far, we have talked about what assessments are supposed to do for us, but we have not talked about what to include in our assessments, or how to write the assessment questions.
Listed below are the 10 Best Practices for Assessment Creation that we follow;
1. Keep the learning objective in mind
2. Do not confuse your learners with the questions or their answers
3. Keep it simple – do not over complicate the questions or answers
4. Write in clear and complete thoughts
5. Use a variety of question types - Do not make every question True/False, or Multiple Choice
6. Its easier to score assessments when the number of questions is in multiples of 10
7. Keep the questions relevant to the learning – no hypotheticals or what-ifs
8. Have your scoring criteria (% needed to pass) determined before you begin
9. Have someone “vet” your questions and answers for accuracy
10. Be willing to be wrong
If you are creating e-Learning and writing assessments there, each of the software types out in the world offers an assessment/quiz maker and will allow you to not only choose the type of question you want to build (create) but also includes a scoring mechanic so that you can track a learners progress. This is especially beneficial if you are going to upload your course or assessment into an LMS (Learning Management System)
If you are not familiar with e-Learning don’t worry! In the late spring, we will roll out a series on creating e-Learning, to help people get started. With all of the tools available now, it doesn’t take much to get started, and we’ll show you how (for free) in that blog series.
There is a standard that is widely considered acceptable for assessment scoring, and that is 80% for a passing grade. This seems to be reasonable, however, I would like to you include something as part of your learning design. At the end of the Knowledge Check/Quiz/Assessment/Test – whatever you are calling it – include follow up or remediation.
Allow for the learners to see what they got wrong, and if possible provide why. Many do not have this practice, and honestly, they should. Earlier we talked about assessments being one-sided, this is why. If we do not provide the learner with a way to learn from what they got wrong, or why they got it wrong, we are taking away their ability to learn from their mistakes. Spend the extra few minutes on the follow-up, either with your instructor who will lead a “Class review” or include it in your e-Learning. This is perhaps the most important part of designing assessments for an instructional designer, and one that is often overlooked.
When time permits for you, also have a colleague or your subject matter expert review your assessment. Have them look at it not just to make sure that it is factually correct, but have them review it for syntax, and that it makes sense. It's always good to have a second set of eyes on all of our projects. Design your assessments to be trackable once they are completed. One of our practices is to make them scalable. This doesn’t mean that the questions get harder the further you go in the assessment, meaning Question 1 should not be more difficult to answer than Question 10.
It means that the Knowledge Check leads into the Quiz; the Quiz builds on the Knowledge Check and leads into the Assessment, and the Assessment builds on both the Knowledge Check and the Quiz to reinforce the learning. This allows for a progressive learning approach and builds on the foundation created at the beginning of the learning.
Phew. That was a lot to digest today, wasn’t it? Anyway… Assessments. Design with the learning objective in mind, keep them simple, and make sure they support the learning. If you get stuck on a question, move on and come back to it, just like if you were taking an assessment yourself.
Assessments aren’t easy to create. To help you out, we have attached a .pdf with a few examples of the question types and answer types. You can do this.
Please reach out if you have any questions, and always be learning! - Instructor Dave
#crossroadlearning #desinginglearningsunday #dls9 #infoactivitiesengagement #education #land #blog #dlsblog #instructionaldesign #madeeasy #assessments #learnerengagement
Comments