5 Things to Do to See If Your Vocabulary Instruction is Effective (& 5 Things to Do If It’s Not)

Last month, we talked a lot about semantic intervention with people who are learning AAC. Once we got started, we realized we could have done another whole month on the topic, but we had to move on. We ran out of time before we could really talk about outcome measures. As clinicians, how do we determine whether our therapy is effective? Here are some things to do after you’ve provided high-quality, well-sequenced vocabulary instruction.

1. Make small comprehension checks a regular part of your instruction. Ask the AAC learner to tell you about ___ . Then score their response as objectively as possible (e.g., complete & correct, correct but incomplete, vague, incorrect). You’ll get some real-time feedback and can clarify or re-teach as necessary.
2. Assess in a standardized fashion. Standardized means doing something the same way each time. Set up appropriate assessment tasks that allow you to judge how well the learner a) understands the new words, and b) is able to use them. Depending on the individual situation, you may be able to make this kind of pre/post testing a regular part of your therapy routine.
3. If the word is pre-stored in the SGD/app, use the device’s internal data collection tools to track the usage of target words. Look for patterns over time.
4. Use informal methods. Set the stage by preparing the learners so they understand what you are listening for and why. “Tessa, here are the words we have been working on this month. I’m going to be listening for these as we ____ and I’ll make a mark next to each one I hear. That will help me know if what we’ve been doing is really working for you, okay? Do your best to show off the new words you’ve been learning.” Then do some language sampling in unstructured or semi-structured activities. Analyze word usage (if you are looking at expressive language) and/or descriptive language (if you are assessing comprehension.)
5. Get data from other stakeholders. Find out what you can about how the learner is demonstrating understanding of new words in daily activities. An aide who is with the learner throughout the school day can be asked to record every time he/she hears the AAC learner use the target word (a good option if the word is not prestored in the SGD/app). Or the aide can report back on ways in which the AAC learners demonstrated their knowledge of the word through descriptive language (e.g., ‘fair to everyone’ rather than ‘equality’; ‘have enough but not too much’ for ‘sufficient’). Look for soft signs, too. “I’m trying to figure out how much Billy knows about our vocabulary words. What do you think he knows about ___? What makes you think that?” Asking a parent or aide questions like this, can give you some insight into the data you’re getting.

Not seeing the progress you were hoping for? Here are a couple of next steps to consider.

1. Check to see whether your implementation is sound. Sometimes we start out strong, but then drift away from the methods and strategies we know to be successful. (Just observe a session of PECS if you don’t believe us. Chances are, the therapy procedures don’t resemble the instructions in the manual very closely.) Over time, we tend to do our own thing. That’s okay if it’s working and you have the data to prove it. If not, go back and freshen up. Re-read some of the posts on vocabulary instruction to see if there are areas where you’ve drifted a bit too far. Nothing to be embarrassed about-it happens to all of us. The key is recognizing it and doing what it takes to get back on the path.
2. Increase the number of elicitations. Focused (aided) language stimulation and direct instruction is most effective when the AAC learner has lots of opportunities to retrieve and say the new words. If you were getting about 10 elicitations in a session, try for 15-20. Coach the other stakeholders on this as well.
3. Spend some time with the data and try to evaluate it relative to the client’s overall learning profile. How does the rate of their vocabulary learning compare to other things they’ve learned? If the learning trajectory is about the same, you may just need to give it more time and/or make the therapy more intense. If their word knowledge is coming along more slowly than other things (e.g., MLU, morphological endings), then you may want to do some more in-depth problem-solving.
4. Adjust the focus. Are the kinds of words you’ve been targeting skill appropriate for these learners? Maybe they would would do better with other kinds of words, or words that are earlier in the developmental sequence. Is the intervention too focused on direct instruction of new words, with too little time for practice and solidification?
5. Adjust the intensity level. Almost all learners acquire knowledge/skills faster when provided with more intense instruction. We may need to provide more intense instruction or increase the amount of practice that the AAC learners get with a particular set of words. This is the time to rally the troops and get other stakeholders on board.

Assessment isn’t the focus of what we do as SLPs, particularly when we work with people learning to communicate using AAC. However, it does play an important role in shaping our therapy. We know that clinicians aren’t researchers and often can’t do the well-controlled, full scale investigations of what the learner absolutely knows/doesn’t know in every clinical situation. On the other hand, if we can’t measure the effectiveness of our treatment, is it really therapy?