The assessment of Pragmatics can be daunting and ambiguous for many speech-language pathologists. Understanding the types of skills that fall into this category can be just as confusing for parents and professionals. Those of us who went to graduate school more than fifteen years ago probably had one unit in a Language Development class on the topic of Pragmatics. Much of what is known in the field now stems from clinician trial and error efforts while working on the therapeutic front lines. I take particular interest in the area of Pragmatics given my extensive caseload of learners with ASD over the last decade. In the most recent four years, I have focused on refining my assessment practices in a way that is all inclusive of the breadth of pragmatic skills, but also quantitative in nature. Quantifying pragmatic skill, and measuring meaningful change in pragmatic skill has been a challenge for clinicians for years.
Foremost for site visitors, I want to clarify the meaning of the term, “Pragmatics”. Pragmatics simply refers to the USE of language. This probably does not seem “simple” and leaves you even more mystified. The term “Pragmatics” more directly refers to Social Language skills. Most people agree that this includes conversation skills and non-verbal communication skills such facial expression and body language. Pragmatic Language also includes the “prosidic” features of speech and language – tone of voice and using vocal inflection to give meaning to a word of phrase. Some people believe that pragmatics ALSO refers to the understanding of the inferred nuances of social language. This includes understanding the perspective, of point of view, of multiple interaction partners. It includes being able to infer how interaction partners might react to what you have to say. Therefore, it refers to adjusting your own social language accordingly. It is more than just conversational turn-taking – it is recognizing the subtle nuances of a break down in a conversation and knowing how to “repair” that conversation to keep the interaction going positively. Social Language also refers to being able to analyze a problem – identify the problem, a potential cause and come up with solutions. Finally, Social Language refers to using perspective taking ability to consider the listener’s level of interest and background knowledge on a topic in order to provide enough information to help the listener understand and relate to a story or conversation.
When one considers what “most people” believe falls under the heading of Pragmatic skill and what “some people” also believe is Pragmatic skill, you may already have guessed that I fall into the “some people” category. One purpose for this site is to help other clinicians, professionals and parents become more familiar and comfortable with this expanded view of Pragmatic skill taking shape in the field today.
There are a few standardized, norm-referenced formal measures of Pragmatic Language. Speech-Language Pathologists visiting the site may be familiar with some of the following:
-
The Test of Pragmatic Language (TOPL)
-
The Pragmatic Profile with the Clinical Evaluation of Language Fundamentals-IV (CELF-IV)
-
The Test of Problem Solving (TOPS)
-
The Pragmatic Judgment subtest of the Comprehensive Assessment of Spoken Language (CASL)
-
(NEW) – The Social Language Development Test (Published by Linguisystems – 2008)
Many times clinicians and parents find that these types of standardized measures indicate that a student has scored in the average range on such previous “pragmatic judgment” types of behavioral measures that explore social language and problem solving skills. However, it is important to note that these types of testing situations are unnatural in many ways, thus the score may reflect average “knowledge base” in this area, but the application of this knowledge in actual social situations can be more problematic. It is important that given any medical diagnoses on the autism spectrum, an educational categorization of Autism Spectrum Disorder, and/or historical social relatedness concerns, that a child’s team take a qualitative look at social cognitive differences that such students display. These differences may include: difficulty perspective taking (“Theory of Mind” deficits), difficulty connecting peers’ reactions to their own social behaviors, reading non-verbal communication cues, reciprocal conversation skills, and interpretation of social situations.
Many times, evaluation of such needs can include an informal “dynamic assessment” that is more qualitative in nature. The time spent with the student is considered more of a “diagnostic therapy session” as opposed to a formal evaluation situation. The student can participate in a variety of tasks requiring him or her to demonstrate perspective taking skills, processing in a gestalt (or holistic) way, interpret non-verbal communication cues, demonstrate emotion understanding, social problem solving, and an in depth analysis of their conversation skills. It is important to note that evaluations should go beyond a traditional speech and language evaluation of “behavioral” pragmatics that examines directly observable use of language skills. Assessments should take into account these BEHAVIORAL pragmatic skills, as well as more subtle COGNITIVE pragmatic skills.
BEHAVIORAL Pragmatic Skills include: Conversational turn-taking, topic maintenance, initiating interactions, sportsmanship, sharing skills, eye contact, and interpreting facial expressions/body language. Essentially a Behavioral approach looks at how a person is using the language they have in a functional way. These skills are traditionally assessed observationally by speech-language pathologists as a part of a traditional communication assessment.
However, when examining the complex neurological underpinnings of a learner with social communication needs, it is equally important to explore the Social COGNITIVE aspects of the learner. These Social COGNITIVE Pragmatic skills include: perspective taking ability, understanding peer’s hidden intentions, knowledge of un-written social rules or “codes of conduct”, the ability to read non-verbal & situational cues, social self-awareness with the ability to adjust personal pragmatic behaviors when needed, understanding figurative language or social “expressions”, being able to socially “filter” responses inappropriate to given situations, be flexible with other people’s opinions, topics and interests, and to independently to analyze a social problem arriving at solution options rapidly, are all skill sets that build a foundation for age appropriate social cognitive skills. These cognitive skills are not traditionally assessed in a finite manner with traditional speech-language evaluations. Including tasks that contain aspects of both BEHAVIORAL and COGNITIVE aspects of social pragmatic behavior provides a comprehensive view of a student’s Pragmatic/Social Language skills. Behavioral and Cognitive Pragmatic skills are not static or isolated skills, rather these two categories are dynamic and interdependent to create the overall social presentation of a person.
While clinicians are likely more familiar with traditional Behavioral assessment techniques such as examining conversational turn taking, topic maintenance, eye contact, etc., I have provided some additional ideas for clinicians to consider that are both Behavioral as well as Cognitive in nature.
Below, I have provided some ideas of informal Pragmatic Assessment tasks you could consider.
False Belief Tasks
There is an abundance of research documenting the use of various types of tasks to examine an individual’s “theory of mind”, or ability to take another person’s perspective. Many of these tasks probe for the recognition of a “false belief”. Here are 3 examples of levels of False Belief:
The Unexpected Contents Task: This is an example of a “First Order False Belief Task.” A First Order False Belief task is one that a student needs to consider the perspective of one other person. Take an empty crayon box and put 5-6 paperclips in it before the student arrives. Show the student a crayon box and ask what is inside it. You are hoping the student will say, “crayons” at this point. Then, open the box an show the student that there are really paperclips inside. Then ask, “If your friend came in here and we asked him what was inside the box, what would he say?” Passing this First Order False Belief task would yield a correct answer of, “Crayons“
The Sally-Anne Task (Change of Location Task): This is well known First Order False Belief task that can take many forms. One can use a variation of what is referred to in literature as, the “Sally-Anne” task. You will show the student 4 pictures with drawings corresponding to a story. The story depicts “Sally” placing a doll in her toybox. When Sally leaves, her brother comes into her room and moves the doll into the top dresser drawer. The last picture shows Sally returning to her room. The student is asked, “Where will Sally look for her doll?” Passing this First Order False Belief task would yield a correct answer of, “her toybox“.
The Mary-John Thought Story: This is an example of a “Second Order False Belief Task”. In a Second Order task, a student needs to take the perspective of another person in the story, who themselves, need to consider another perspective. It is like perspective taking of 2 layers. This story also has pictures you can draw to correspond to the following story. “Two children are playing in a park when the ice cream man arrives. Johns leaves to get some money. Mary watches the ice cream man leave the park to go to the church to sell ice cream. On the way, he sees John and tells him about his plans. Mary still thinks that Johns thinks the ice cream man is at the park.” Then, the student is asked, “Where does Mary think John will go to buy ice cream?” Passing this Second Order False Belief task would yield a correct answer of, “the park.”
All of these tasks require very basic level perspective-taking. Failing performance in these tasks indicates highly significant needs in perspective taking. Research shows that typically developing children are able to pass the first two tasks by the age of four, and the Mary-John Thought task by the age of six. Passing these tasks offers a good prognostic indicator for students to be able to continue to build skills in this area. However, higher order perspective-taking requiring students to consider and rapidly adjust their own social behavior based on multiple perspectives during actual “real-time” peer interactions can be quite problematic for students.
Social Sequence Narrative
Citation: Michelle Garcia-Winner, Thinking About You Thinking About Me (2007), Think Social Publishing, San Jose, CA. Available at: www.socialthinking.com
In this task, the student is given a set of 4-8 sequence pictures that have a social theme. You can use sequence pictures from any vendor. Consider the age and language level of the student when deciding upon how many pictures to include in the sequence. For my elementary age students, I typically choose 6-8 photos. You must make sure that the sequence contains 2 characters. Be sure to choose a storyline that contains at least one picture where two characters appear to be talking to one another, or at least, could be talking to one another. Another criteria point to consider when choosing the pictures is to find a storyline that depicts some sort of problem-solution. I use a series of 6 pictures depicting a young boy and his mother finishing breakfast and leaving the house to walk to school. One picture scene shows that they forgot the boy’s backpack. As they arrive at school, they realize the problem. The final picture scene shows the mother arriving in a car to drop off the backpack.
First, ask the student to sequence the pictures in correct order. Then, tell them you want them to “tell the story”. They will create a verbal narrative to tell the story, which should include evidence of social inferencing. Finally, ask the student to provide a title for the story, and create some “dialogue” in one of the pictures you choose. In my story, I choose one picture in which the mother and boy look like they are talking. This portion of creating dialogue two characters can demonstrate higher level perspective-taking skills beyond what the False Belief tasks yield.
This task is important to explore how a student is able to identify a context of a situation as a whole. Many times, learners on the autism spectrum process information in a very detail focused, splintered manner, rather then holistically, as one main idea. When the student is asked to give the whole story a title, you are looking to see if they can come up with a single title that would encompass both the context and the problem in the scene. Difficulties in the area of processing information as a whole context can impact a student in the ability to summarize a chapter, write a book report that requires more global interpretation of material, or describe longer verbal answers in a class discussion in a succinct manner. Further, this task examines how learners can interpret people’s intentions and thoughts (perspective taking) while reading body language, facial cues, and looking at context cues. Difficulty in this area can impact a learner’s reading comprehension, especially as he or she moves into more complex chapter book, fiction material, or material that contains a great deal of figurative or abstract language.
Emotion Understanding
Citation: Michelle Garcia-Winner, Thinking About You Thinking About Me (2007), Think Social Publishing, San Jose, CA. Available at: www.socialthinking.com
Assessment task examining emotional understanding encompass the comprehension and use of non-verbal communication skills such as facial expression and body language. These tasks also incorporate the COGNITIVE aspects of Social Thinking which also greatly impact a person’s Pragmatic skills. Note that all of these informal tasks provided have the primary purpose of obtaining more complex Pragmatic information about a student, but ALSO serve the purpose of yeilding some initial baseline quantitative data about a student’s performance. Measuring Pragmatic progress on IEP or Treatment Plan goals is a confusing, frustrating, and time consuming process for many clinicians. Note the occasions to write IEP objectives highlight below. You can use these tasks to weave the assessment results directly into a subsequent IEP or Treatment Plan.
I use three components to examining a student’s Emotional Understanding.
1) Interpretation of Facial Expressions and Context Cues
2) Understanding Emotions in Oneself and Others
3) Emotional Vocabulary base
Interpretation of Facial Expressions and Context Cues
In order to work on this task, you will need to do some preparation. Don’t say Ugh. Once it is done, you can use this tasks over and over! 
THE PREPARATION:
First, gather about 10 photographs of various emotions being displayed. Then, gather about 10 line drawings of emotions being displayed. Some research states that some students have an easier time identifying facial expressions with line drawings versus photos or vice versa. Using two different stimuli could reveal some interesting data on this front. Secondly, gather about 5-10 cards depicting social scenes. There are several speech-language therapy products with which you can pull some cards. A pack of problem solving cards, inferencing cards, or even copy some picture scenes out of a regular speech-language treatment resource. As long as there are at least two characters and some kind of basic problem depicted, the cards will suffice. I have pulled five cards from an LDA Social Problem Solving card pack. Then, write 3 questions for each card. The first question is always, “What is happening in the picture?“ You are looking to see if the student can identify a main idea and overall description of the social problem. The next question is an “Emotion Question”, such as, “How do you think ___(point to someone in the card), feels?” You can follow this question up with, “How do you know that?” or,”What clue in the picture made you think that?” to elicit deeper explanation and understanding. The third question is a “Social Inference Question”. This question is a bit tougher to write. The key is to focus on eliciting information about what a character might be THINKING or INTENDING. This is different from the emotion question. A Social Inference question might be, “What is the girl in the picture thinking?”, or “What does the boy in the picture really want?” , or “Does the other character know that he is being tricked?” You may need to write more than one Social Inference picture for each cards to really get at the student’s perspective taking skills. Scenarios that involve some degree of minor deception work well. Scenarios where the two characters have different beliefs about the situation work well. One of my favorite pictures I use depicts a young boy about to steal a picture from a cookie jar. He is sneaking a peak to see if anyone is looking. What he does not see, is that his sister is standing nearby watching him. His mother is in the picture with her back turned to the whole scene. Each of the three characters have a different perspective or set of “beliefs” about the moment in the picture. While this task does take a bit of planning and preparation, if you take the time to create a little recording protocol for yourself with the questions listed, and package the cards with your self-made protocol, you could pull this little “kit” out when you are doing some pragmatic assessments.
USING THE TASKS:
Now, you are ready to work with the assessment task. First, show the facial expression photos to the student. Tally using a +/- system the facial expressions correctly identified. You then can get a percentage that serves as a baseline for an IEP objective. <STUDENT> will increase his ability to identify facial expressions in photos from a level of 30% accuracy to 80% accuracy! Even though this is an informal assessment task, it directly can yield baseline data and provides you with a unit to measure progress.
Next, you are ready to show the student the Social Scenario pictures. Show the student the picture and without any story or preface, ask “What is happening in the picture?” Remember, you are examining if the student can process information holistically, as main idea or theme, or if he or she gets “stuck” on small details. Then ask your second “Emotion” question about the picture aimed at seeking the student’s ability to describe emotions of the people in the picture. Finally, ask your third “Social Inference” question(s) to assess the student’s ability to make Social Inferences using non-verbal cues or context cues in the photos. Total up all the correct versus incorrect answers for each of teh three categories to yield a baseline percentage. You now have some MORE baseline data to create an IEP objective. For example, you could write an objective, “Given pictures of various social scenes, <STUDENT> will demonstrate the abiity to make social inferences from a level of 40% accuracy to 80% accuracy.” Once again, even though these Pragmatic Assessment tasks are informal in nature, they directly examine both BEHAVIORAL and COGNITIVE aspects of Pragmatics AND are user-friendly in that directly provide a quantitative baseline and IEP objective. What could be better!
Understanding Emotions in Oneself and Others
A second task used to assess a student’s Emotional Understanding requires them to describe a time that he/she felt a particular emotion AND a time that another peer their age might feel that given emotion. The description of two different scenarios is important here to assess perspeective taking. Sometimes students can only think of one scenario for an emotion, usually the one they have personally experienced. They just merely keep re-stating the same scenario in different ways. It is important to see if the student can “put themselves in another person’s shoes” and surmise a totally different scenario. Or as an alternative, you could ask the student to describe a scenario that would elicit the feeling in a child versus an adult. This also requires higher level social thinking skills. For my own elementary age students, I typically ask about the following emotions: excited, disappointed, frustrated, jealous, scared, proud, bored, and worried. Once again, gather data to yield a percentage for the two scenarios: 1) Ability to explain emotions related to oneself, and 2) Ability to infer a DIFFERENT scenario that another peer their age (or adult) would experience that emotion. For the second context, be sure that they really can demonstrate some social perspective taking and describe a totally different context that they described for themselves.
Emotional Vocabulary Base
There are a couple of different tasks you can pursue to access information about a child’s emotional vocabulary base. First, you can ask the student to give a “synonym” for a list of emotions in order to get a sense of the student’s emotional vocabulary in general. You could also provide a list of 5-7 synonyms for a general broad emotion and have the student rate each synonym on a level of intensity for the emotion. For example, given the general “Feeling Family” of MAD, you might have synonyms such as: annoyed, irritated, upset, angry, furious, and irate. You can write these words out on cards and have them place the cards on a scale of Big to Little MAD feelings. Or, you could use a 5 Point Scale if the student is familiar with this strategy. You could even label the most extreme level of intensity, “The MOST” and the other extreme, “THE LEAST”. Use whatever structure makes sense for the child. Assessment of emotional vocabulary and an initial sense of understanding of gradations of emotions is a valuable tool to use when working with non-verbal pragmatic language skills. AND, yet again – you have an informal assessment task that can directly relate to the formation of an IEP objective! You could write an objective such as, “Given a named emotion, <STUDENT> will provide at least 2 synonyms for that emotion from a level of doing so for 20% of emotions to 80% emotions named.” Or, you could write an objective such as, “Given a list of 5-7 synonyms for an emotion category (i.e. happy, sad, mad, scared), <STUDENT> will place the vocabulary words in order of intensity for 3 out of 4 emotion categories.”
Conversation Skills
An in depth conversational analysis is essential for the assessment and measurement of progress for learners on the autism spectrum. There are numerous variables to examine. In many student’s case, it is valuable to examine how a student is able to balance conversational turns with a conversational partner, rather than dominating the conversation-especially if the topic is really interesting to him/her. It is also important to look at how a student is able to maintain a topic of conversation when it is a topic that someone else has initiated. In both of these cases, a child should be able to have approximately an equal percentage of conversational “turns” for themselves and the conversational partner, when in a dyad.
It is also important in conversation that participants have a balanced number of comments and questions in a conversation. Many times, students on the autism spectrum demonstrate a larger percentage of comments versus questions when examining their conversation contributions. Comments largely serve the purpose to describe one’s own thoughts and experiences about a topic. Questions however, demonstrate social awareness by seeking to inquire about another person’s thoughts and experiences. Questions make the conversation reciprocal in nature and show the conversation partner that you are interested in what they have to contribute as well. A larger percentage of comments in a conversation may indicate lacking social thinking skills and an inequitable social exchange.
In addition to thinking about meeting the needs of both partners when conversing, one needs to continually take the perspective of the listener to monitor for comprehension and interest in the topic. At times when interacting with my students, I have observed that they do not provide enough background knowledge for me as a listener when they are talking about a topic of special interest to them personally, or when the student is telling about a personal event or story. As a listener I am left to “work hard” in the conversation, continually asking questions to comprehend. It is not surprising that this is a conversational behavior for learners on the autism spectrum, because it requires adept perspective taking skills to think about what kinds of prior knowledge different conversational partners have and how much background information you might need to share with them to help them understand.
Recording of conversational data has been a challenge for clinicians in many arenas. While some clinicians prefer to audiotape or even videotape conversation samples, I have found myself with limited time to analyze these recordings, necessitating a “real time” data collection system. I share this system here – it may be helpful for some clinicians as a starting, or could be used to supplement a system that already works for you. There are 2 parts to my system of Conversation Analysis, each with different goals.
PART I: “Conversation in Progress”
PART II: “Conversation in Distress”
NOTE! You do not need to do the conversation samples all at once! As the student visits over the several testing sessions, incorporate the conversation opportunities during break times between subtests of traditional language tests, prior to testing sessions, or at the conclusion of testing sessions. Not only will this make your assessment easier, the conversations are elicited in more natural contexts.
PART I: “Conversation in Progress”
Goals of Part I:
- Assess the “balance” of turn-taking
- Compare topic maintenance when the topic is initiated by the target student versus one that another person initiates.
- Balance of Questions versus Comments
- Level of cueing required for question asking
In Part I, aim to gather conversation samples with the student and another peer. You will record three distinct topics of conversation between the students. You record your data as an observer (not a participant) in the conversation using a coding system. You will then analyze the data to obtain averages and percentages – thus, quantifying conversation skills to track progress in an objective, numerical way.
THE CODE:
On a recording sheet, you use two lines to record data. While doing the actual assessment, you will record BOTH lines at the same time while observing a conversation. For the purposes of teaching the coding system here, the two lines will be explained and displayed separately.
The TOP LINE has codes to represent WHO the speaker is. In most cases, use the first initial of the conversation participants. For example, this is a conversation between 2 students named Dillon and Bryan. Note that I am not writing down WHAT they are saying – I am merely notating who is taking the conversational turn.
Example:
D = Dillon
B = Bryan
Topic: Going to football practice this weekend
Initiated by: Dillon
D D B D B D D D B D B D
Next, the second line of the code represents if the conversation contribution is a COMMENT or QUESTION. Write a “Q” or a “C” under each person’s initial to indicate the type of contribution. The code looks like this with the second line inserted:
Example:
D = Dillon (Target student)
B = Bryan (peer)
Topic: Going to football practice this weekend
Initiated by: Dillon
D D B D B D D D B D B D
C Q C C Q C C Q C C Q C
Once again, this particular system is not concerned with writing down the content of the conversation, only coding conversational behaviors.
The example above represents a fully coded conversation of ONE topic. When other topics are introduced, begin this same process on a new piece of paper or lower on the recording sheet. I typically use post-it notes when coding. I use a new post-it note for each topic.
Finally, to analyze data from this conversation, you will determine the following:
1. Number of Conversation Turns taken by the target student
“Target student took 8 of the 12 conversational turns (66% of the turns)”
2. Percent of Questions versus Comments given the target student’s conversation contributions
“Of those 8 turns, 6 were Comments (75%), and 2 were Questions (25%)”
In order to obtain more thorough data, continue this process for two more conversation topics. For EACH topic, obtain the data described above. Then, take the data and average them together.
Here is an example of the complete process:
CONVERSATION ANALYSIS
D = Dillon (Target student)
B = Bryan (peer)
Topic: Going to football practice this weekend
Initiated by: Dillon
D D B D B D D D B D B D
C Q C C Q C C Q C C Q C
D – 8 Turns (66% of total turns) – (of those, 75% C, 25% Q)
B – 4 Turns (33% of total turns) – (of those, 50% C, 50% Q)
Topic: X-Box 360 Star Wars Complete Saga game
Initiated by: Dillon
D D B D D B D D B B D
C C C C C Q Q C C Q C
D – 7 turns (64% of total turns) – (of those, 86% C, 14% Q)
B – 4 turns (36% of total turns) – (of those, 50% C 50% Q)
Topic: the new Basketball hoops on the playground
Initiated by: Bryan
B D D B D D B
Q C C Q C Q C
D – 4 turns (57% of total turns) – (75% C, 25% Q
B – 3 turns (43% of total turns) – (66% C, 33% Q
ANALYSIS SUMMARY: (3 topics averaged together)
D – average of 6.3 turns in conversation with 1 peer (8+7+4 = 19/3 = 6.3)
D – takes an average of 62% of conversational turns (66+64+57 = 187/3 = 62)
Of those turns, an average of 79% of Dillon’s contributions are Comments (75+86+75=236/3=78.6)
and 21% of Dillon’s contributions are Questions (25+14+25 = 64/3=21.3
In an ASSESSMENT, I would report the findings like this:
“Three conversation topics between Dillon and a familiar peer from his classroom were observed by the clinician and analyzed. The conversation topics were: football practice, an X-Box video game, and the new basketball hoops on the playground. Dillon initiated 2 of the 3 topics with the peer. Data was taken from the three topics and averaged. On average, it was found that Dillon takes an average of 6.3 turns in conversation when with 1 familiar peer. In the three conversations, Dillon took 62% of the conversation turns, and the peer took 38% of the turns. Of Dillon’s turns, 79% of Dillon’s contributions to the conversation were Comments and 21% were Questions. This data indicates that Dillon tends to dominate conversations with peers somewhat given the fact that spends nearly two-thirds the time talking (62% of the conversation turns). Additionally, the data shows that Dillon does not seek information about his conversation partner’s thoughts or experiences through questions as much as his peer did. A small percentage of Dillon’s conversation contributions were questions, 21% as compared to his peer who used 44% of his conversation contributions to ask questions in efforts to show interest in Dillon. In summary, Dillon would benefit from Social Language work focusing on increasing his ability and comfort asking questions to others to show interest in his conversation partners in order to create more reciprocal and mutually satisfying interactions between peers.”
The overall result of coding various conversation behaviors is a method of quantifying data to show progress in a skill that has traditionally been difficult to measure. This method lends itself to the creation of IEP goals/objectives that have a definitive baseline with which to measure progress. An objective focusing on increasing a student’s skill in asking questions to “balance” the conversation to be more socially reciprocal might be: “Given three conversations with one peer observed and analyzed by the speech-language pathologist, <STUDENT> will increase his frequency of asking questions to his partner from 21% of his contributions being questions to an average of 35% of his contributions be questions directed toward his partner.”
Part II: “Conversation in Distress
Goals of Part II:
- Recognition of breakdowns in conversations
- Attempt to repair breakdowns in conversation
Part II examines a student’s ability to recognize a “breakdown” in a conversation, and how the student attempts to “repair” that breakdown. Use different conversation samples than used for the Part I analysis.
There two contexts that I explore Conversation Breakdown and Repair skills:
· Context 1 – Examine the Frequency and Type of the student’s conversation break downs
· Context 2 – Examine how the student attempts to repair a conversation breakdown
CONTEXT #1:
The three types of conversational breakdowns I explore include:
- Request for Clarification: a listener does not understand, so they may verbally request for clarification by saying something such as, “I don’t understand”, or they may non-verbally request clarification by giving a confused facial expression.
- Non-acknowledgment: a listener does not show signs that they are interested, or even truly listening during a conversation.
- Wrong Shift: a listener abruptly switches the topic, or terminates the conversation without any of the expected and social appropriate subtle signs they want to talk about something different or end the conversation.
Count the number of conversational breakdowns that occur in each of the three conversation samples. Average the three samples together to arrive at an average number of breakdowns occurring. Record which category each breakdown represented. In an evaluation report, I might report this information like this:
“Given three, 5-minute conversation sample periods, Johnny demonstrated an average of 4.6 conversation “breakdown” behaviors. “Breakdown behaviors” indicate that Johnny may seem confused about the conversation- yet not seek clarification, he would describe a personal story without providing adequate background information for the listener to comprehend, he may ignore his conversation partner, or he may unexpectedly change the topic to one of his own interest. Johnny would do these types of behaviors an average of 4-5 times in a 5-minute period. Most of the time, Johnny was observed to either ignore the partner’s comments and questions, or quickly switch the topic back to his preferred topic of Pokemon.”
This data can directly transfer to a Social Language IEP goal/objective such as: “Given three 5 minute conversation samples, <STUDENT> will decrease the number of conversation breakdown behaviors from an average of 4.6 to an average of 2.0.” You can use the same type of data collection and measurement procedures as you did to obtain this baseline in the assessment.
CONTEXT #2:
Context 2 examines how a student can recognize breakdowns and how they attempt to repair them. To do so, the clinician purposely “sabotages” conversations by initiating breakdowns in order to see how the student responds to them. There are five categorizations of Conversational Repair strategies that I examine:
· Repetition: person repeats what they have said nearly verbatim
· Recast: the person changes the form of what they are communicating
· Modification – Addition of Content: person adds more detail
· Modification – Deletion of Content: person simplifies the message
· No repair strategy attempted.
“Sabotage” behaviors on your part may include actions such as: giving an exaggerated confused facial expression, saying “I don’t understand”, turning your back while the student speaks, walking away during the conversation, abruptly switching the topic, etc.
I typically do the following 6 Conversation Break-Down “sabotages”:
- Exaggerated confused facial expression (Type: Request for Clarification non-verbally)
- Say, “I’m not sure what you are talking about” (Type: Request for Clarification verbally
- Turn my back while the student is speaking (Non-acknowledgment)
- Yawn and sigh loudly while laying my head on the table (Non-acknowledgment)
- Say, “I once baked the greatest apple pie” in the middle of a topic (Wrong Shift – Topic Change)
- Say, “Whoops – I gotta go” and pretend to pack up to leave (Wrong Shift – Termination)
Record the types of sabatoges you used and record whether or not, a) the student appeared to react to them, and, b) which repair strategy they attempted to use. I might report this information in an evaluation report like this:
“Throughout the conversation samples, Johnny was given opportunities to encounter purposeful conversation breakdowns that I inserted into our interaction. For example, in one instance I said, “I’m not sure what you are talking about” to assess whether he clarified his information for me. In another instance, I abruptly changed the topic by saying, “I once baked the greatest apple pie.” I found that Johnny attempted to repair 2 of the 6 planned breakdowns. In both instances, he used the repair strategy of “Repetition” – repeating verbatim what he had just said. Social Language intervention should focus on helping Johnny identify when a breakdown has occurred, what type it is, and to learn and practice a larger variety of “repair” strategies to use in such situations. He needs to work on looking for opportunities in his interactions where he did not communicate clearly, or he did not understand his partner’s viewpoint.”
This data can directly transfer to a Social Language IEP goal/objective such as: “Given a 5 minute conversation with the clinician, <STUDENT> will demonstrate conversation repair strategies of: asking questions, adding details to increase background knowledge, or re-stating his point, for 4 out of 6 conversation breakdowns executed by the clinician.” As therapy progresses, one can use the same type of data collection and measurement procedures as those used to obtain baseline data in the assessment.
Interpretation of Figurative Language
Pragmatic comprehension also depends on one’s ability to recognize that most language/communication is not intended for literal interpretation. To interpret adequately, one must be able to be flexible enough to infer the intended meaning of the message; and often one must pursue the analysis of language/communication in context to seek the intended meaning. Abstract and inferential meaning is often carried subtly through verbal and nonverbal means of communication. This skill begins to develop in preschool and continues across our school years as the messages we are to interpret, both socially and academically, become more abstract. Interpretation depends in part on one’s ability to “make a guess”; it also depends on one’s ability to take perspective of another’s thoughts based on what you know about them. Idiomatic language, or “figures of speech” is often an effective way to examine a student’s ability to interpret non-literal meaning.
Put together a list of 10-20 idioms that a child the student’s age may hear. State the idiom and ask the child what it means. For each expression, you can discuss the silly “Dictionary Meaning” of the words, and then ask what the “Social Meaning” of the words is. Gather a percentage of idioms they are able to define without any context clues provided, as described above. Then, contrast this with a percentage of idioms the student can define when provided a brief context, as you use the idiom in a short story or sentence. Once again, the beauty of these informal assessment tasks is that you can directly form a measurable IEP objective such as, “Given a list of common idioms, <STUDENT> will describe the social meaning of the idiom from a level of 70% accuracy with a context to 80% accuracy without a context provided.”
Informal Tasks created by: Michelle Garcia-Winner
Michelle Garcia-Winner, M.A., CCC-SLP is a pioneer in the work of social cognition. She has authored several books and resources on the topic and speaks worldwide to speech-language pathologists and other professionals about assessment and intervention for individuals on the autism spectrum. The following two tasks are derived from her work.
· Source: Winner, Michelle G. (2007). Thinking about You, Thinking about Me – 2nd edition. Think Social Publications, San Jose, CA. http://www.socialthinking.com
“Listening with the Eyes”
First, tell the student that you are going to play a game with your eyes. Ask the student to “look at my eyes to tell me where I am looking”. The expectation that student would look at the clinician’s eyes, look toward what he/she was looking at, and then check back to make sure they was figuring it out accurately. Next, you will complete the same task, but this time, you ask the student to tell you what you are “thinking about.” I typically do 5 trials with each context for a total of 10 trials.
This task allows for an examination of how a student uses visual referencing determining what other people might be thinking about. Children naturally learn to actively explore how to “thinking with one’s eyes” helps to figure out people’s physical plans as well as to try and figure out their mental states. Difficulty with this skill can impact success socially as well as in the classroom. Students with difficulty visually referencing may have extreme difficulty in a larger mainstream classroom where they need to actively track the eye gaze rapidly of other students and the teacher. Students may be unknowingly tricked or taken advantage of by peers given that they have difficulty reading the intentions of the eyes. These statements are examples of statements that can put into an evaluation report in order to substantiate the educational impact of this pragmatic deficit.
The “Double Interview”
This is an evaluation task also developed by Michelle-Garcia Winner. There are 3 parts to the Double Interview task.
Part I: Examiner interviews the Student
Part II: Picture Interpretation
Part III: Student interviews the Examiner
PREPARATION:
Find three photos of yourself that depicts you involved in some kind of social scenario. You can choose “posed” family pictures or “action shots.” It is helpful to choose photos where there is at least one element in each photo that would elicit a question or comment about which the people in the photo are, the setting of the event, or why there are certain objects in the photo. Prior to using this assessment, you may want to show the photos to a few children from each grade level you work with in order to get a feel for types of comments and questions you could expect from typical children. The photo contexts I use are as follows:
1) A photo of me with my husband and two young sons-each wearing tuxedos
(Questions typical peers asked for this photo were: “Why are they wearing tuxedos?”, “Who are those kids?”, “Is that your husband?”, etc.)
2) A photo of me with my 1-year-old niece sitting at the bottom of a slide at a playground. (Questions typical peers asked for this photo were: “Who is that girl?”, “I thought you had sons, who is that?”, “Where are you?” “Is that the Westview playground?”)
3) A photo of me, my husband, two sons, and parents celebrating my three-year old’s birthday at Chuckie Cheese. (Questions typical peers asked for this photo were: “Is that the Burnsville Chuckie Cheeses?”, “Whose birthday is it?”, “Who are those other people?”, and “What games did you play there?”)
COMPLETING the DOUBLE INTERVIEW:
PART I
In this section, the clinician “interviews” the child, asking questions about hobbies, chores at home, feelings about school and friends. Include questions related to the student’s ability to take the perspectives of his/her family members such as “If your mom had one day all to herself, what would she do? What would be her perfect day?” or, “What are two things that annoy your sister, but do not annoy you?” or “If you want a special privilege or permission for something, is it better to ask your mom or dad? Why?” Record answers to these questions while attending to the ease at which the student shares, visual referencing/eye contact, perspective taking ability/insight into family members, etc. Look for patterns of idiosyncratic speech, prosodic differences, perseveration of specific topics, and how the child is able to adjust the amount of background knowledge they provide.
PART II
The second part of the Double Interview is to show the student the three photos that you prepared. Tell the student that you enjoyed getting to know them a bit; you now have some photos of yourself you would like to share. When showing these photos to the student, say nothing at first. It may seem like a very long awkward silence. Just wait for any questions, comments or responses. When I have completed this task, I have had several instances where the student merely looked at the photos with no verbal response. When this occurs, ask, “What do you think about this picture?” to elicit responses. Record all comments and questions that the student says. In some instances, they may say nothing, or say, “Oh.” As you become more comfortable with the Double Interview, especially through completing it with children that do not exhibit social language deficits, you will become more adept at identifying lacking or odd responses to the photos.
PART III
In the third portion of the “Double Interview” task, tell the student that they are now going to “interview” you. Explain that they can ask anything they want to know about your family, hobbies, etc. The goal is to assess whether the student can ask questions that are clearly “other-person directed” – questions that clearly seek information about another person’s thoughts, experiences, etc. Many times learners will ask superficial, or scripted types of questions like “What’s your favorite color?”, “What’s your favorite food?”, etc. This is typical for elementary age students. I would judge these types of questions as a genuine attempt to learn about another person. However, in other instances, learners will subtly ask questions to a person that really are targeted attempts to talk about themselves, such as “Have you ever played Skeeball at Chuckie Cheese? Once I played and…” and so on. Or, they may ask, “What’s your favorite Pokemon?” in an attempt to initiate a topic they are especially interested in, but within the “supposed” context of learning about you.
Difficulty with the Michelle G. Winner’s Double Interview task has obvious social implications for students with social language challenges. There are also academic and classroom implications. In the classroom, students may appear either very shy, talking to few, or be perceived as odd because they gregariously engaged in topics that only pertain to their areas of interest or life events. Students may be viewed as a challenge in the classroom due to lack of participation or over participation as they tangentially relate all subjects back to their own experiences and preferred topics. Finally, students will likely have difficulty with cooperative group work efforts as they have difficulty taking other’s perspectives and adjusting their own social behaviors to make the group interaction successful. Once again, these statements can be used in a social language evaluation report to document educational impact and need for social language services.
You have explored several examples of informal Pragmatic assessment tasks on this page. Check back at a future date as more ideas will be shared!
© Jill D. Kuzma, Minneapolis MN, 2008. All Rights Reserved.
Neither this document nor its concept may be duplicated, distributed, or re-published in any format without written permission from the author/owner.
