The field of autism is currently highly focused on "measurable" outcomes. The problem is that what can be most easily and reliably measured may have little to do with what you're actually hoping to evaluate. In the words of Albert Einstein:
"Not everything that can be counted counts, and not everything that counts can be counted."For example, consider trying to measure success in a social situation:
What separates a fabulous party from one that really tanks? The variety of refreshments served? The density of decorations per square foot? The frequency of greetings heard by the doorman? The number of jokes told per person? Any of these measurable factors may be related to what you want to know, but they're not exactly definitive, are they?
To truly define what separates a great social occasion from a disastrous social occasion, you have to move into concepts that are more subjective, and by definition, less easily measured. Did you feel comfortable, warm and welcomed, or anxious, stiff and marginalized? Did the event flow so that you lost track of time, or were you checking your watch every few minutes to see if you'd reached the minimum polite threshold and could exit? Did you relax with friends, laugh and enjoy yourself, or did you analyze and approach each encounter like a military engagement, guarding your words and worrying about the reaction of your conversation partners? Did you enjoy yourself? Would you go back voluntarily?
Social success is not determined by saying the perfect words or greeting a target number of people - instead it's defined by inner feelings of comfort and enjoyment while interacting with other people.
And another question to consider about "success" in therapy:
Sometimes the knowledge and skills we teach make things more (not less) distressing. I regularly teach ASD students how to pay attention to and "read" the non-verbal social communication cues broadcast by other people - this is critical information for social interaction. But it can also be overwhelming information, and once you know it's there you can't ignore it.
When Adam was younger, he was not aware of how his behaviour affected others. When he began to draw, he started to notice more about other people, with both positive and negative effects. The positive effects included voluntarily interacting with a wider circle of people, accompanied by growth in his functional communication skills. The negative effects were more subtle and best illustrated by Adam himself.
This is a drawing from Adam's high school years. During allergy season (spring and fall), Adam would become noisy and would make non-verbal sounds as he worked in the school classroom. This was distracting for the teacher and other students, so he and I addressed it during one of our language sessions. I wrote the sentence "Adam is making noise while the teacher is talking" and he illustrated it. The black drawing is what he drew first (look at his happy facial expression). Then I added in the blue information (the noise that he wasn't aware he was making out loud, and the reaction of the other people in the room):
So what is success then?
Following ASD individuals over the long term in my private practice has challenged and changed my ideas about what constitutes "best practice" in autism intervention. Defining success in social and communication interactions with other people is largely about inner perception and emotional reaction. Intervention is only successful if the ASD individual also (eventually) agrees that the change is positive and something that they want.
When we devise programs aimed at "improving" life for individuals with autism, we need to tread lightly and keep the wishes, perspective and personal preferences of the ASD person at the top of our priority list. Intervention will not look the same for every individual, and that's not only okay, it's desirable. The challenge of evaluating progress using subjective outcome measures is more than offset by the fact that you're helping the individual to build a life that they like, that works, that's "successful".
I started this post with a quote, and so I'll end it the same way, this time with words from Aristotle:
"The worst form of inequality is to try to make unequal things equal."