In all my teaching, training, and consulting, I’ve seen a continuum of attitudes about data, evaluation, learning, performance management and the like. Like any continuum, there are extremes at either end. There are those who love and value it (like me, and maybe you?) and those who dread or resent it.
Here’s some of what those extremes look like, in my experience:
| I hate it! | I love it! |
| Data is lifeless and limiting. | Data is power and possibility. |
| Data slows me down and gets in the way. | Data can help me learn and grow. |
| Evaluation and reflection are luxuries. If only! | Evaluation and reflection are critical to success. |
| Data misses the point or tells me what I already know. | Data opens my eyes and uncovers secrets. |
| Collecting data is a distraction and a burden. | Collecting data is the means of describing our work. |
| We can’t afford to spend the time or money on this. | We can’t afford not to! |
When I teach, train, blog, and consult, I’m usually preaching to the choir (or at least to the converted). And almost every time I teach or train, I get asked, “How do I get _______ to care about data/outcomes/evaluation?” Fill in the blank with whatever group of front-line service provider the questioner needs to get on board with their efforts: case managers, therapists, teachers, nurses, mentors, tutors, volunteers, etc.
I’ll share with you my best recommendations, by way of explaining why (I think) they currently might not care (much) about your data.
Why Don’t They Care About Data?
Because It’s the Wrong Data
- It’s irrelevant. For staff to be invested in collecting data in a timely, accurate, and complete way, it has to be relevant and meaningful to them. If they see what you’re collecting as too obvious, too general, too specific, or unrelated, they won’t be interested in collecting it or using it.
- It’s invalid. If the indicators or measures you’re using are missing the mark, your data will tell a story that doesn’t reflect the reality of your staff’s experience or tell the story of their work. I can’t tell you how many times I’ve heard staff teams respond to data with, “Yeah, but . . .” and then rattle off a list of reasons why that data isn’t valid or trustworthy.
- It’s meaningless. If the outcomes you aim for are too general or the goals you set are too low, the data won’t add any value. It won’t tell anyone anything they can use. They’ll look at the data and say, “So what?” “What’s that mean?” or “Tell me something I don’t know!”
How do you get the “right” data? Involve several levels of staff (and clients!) and different perspectives in identifying your evaluation and learning questions. Ask them what they wonder about and want to know. Ask them what information would help them make decisions and be more effective.
Just for kicks, here’s one of my favorite cartoons that illustrates this point so well.
Because You Don’t Use It
- Share it. If you never share the data they collect, and front-line staff never get the chance to see or interact with it, they reap no rewards from it, and therefore will be less invested in gathering it. It disappears into a black hole, along with the time and energy they spent collecting it!
- Interpret it. Even if you do share it, but the data isn’t interpreted or translated, it’s not useful. Seeing a bunch of numbers or percentages on a page isn’t very compelling or engaging. Again, that just leaves them asking, “So what? And?”
- Use it. If the data is irrelevant to decision-making, and no one is willing to put their weight on it, staff feel the futility of that. What’s the point?
For data to turn into information, and for information to translate into learning, it’s got to be shared, engaged, and used! Share summary reports. Ask questions about what the data means, what might have caused an increase or decrease, why this group does better than that group, etc. And make sure that during planning and decision-making processes, your data is one of the factors at play.
I’ve got a cartoon for just about everything!
Because Your Process is a Pain
- It’s extra. If data collection is a separate, extra task that gets tacked onto their already huge workloads, it will be the first thing they sacrifice when the work they care most about demands their time and attention.
- It’s redundant. If staff have to document the same information in multiple places or formats, they’ll resent the wasted time and effort. Rightfully so!
- It’s confusing. If it’s not clear what a question, field, or response choice means, staff will get confused and frustrated and you’ll end up with data that incomplete, invalid, or inaccurate.
To the extent possible, integrate your data collection into existing tools, forms, and processes. Make it part of how they do the parts of their job they care about and value most. Weave it seamlessly into existing practices, so it’s not this clunky extra thing that gets in their way and slows them down.
And provide adequate instructions and training, so staff feel confident in what you’re asking them to do. By the way, that training should emphasize why the data is important, how it will be used to make their work more efficient and effective, and how it improves the quality of services to clients!
Now What?
Take a look at your data, and see how irrelevant, useless, and painful it might be and try some of the strategies above to make improvements where you can. Also, please share any strategies that have worked in your organization to create cultures of learning and improvement and to make data-informed decisions.
I’m teaching a workshop at the Brown School at Washington University on September 8th, 2017 all about measurement strategies. Come check it out!
Great points! For the past year and a half, I’ve been sharing my program’s quarterly outcome and output results via Power Point during team meetings. Since doing so, I feel my team (therapists at a non-profit social service agency) better understand the impact of their work and our program’s goals. I engage them in discussions about the “why” and get their input on interpreting the results. It’s always a strengths-based and growth-oriented discussion. One downfall of these efforts is hearing buzz that people are worried the data may be used against them or the program, cared about too much over the qualitative indicators of their work, etc. I try to toe the line of reflecting that data is not the only indicator of quality work, but is one of the most dependable (if you are doing it correctly, as you mentioned). I wanted to point out another reason I think direct service staff (and probably many program leaders) may be resistant to data collection and sharing is the fear that the information will be used to judge them or their program negatively… in terms of who gets cut or who gets raises… or conversely, in a climate when many non profits don’t provide performance-based raises, there may not be enough motivation to hit every goal individually.
You’re right, Julia. It is a fine line to walk. I like your approach – data isn’t the ONLY measure of quality or success. It is one language, of many, we can use to tell our stories. And where staff are afraid the data are missing part of the story, there’s an opportunity to explore ways to capture those missing pieces.