That might not sound like an earth-shattering proclamation, and I don’t know if I’m the first or only one to say it. But I’m going to make that my primary call to action for nonprofits!
Nonprofits know we are (supposed to be) making a difference in people’s lives, and we know we are supposed to measure things. Unfortunately, too often, what we measure doesn’t demonstrate the difference we are making. Yes, we could attribute that to the lack of resources (time, money, tools) for evaluation. We could stop there and relieve ourselves of the responsibility of smarter inquiry and measurement (and end up wasting energy and time on useless evaluation) . . . or . . . we could increase our focus and clarity, so we get more information from the tools and resources we do have.
If you have a ton of data that doesn’t answer any question you care about, if you don’t know how to interpret or apply the data you have, and if your measurement tools are disorganized and confusing, read on.
Indicators: Defining Success and Guiding Measurement
I strive to set aside jargon and translate the concepts of evaluation and performance management into intuitive language and processes. And because I tend to focus on evaluation as an exercise in reflection, learning, inquiry, and curiosity (not just an exercise in compliance, accountability, and reporting), I focus on asking and answering questions.
Outcomes should answer the question: What difference did it make?
Indicators should answer the question: What will that look/sound/feel like?
Here’s a generic example. Say my New Year’s Resolution (eye roll) is that I want to get healthier in 2018. That’s the difference I want to make. But how will I know that I’m healthier? What will that look, sound, or feel like? Will I weigh less? Will my cholesterol be lower? Will I be stronger? Will I have more energy? There are many different indicators for most outcomes.
Too often, nonprofits don’t define their indicators in clear, specific, and meaningful ways. As a result, they end up with assessments and surveys that might not be measuring what they care about, and they end up with data that may or may not reflect the change they set out to make.
After all, how you define success determines which measurement tool you need.
| Indicator | Measurement Tool |
| Weight Loss | Scale |
| Lower Cholesterol | Lab tests |
| Strength | # of pounds lifted, # of reps |
| More Energy | Self-report |
Here are some example outcome statements that are impossible to measure or interpret, without indicators:
- Youth will be better prepared for college.
- Parent/child relationships improve.
- Clients mental health improves.
- Youth increase social skills.
Oftentimes, organizations proceed directly from outcome statements like those to the selection or creation of assessments and surveys to measure those broad outcomes. If those are all you have to go on, how on earth do you know what questions to ask in your survey? Or what changes to ask your staff to observe and document? How will you know if someone is prepared for college? What does an improved relationship look like? How will you know mental health has improved? Which social skills increase? How can you tell?
Indicators force you to answer those questions in clear and concise ways. Indicators serve to keep your team on the same page, so everyone is measuring the same things in the same ways (inter-rater reliability). Indicators make it perfectly clear to your external stakeholders what difference you make AND how you know you’re making it.
Types of Change
Another challenge a lot of us face is articulating what type or level of change we are seeking to make.
Here’s a simple example. If you teach a financial literacy class, how might you measure the impact of that class?
| Type of Change | Measurement Tool |
| Increase knowledge | A pre/post quiz of concepts: Ex. “Name 3 types of bank accounts.” or “In banking, what is interest?” |
| Increase perception of skills | A client survey or self-report assessment: Ex. “I know how to balance my checkbook.” or “I can set a realistic budget.” |
| Increase skills | A pre/post quiz or observed activity that requires skill application: Ex. “Using the following receipts and bank statement, balance this sample checkbook.” |
| Change Behavior | Ask clients to report their behaviors: Ex. “How many times in the last year was your account overdrawn?” or
Ask staff to observe their behaviors: “Please provide your last 12 monthly bank statements.” |
I’ve seen many organizations say they increase skills, but they are measuring increased perception of skills. Similarly, many say they are changing behavior, but they are only measuring changes in knowledge or skills. I don’t think these are intentional attempts to mislead anyone or misrepresent a program, but I do think it represents a lack of clarity and focus in their outcomes measurement strategies.
Bottom Line
You have to know exactly how you are defining success before you can effectively measure success! When organizations put the cart before the horse and build surveys and checklists before defining their indicators, they end up wasting valuable time and resources and get little useful information in return.
Does your organization have clearly defined, measurable indicators? If not, try asking and answering those two key questions above. Listen for different definitions of success among your team. Identify themes. Consult the research. Then state clear, measurable indicators and align your measurement tools.
Need help to define your outcomes and indicators, select or create measurement tools, and pull it all together to answer the questions you care about?
Through our 12-month training and coaching program Operation: Outcomes, organizations receive the training and coaching they need to articulate meaningful and measurable outcome statements and develop the tools and processes to measure them.