I love surveys. I love designing them and completing them. Ever since I was a kid, I’ve filled out satisfaction and feedback surveys every time I’ve been asked. My love of surveys has won/earned me free meals and theater tickets, and discounts or refunds on all kinds of products and services. I love surveys! I’ve even written before about how to design a good survey. So, it might seem odd that I’m going to tell you three reasons not to design a survey.

Nonprofits rely heavily on surveys to assess client satisfaction, staff morale, board engagement, community needs, and more. We can include home-grown post-test outcome assessments in this list, too. When designed purposefully, thoughtfully, and properly, they can be powerful tools. When thrown together quickly, carelessly, and unnecessarily, they can waste everyone’s time and burn out the people you ask to complete and administer them.

So, here are three reasons not to design a survey.

1. It’s not the right tool

If all you have is a hammer, everything looks like a nail. Hopefully, surveys are not the only tool in your measurement toolkit, because they are not the right tool for every job.

Other data collection tools and methods include focus groups, interviews, brainstorming exercises, or ongoing mechanisms for submitting open-ended, unsolicited feedback and input like comment cards.

2. You have not defined the ideas you want to assess.

I would say this is the most common challenge organizations face when designing surveys. Usually, a team or leader has said that the organization needs X. But the team has not yet articulated specific, thoughtful, meaningful, and measurable definitions of of X – that is, indicators. “Satisfaction” with what or whom? What does or should “engagement” look like? Engagement with what?

Outcomes can be particularly difficult to define in measurable terms. I’ve written several times before about strategies for making outcomes measurable, including those that you cannot directly observe. Those same tips can be applied to the process of defining any idea for which you want to define a measurable indicator.

What’s my motto? “You cannot measure what you have not defined.” And the person designing the survey is usually not the (only) person who should be defining these concepts, so get together as a team and decide what exactly it is you’re aiming for, before you try to design a tool to measure it. The time to define your intentions is long before you sit down to write a survey question.

3. You have no plan for using it.

Everyone knows I hate waste, and I am appalled by the amount of time, energy, paper, and postage spent on designing and administering surveys and entering and analyzing data no one knows how to, plans to, or can use!

Articulating your survey’s purpose can be a good safeguard against creating an unnecessary or useless survey. Answer these questions with a purpose statement:

• Why are we gathering this data? What are we hoping to learn?
• Can we use what we learn? If so, how will we use what we learn? To make decisions, inform improvements, plan, make our case, tell our story, etc.

Do not gather data you cannot use. If you can’t change employees’ benefits packages, don’t ask them how satisfied they are with them. If you can’t change the hours the program is offered, don’t ask what times clients prefer. People expect you to act on their feedback.

Do not gather data until you’ve identified a use for it. We are in favor of curiosity and exploration. We talk all the time about the value of asking questions – the right questions, for the right reasons! But if you continually ask questions on surveys just because you’re curious, and you have no plan or ability to use the data for one of the purposes listed above, you’re wasting everyone’s time and diminishing the likelihood people will answer your survey when you really need them to.

Bottom Line

Do not pass go, do not collect (spend) $200 before you are sure a survey is the right tool to assess the ideas you’ve defined and to garner data you can and will use.

Once you know a survey is the right tool, know exactly what you want to learn, and have a plan for using what you learn, then you can go about the business of crafting clear, answerable survey items, which I’ll tackle in my next post.