Research panels and the data that we gather are key tools that marketers, academics, and other professionals use to validate their ideas, inventions, and strategy -- but using a research panel provider can be daunting, especially if you’re unfamiliar with survey design. Crafting a thoughtful, well-structured questionnaire is crucial to conducting surveys. In this article, we’ll explain the top seven best-practice tips for survey design:
***toc
Let’s get started!
Determine your research goal
At Centiment, we help everyone from startups assessing the viability of a new product to enterprises looking to better optimize their offerings that have been in the market for decades. No matter what stage of its lifecycle your product or service is in, you’ll need to distill your end goals down to a single focal point to conduct a successful survey project.
While Centiment can introduce you to your target audience, that audience has a finite amount of attention. You have just a few minutes to convey your topic and dive into specific elements for which you need respondent feedback.
We suggest working backwards. Identify 1-3 crucial topics or elements that would benefit from respondents’ feedback and opinions. By identifying your goals ahead of time, you can ensure that the first draft of your survey questions feature pointed opportunities to collect direct feedback that you can then use to better inform your business decisions.
Sharing your research goal with your Centiment fielding expert will bolster our combined opportunities for success, so don’t be shy: tell us what you want to achieve and we’ll help you get there!
Data analysis dictates structure
How do you plan to leverage the information collected from your target audience into something actionable?
Think about your research goal and the information needed to accomplish that goal. For example, if your research goal is to determine if your target audience would be interested in a new frozen pizza product, then it would be helpful to gather data on their current shopping habits, spend on groceries, and their first impressions on your product design. It would also be important to gather demographic data, such as gender, age, and household income, to build your product’s buyer persona.
Use your research goal to structure your survey by considering how you will use the data post-collection, then build in relevant questions to allow those data analysis opportunities, such as segmenting by demographic data. It’s important to note that, while we target respondents’ demographic data on the back-end, any information you’d like in your final output will need to be asked in-survey.
With that said, there are several strategies you can use to thoughtfully structure both your survey and its questions in order to yield the best results; the first of which is brevity.
Brevity is key
Brevity increases the probability that respondents will stay engaged with your survey build. Engaged respondents read questions and corresponding response choices carefully in their entirety before selecting a response to find the answer that best aligns with their opinion.
Ask the fewest number of questions possible to achieve your research goals. Don’t bloat your survey with unnecessary questions or provide too many response options for each question. Respondents typically stop reading response options the moment that they find the first choice that aligns with their opinion. Surveys that are shorter in length and don’t ask an excess of taxing questions have better engagement levels than long, drawn-out surveys which may prompt respondents to adopt less-than-optimal completion strategies.
Brief questions feel manageable for respondents and keep them engaged. They also lessen the opportunity for respondents to feel confused or lost in the survey. Another strategy to help survey respondents help you is to make your question set accessible.
Write clear questions in Plain English
As a researcher, your job is to write unbiased, non-leading questions that survey respondents can easily understand -- in other words, respondents should know exactly what’s being asked of them every step of the way. Ask respondents about things that they know in order to boost the accuracy of your data set.
To write questions clearly, use simple language free of industrial jargon and acronyms. If you do need to use an acronym, define it for your respondents.
Your aim in creating a survey is not to confuse, trick, or trap your respondents -- you’re looking for their honest feedback and they’re looking to give it, so eliminate anything that doesn’t serve this goal. For example, don’t include convoluted or misleading verbiage and/or opportunities for a respondent to contradict a previously-stated opinion. Be sure that each question is clear, direct, and brief with only one possible interpretation. Avoid metaphors and other pieces of speech that lend themselves to misinterpretation.
Need a little help writing your questions? We’ve written a crash course to teach you how to avoid common question-writing errors. If you’re still feeling overwhelmed by the idea of writing clear questions that yield actionable data, our research pros can help.
Our next tip for building a successful market research survey questionnaire is to maximize your data’s impact by relying on closed-ended questions.
***cta
Closed-ended questions reign supreme
We define closed-ended questions as questions with all response choices listed so that respondents can select an option that aligns with their personal opinions/experiences. Multiple choice questions, true/false questions, and matrix grids are all examples of closed-ended questions.
Thoughtful question format and design are critical to collecting a data set that will provide actionable insights. When working with quantitative panel providers such as Centiment, closed-ended questions will yield the best quality data. We consistently find that closed-ended question formats lead to the most accurate and actionable data outputs. We have a list of different question formats here if you’d like to be inspired, but below is a quick example of a closed-ended question for context.
Closed-ended Question Example
- Please select all major pizza brands that you’ve ordered from in the past three months.
Open-text questions are questions that ask respondents to answer through a written response. In terms of survey questions, open-text questions lend themselves to the greatest degree of interpretation - from both the researcher and the survey respondent. This lack of structure often leads to an array of text blocks which can make it difficult to draw clear distinctions and/or identify patterns that will inspire actionable results.
Open-text Question Example
- What are the three most frustrating aspects of ordering pizza online?
Respondents from survey panels are not accustomed to creating their own custom content and will easily burn out. Here at Centiment, we ask that researchers limit the number of open-text questions to two or fewer per survey experience.
Please note that you’ll want to be very clear about how you’d like respondents to answer any type of open-ended questions. Make it as easy as possible for respondents to provide the information you need by defining your terms and asking specific questions.
Worried that you’ll miss an important response option if you used closed-ended questions? Many survey tools have an ‘Other’ field response option with closed-ended questions, such as multiple-choice questions, which combines the flexibility of open-ended questions with the structure of closed-ended questions.
One final note on closed-text questions: be as mindful in writing your response options as you are in writing your questions. Ensure that your closed-ended response options encompass every respondent experience to the best of your ability. Additionally, there should be no overlap between numeric response options as that will cause confusion and will decrease the quality of your data. See the example below to better understand overlapping response choices so that you can avoid this common survey mistake:
Closed-Ended Question: What is your estimated annual household income?
Poor bracketing (with overlap)
- $30,000-34,000
- $34,000-40,000
- $40,000-45,000
Thoughtful bracketing
- < $30,000
- $30,000 - 34,999
- $35,000 - 40,999
- $41,000 - 45,999
- $46,000+
Curious about the most common closed-ended question types and the data that they yield? Check out this quick article that can help you choose the perfect question format to support your unique research goals.
Your job isn’t done after your questions are written. The next step in the process is to consider how you’ll keep your survey respondents engaged with your questionnaire.
Engagement - the good, the bad, and the boring
Happy respondents are engaged respondents -- and engaged respondents are those who read each question in your survey carefully before selecting the response option that most closely aligns with their lived experience. In short, engaged respondents provide the highest-quality data outputs. As you write your questions, keep the respondent's experience at the forefront of your mind. You don’t want to bore respondents, but how can you keep them engaged without losing sight of your research aims?
The simple answer is by keeping your survey fresh and accessible.
Using a mix of question formats, incorporating visuals, and limiting your questions to 4-6 per page will make for a more dynamic and engaging survey. Remember that the respondents providing you with feedback are human beings like you and me, so show empathy for your respondent and create a survey that fosters engagement; it will pay off in your data quality.
Refrain from using repeated question formatting consecutively within your survey — mix it up! Very few people who have signed up to join a survey panel actually enjoy wading through 80 lines of a matrix set. Being thoughtful in your survey design will create a good experience for your respondents, which translates into high quality data for you.
As you organize your survey questionnaire, consider where you could add an attention check question. An attention check question is a tool to measure respondent engagement that we recommend adding one in within the first three minutes of your survey. For more details on attention check questions and other key question types, check out our accompanying article here.
So far, we’ve covered strategies for what to include in your survey design and how to structure your questions. Next we’ll explore what you cannot include in your question set in order to protect our respondents.
Personally Identifiable Information (PII) - it’s…personal
In these days of governmental regulation of data processing, personally identifiable information (PII) is more protected than ever before. Respondents cannot be required to provide PII to complete a survey. Your Centiment fielding expert will advise you upon reviewing your survey questionnaire if there are PII requests within your question set, but here are some typical examples of PII to avoid collecting:
- First and last name
- Company name
- Phone number
- Email address
Researchers sometimes question how they can trust their data if they’re not allowed to collect PII from respondents. If you have questions about who’s providing you feedback, take a look at our article on respondents and the security methods we use to ensure top-quality data collection.
Conclusion
Online survey panels like Centiment are fantastic resources to aggregate actionable data -- but only if you thoughtfully structure your survey questions with the respondent experience in mind.
First, determine your research goal and consider how you’ll use your data to support that goal early on. Your data analysis strategy will inform the overall design of your questionnaire. Add in demographic questions for segmenting opportunities if you’ll use them, but be mindful and only ask relevant questions. Be brief, clear, and concise with your question set. Don’t use jargon or acronyms without defining your terms. Use closed-ended questions to yield the best quality of data -- but if you do need some open-text feedback, remember to limit your usage to 2 or fewer questions per survey experience.
Keep respondents engaged by using a mix of question types and incorporating visuals where appropriate. Your respondents are ordinary humans and they’re here to help you reach your research goals, so help them help you by creating a great survey experience.
If you follow these best practices in survey design, you’ll end up with a thoughtful questionnaire that engages respondents. Engaged respondents are happy respondents and happy respondents produce the best, most actionable data. Top-quality, actionable data will allow you to verify your hypothesis, whether you’re focusing on growing your business, establishing your buyer personas, or simply testing a new logo. The opportunities are limitless -- so get out there, start designing, and, when the time comes, Centiment will be here to help you collect your data!
