Building Surveys the Right Way

  • 0

Building Surveys the Right Way

Category : Marketing

surveyConstructing a survey begins with a thorough understanding of what questions need to be answered from the research and what decisions will be made as a result of the survey. Without that knowledge, the survey cannot be designed effectively.

Often, a research company account analyst or a corporate research analyst is responsible for designing the survey. If the survey designer has not been involved with the proposal, be sure the following information is in hand:

  • What are the research objectives
  • What decisions will be made
  • Description of the type of respondent
  • Any estimating parameters (e.g., length of survey)
  • Any special analysis expected
  • Any client or industry information that will provide backgrounding on the topic

There are a number of decisions required when constructing a questionnaire so that the design and implementation is as effective as possible.

Questionnaire Construction

Each component of the questionnaire is important in gaining respondent cooperation and in collecting reliable information, including:

  • Introduction
  • Screening
  • Main Questionnaire
  • Closing
  • Validation

A discussion with examples of each section follows:

Survey Introduction

Begin the questionnaire with an introduction. The introduction should include the following:

  • Reinforce that the respondent’s time is appreciated
  • Invite participation
  • Name the company the interviewer represents, and the name of the interviewer and the client/sponsor of the research when possible
  • Explain the nature of the study topic
  • State that no selling is involved
  • Tell respondent the approximate length of survey
  • Inform the person that the call may be monitored for quality control purposes

Here is a sample introduction:

Hello, my name is ________ and I’m calling from (company). Today/tonight we are calling to gather opinions regarding (general subject), and are not selling anything. This study will take approximately (length) and may be monitored (and recorded) for quality purposes. We would appreciate your time. May I include your opinions?

Screening

The screener is the first part of a questionnaire. It is used to establish the eligibility of the respondent.

Avoid asking “extra” questions that do not establish eligibility. Use truncated versions of sensitive questions: “Is your family income above or below $25,000?”

The main questionnaire is used to address the key objectives of the research.

Questionnaires need to flow naturally by keeping all questions on one topic together. Write transitional sentences: “Up until now we’ve been talking about _______in general. Now I’d like to ask you a few questions about brands of _________.”

Questions that come earlier can affect responses to later questions.

Typical flows:

General category -> specific brands;

What activities, if any, have you participated in during the last 12 months? What products have you purchased? What brands have you purchased?

Evaluative -> Diagnostic -> Classification;

How satisfied are you? Why are you satisfied/not satisfied? What is your household income?

Unaided -> Aided;

What brands, if any, have you heard of? Have you heard of any of the following brands?

There are two main types of questions that can be used in a questionnaire or survey. They are closed-ended or open-ended questions.

Closed Ended Questions

Closed ended questions are where respondents select from a finite set of responses.

Four types of closed ended questions are most commonly used: Rating Scale, Forced-Choice, Dichotomous and Demographic questions.

Rating Scale Questions

Respondents place the object on a dimension. The dimension starts with an anchor and continues on a scale of points usually from 5 to 7 points.

For example:

 

Dark                          Light
6     5     4     3     2     1

Three types of rating scale questions are Likert, Balanced Scale and Semantic Differential.

Likert Type Scales

Example: “Using a scale of ‘1’ to ‘5’ where ‘1’ means very dissatisfied and ‘5’ means very satisfied, rate your satisfaction with the performance of ________ on the following factors…”

Unique Uses: To measure overall attitude toward a product or service or attitudes toward specific facets of a product or service.

Strengths: It’s quick, flexible for analysis and enables easy comparison with competitors.

Weaknesses: It is susceptible to error. It doesn’t answer “why?” questions and you need clear, complete development work before the survey to identify all necessary facets to be measured.

Balanced Scale

Example: “Based on the product description, please tell me if you would definitely buy, probably buy …”

Unique Uses: It attempts to predict future purchase behavior.

Strengths: It’s quick and flexible for analysis.

Weaknesses: It doesn’t answer “why?” questions and you need clear, complete development work before the survey to identify all necessary facets to be measured.

Semantic Differential

Example: “Now, I would like to read a variety of opposites that might be used to characterize the brands. On a scale from ‘1’ to ‘6’ where ‘1’ is slow and ‘6’ is fast, how would you describe ______?”

Unique Uses: It can assess an intuitive or conceptual response to a product or service.

Strengths: It’s quick, flexible for analysis and enables easy comparison with competitors.

Weaknesses: Depending on the abstraction of items, you can have respondent resistance. It doesn’t answer “why?” questions and you need clear, complete development work before the survey to identify all necessary facets to be measured.

Forced-Choice Questions

Respondents must choose among a set of alternatives.

Paired Comparisons

Respondents must choose between two alternatives.

Example: “When you are deciding which ____ brand you are going to purchase, which of the following characteristics is more important? Would you say ___ or _____?”

Unique Uses: It can assess rank ordering of objects and also identify degree of difference between objects. It also forces discrimination among objects.

Strengths: It’s quick and avoids leniency and halo. It forces discrimination among alternatives.

Weaknesses: There are a limited number of objects. It can be fatiguing for the respondent. It doesn’t answer “why?” questions. It has decreased reliability for mid-range responses.

Forced Preference

Respondents choose among several alternatives.

There are two types:

  1. Forced preference ranking approach requires sequential ranking from high to low until all factors are ranked.
  2. Alternation ranking approach requires rank ordering that alternates between most favorite and least favorite until all factors are ranked.

Example: “Looking at the list of ______ and thinking about all of them again, please tell me which one you think is best.”

Unique Uses: It enables identification of “best” and “worst.”

Strengths: It’s quick and avoids leniency and halo. It forces discrimination among alternatives.

Weaknesses: There are a limited number of objects. It can be fatiguing for the respondent. It doesn’t answer “why?” questions. It has decreased reliability for mid-range responses.

Dichotomous Questions

Respondents must choose between two alternatives.

Example: “In the past 30 days, have you seen or heard any advertising for ______?”

Unique Uses: It enables identification of quality.

Strengths: It is quick and does not allow ambivalent answers.

Weaknesses: It does not allow ambivalent answers.

Demographic Questions

Demographic Questions are asked at the conclusion of the study for classification purposes in analysis.

Example: “Which category does your age fall into?” “What level of education do you have?” “What is your racial heritage?”

Unique Uses: It enables researchers to place respondents into categories.

Strengths: It is quick.

Weaknesses: It may be offensive to respondents and doesn’t answer “why?” questions.

Open-Ended Questions

Open-ended questions enable an interviewer the opportunity to probe (digging deeper to get the true meaning behind the response) – be sure probes are open-ended.

There are two types: Free Response and Free Response with Pre-Codes.

Free Response

Example: “Why do you say you are very interested in ______?”

Unique Uses: It permits respondents to define central issues. It enables the development of closed-ended questions. It can collect a respondent’s actual words.

Strengths: It answers “why?” questions. It fully represents a respondent’s understanding of an object.

Weaknesses: It uses more time than closed-ended questions. It requires interviewer probing and requires interviewers to record exact responses.

Free Response with Pre-Codes

Example: “What do you see as the strengths of this product? A. You don’t have to use it with other products. B. Ease of application. C. It is effective. D. Other (specify). E. None. F. Don’t know.”

Unique Uses: It permits respondents to define central issues.

Strengths: It is less time consuming than Free Response. It answers “why?” questions.

Weaknesses: It requires interviewer probing. There is a possible tendency for interviewers to force “other” responses into pre-existing categories.

Other Measures

Attribute Preference
It identifies strengths and weaknesses of a product compared to another product. It also is helpful in determining why a product is preferred overall.

Directional Evaluations

It measures the degree of product strengths or weaknesses.

Constant Sum

Respondents allocate from a constant frame of reference the number of points or purchasing occasions she would give to a list of items.

Communication Series

Used when testing print, video or audio copy, it elicits reactions as well as playback of intended message. Interviewers can probe thoughts and feelings, description to a friend, main point, ad recall, believability, comprehension/confusion, likes/dislikes, and uniqueness.

Lifestyle Battery/Psychographics

It enables researchers to group respondents based on non-demographic traits and analyze data by these groups.

A&U Grid

It provides in-depth information on a respondent’s awareness and usage (ever, past year, etc.) of key brands.

Adjective Checklist

It enables a respondent to choose adjectives that apply to users of a particular product.

Addition/Replacement

It determines intended usage of product within the framework of current purchase behavior. Products considered for replacement are usually identified as the test product’s competitors.

At the conclusion of the survey thank the respondent for his time. Remind respondent that his opinion counts and encourage participation in future market research.

Model Closing

“Thank you for taking part in this survey. Because consumers like you are such a valued part of what we do, I’d like you to think about the survey you just participated in.

To help us assess our own effectiveness, on a scale of ‘1’ to ‘7’

where ‘7’ means ‘very satisfied’ and ‘1’ means ‘very dissatisfied,’ how satisfied are you with your participation in the study today? Of course you may use any number between ‘1’ and ‘7.’

07 Very satisfied

06 Satisfied

05 Somewhat satisfied

04 Neither satisfied nor dissatisfied

03 Somewhat dissatisfied

02 Dissatisfied

01 Very dissatisfied

99 Don’t know

IF previous question answer is 03, 02, or 01:

You said you were dissatisfied with your participation today.

Could you share the reasons for your dissatisfaction so that we might do a better job next time?

(MULTIPLE RESPONSES ACCEPTED. DO NOT READ.)

(PROBE: What else was dissatisfying?)

01 Calling at an inconvenient time

02 Questionnaire was repetitive

03 Poor interviewing skills

04 Poorly worded questions

05 Questions were difficult to answer

06 Questionnaire was too long

97 Other (SPECIFY)

99 Don’t know

Validation

Explain to the respondent he may be contacted again to conduct a quality check.

Model Statement:

“As part of our quality control procedures, someone from our project team may re-contact you to ask a couple of questions verifying some of the information we just received.”

When writing the survey questions, there is a lot to think about. Use these tips as reminders:

  • Be careful of double questions (questions which ask about multiple objects). You do not know which item respondents are reacting to and respondents may be frustrated by the question.
  • Make questions as specific and concrete as possible given the objectives. If you are interested in a specific feature of a product, ask about that specific feature.
  • Use the respondent’s language in asking questions.
  • Ensure that questions have only one interpretation.
  • Make questions as direct as possible.
  • When a single response is required, ensure that categories are mutually exclusive and non-overlapping.

Into which of the following categories does your age fall?
Poor:

01 Under 34

02 35 to 54

03 Over 55

98 Refused (DO NOT READ)

Better:

01 Under 35

02 35 to 54

03 55 and over

98 Refused (DO NOT READ)

Use response categories that are as clear and specific as possible.

How frequently do you use ___________?
Poor:

04 Always
03 Frequently
02 Occasionally
01 Never
98 Refused (DO NOT READ)
99 Don’t know (DO NOT READ)

Better:
05 All the time
04 Most of the time
03 Half the time
02 Less than half the time
01 Never
98 Refused (DO NOT READ)
99 Don’t know (DO NOT READ)

Always provide respondents with an out if they do not feel comfortable answering the question.
However, in telephone interviewing, clearly instruct interviewers that these responses are not to be read to the respondent.

How frequently do you use ___________?
Poor:
05 All the time
04 Most of the time
03 Half the time
02 Less than half the time
01 Never

Better:

05 All the time
04 Most of the time
03 Half the time
02 Less than half the time
01 Never
98 Refused (DO NOT READ)
99 Don’t know (DO NOT READ)

Be sure your scales are balanced.
Having more scale points in one direction than the other biases your results in that direction.

Please rate the value of ______ brand on the following scale.

Poor:
05 Excellent
04 Very good
03 Good
02 Fair
01 Poor
99 Don’t know (DO NOT READ)

Better:

05 Very good
04 Good
03 Neither good nor poor
02 Poor
01 Very poor
99 Don’t know (DO NOT READ)

When asking open-ended questions that ask for how many (number of units sold), be sure to limit the range of acceptable values.
This will limit the outliers in your data and also ensure that respondents are answering the question in the same units in which it was asked.

How many times have you visited a ______ in the past 30 days?

Poor: ________ (Number/DK/RF)

Better: ________(0-30/DK/RF)

When asking open-ended questions that ask how many (number of units sold), be sure to note not to accept a range, only a specific number.
By the same token, when a respondent replies “a 4 or 5,” the interviewer needs to probe to identify which response the respondent prefers.

When using a rating scale, make sure the positive end of the scale is associated with the higher end of the scale.
This makes analysis less confusing.

Make sure all “skip instructions” — which are directions to the interviewer to not read questions in some circumstances — are clear and mutually exclusive.


Leave a Reply


9 − = eight