One of the key quality measures in survey research is response rate.

Simply stated, the response rate is the percentage of people who respond to a survey. It is calculated by dividing the number of people who submitted completed surveys by the number of people the researcher attempted to contact.

This rate is important because good survey response rates help to ensure survey results are representative of the target population.

A survey must have an acceptable response rate to produce accurate and useful results. Yet, good response rates are becoming more challenging to achieve. The proliferation of cell phones, the decline in households with landlines, new do-not-call rules, and people’s growing reluctance to participate in surveys are all contributing to declining survey response rates.

This poses two important questions for survey researchers:

1. How low can response rates go and still provide reliable results?
2. What can researchers do to ensure the best possible response rates?

Unfortunately, there are no easy answers to either question.

How low can response rates go and still provide reliable results?

As with most questions of this type, the answer is: “It depends.” Several factors must be considered when answering this question, such as the size and composition of the survey population, mode of data collection, and willingness of the survey population to participate in the survey.

The problem that is exacerbated by declining response rates is called non-response bias. Non-response bias occurs when there is a significant difference between the people who responded to your survey and those who did not.

There are a variety of reasons for non-response, including:

• Some people simply refuse to participate in the survey.
• The survey is poorly constructed and places an undue burden on some respondents.
• The survey didn’t reach all members in the sample, such as a telephone survey that excludes respondents without landlines.
• Certain groups are more — or less — inclined to participate.

If we manage these problems effectively, we can accept somewhat lower response rates, because the overall response rate is not as important as getting a representative sample of the population we are interested in.

The potential for non-response bias is greater when response rates are low, but low response rates alone don’t necessarily mean that non-response bias exists. Many studies have found that when non-response bias is managed, lower response rates do not negatively impact that quality of the survey results.

At Nielsen Scarborough, we employ a variety of procedures to manage nonresponse bias in our local-market syndicated surveys:

• First and foremost, we design our surveys to minimise the burden on respondents and move them through the survey logically and efficiently.
• We include cell phone-only adults in our sample frames so as not to exclude this growing population.
• We attempt to develop a relationship with the respondents by contacting them regularly with reminder notes and phone calls.
• We employ differential survey treatments for groups that are less likely to respond to surveys, such as younger adults and Hispanics.

These differential survey treatments include supplemental sampling, carefully calibrated cash incentive regimes, and a variety of respondent contacts, reminders, and incentive promises. The result of this work is a better representation of these harder-to-reach groups within the sample.

What can researchers do to ensure the best possible response rates?

First, you must determine how your response rate will be defined. Earlier, we defined response rate as the number of people we surveyed compared to the number of people we attempted to survey. But there is more than one way to measure response rate.

A good resource is the American Association of Public Opinion Research (AAPOR). The AAPOR guidelines can help you determine acceptable response rates for various data collection modes, such as telephone, mail, and online.

Generally, e-mail surveys have lower response rates than mailed paper surveys, and face-to-face surveys have the highest response rates. Telephone surveys can have higher response rates than mail surveys, and vice versa. As we have seen, a number of factors can impact response rates.

There are plenty of tips for improving response rates. Here are a few that have worked for us over the years.

• Notify survey participants in advance that they will be receiving your survey.
• Explain the purpose of the survey and how their responses will be used.
• Be considerate of the respondent’s time and design the survey accordingly.
• Keep the respondent engaged with frequent reminders.
• Brand your surveys. Respondents want to know you are reputable.

At Nielsen Scarborough, we invest in an ongoing R&D programme to develop and test new ways to engage respondents and keep our response rates strong. We test different graphic designs for our survey materials and respondent contacts. We test the cadence, frequency, and mode of our respondent contacts and reminders. We also test incentive amounts and the relative advantages of cash up front versus incentive promises.

We do this on an ongoing basis, and we continually tweak our procedures. Because if there is one thing we have learned for certain from years of testing, it is this: If you keep doing the same thing, your response rates will decline.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .