In the age of ubiquitous cellphones and disappearing landlines, is it still possible to do quality research using telephone surveys?

For more than 40 years, the telephone has been the primary data collection instrument at Nielsen Scarborough, and it still is in many local markets. With cellphone-only households in the United States approaching 50%, according to the U.S. Centers for Disease Control, we may have reached the tipping point of the viability of telephone interviewing for general market, consumer behaviour surveys.

At Nielsen Scarborough, we believe that, to accurately represent the populations in our local-market surveys, it was necessary to explore data collection methods beyond the telephone interview.

For many years, we have employed a number of procedures to include cellphone-only adults within our telephone samples — and we still do. But it is very challenging to get people to agree to participate in a survey on their cellphones, much less complete the survey accurately. Due to the mobile nature of cellphones, respondents are not always in the best environment for completing surveys.

In addition, cellphone-only adults are more likely to be younger, Hispanic, and have lower-than-average annual household incomes. To better represent these groups within our samples, we recognised the need to develop a better way to reach respondents without landlines.

Beginning around 2011, we started testing an alternative methodology that utilised online and mailed surveys instead of telephone interviews. The goals of our tests were to evaluate the viability of the methodology for data collection, while maintaining the accuracy of the data collected and preserving the comparability of the results to surveys conducted using our legacy methodology.

The result was a new methodology that we call “eFirst,” which we began implementing in our syndicated studies starting in 2014.

The eFirst methodology starts with an addressed based sample (ABS) frame instead of telephone numbers. The ABS sample ensures the largest possible frame because it includes landline households with both listed and unlisted telephone numbers, cellphone-only households, and households with no phone.

We send a mailer to these households with an invitation to participate in an online survey. If the household does not respond to the invitation to participate in the online survey, a paper survey with an incentive is mailed to the household.

We implemented the eFirst methodology in five of our syndicated markets in 2014 and experienced remarkable results. In addition to improvements in response and completion rates, the representation of younger adults within our samples improved dramatically.

Using an index of 100 to indicate the best possible representation of younger adults within our samples compared to the market as a whole, we have seen our indexes move closer to 100 for adults ages 18-24 — both men and women — compared to our legacy methodology. By better representing younger adults within the sample, less sample balancing is required on the back end.

Because the ability to analyse trends is so important, another key goal of eFirst is to preserve the comparability of the results to surveys conducted using our legacy methodology. We examined hundreds of data points, including the size and composition of local media audiences, and found that, in most instances, there was little to no variation in the trends between the legacy methodology and the introduction of the eFirst methodology.

As of today, we have implemented the eFirst methodology in 19 markets, with plans to include 58 more markets by the end of 2018.

Because eFirst is a significant change in the way we collect data, our implementation process has been cautious and deliberate. We pause after each implementation and carefully review the results before moving on to the next implementation. We are also continuing to test the methodology in larger markets and markets with large Hispanic populations so we can modify the methodology as necessary to adapt to different market types.

So far, the eFirst methodology has performed very well against our goals. As technology advances and consumer behaviour changes, our methods of collecting information have to change as well, but changes in survey methodology must be thoroughly tested and evaluated before implementation.

Otherwise, the results of change may be ones you never intended.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .