Improving Survey Response Rates: Four Tactics to Increase Participation
More and more organizations are using surveys to find out what their employees, customers, patients, and suppliers think and want. Surveys can provide them with accurate, measurable data that they can’t get any other way. Not only are surveys an excellent way to keep abreast of needs, problems, and the current state of affairs, but the results are essential when making plans for the future.
Low response rates are a continuing problem for surveys. Some people simply refuse to participate in surveys, while others, for a wide range of reasons, cannot participate. Still, a well-designed survey, coupled with incentives and techniques to elicit response, can help guarantee a healthy response rate.
Should you be concerned about low response rates? Absolutely. Low response rates usually do not include a true random respondent sample and may bias survey results. Conclusions drawn from unrepresentative data may be erroneous and cause serious problems when used to plan organizational, product, or marketing outreach changes.
Customers frequently ask us what kind of response rate they can expect for their survey. Given the vast number of variables with the potential to affect response rates, it can be difficult to estimate. We can, however, provide a set of guidelines or tools we believe will enable you to maximize response rates.
Why Do People Participate in Surveys?
Briefly, researchers divide approaches to survey participation into two distinct groupsthe reasoned action approach and the psychological approach.
Reasoned Action Approach
The reasoned action approach relies on the theory of social exchange to explain why someone fills out a survey. Basically, the participant is more likely to participate if the rewards of participation outweigh the costs. Examples of this approach would be strategies such as including monetary incentives for participation or reducing survey length so that the time cost is perceived to be low and non-intrusive.
The use of incentives is a heavily researched area in response rate literature. Although several meta-analyses came to different conclusions, published reviews paint a very clear picture with respect to two issues: first, incentives are effective in increasing the response rates for mail surveys; and second, promised incentives are not as effective as enclosed incentives. Numerous studies demonstrate that postpaid incentives have no impact on response rates. While these two findings are almost universal, the effects of incentive size are less clear.
Unfortunately, some of the evidence on the role of incentive value is conflicting. The bulk of the data suggest that there is some merit in increasing the value of the incentive; however, this issue is far from settled.
Incentives use social exchange theory by causing participants to feel obligated to respond. However, some researchers suggest that providing large incentives is a model of diminishing returns. A small token may cause potential respondents to feel obligated to respond. But an incentive cannot be so large as to suggest payment for services rendered.
The psychological approach relies on heuristic factors such as reciprocity, helping, compliance, and selectivity as a way to increase response. Reciprocity is the behavioral norm that says people should treat others as they have been treated. This is the approach used when a dollar bill is included in a survey. If a benefit has been provided to the participant, some sort of reciprocal benefit should be in turn provided. The helping approach uses a strategy of specifically requesting help, for example in a cover letter, as a way to compel participation. The compliance approach focuses on increasing participation by using an authoritative organization like the government as the sponsor rather than an anonymous source. Finally, using selectivity, or emphasizing the participant as being part of an exclusive group has also been used to increase participation.
Salience is simply how important or relevant a survey topic is to the survey recipient. Unfortunately, survey salience may be out of the researcher’s hands and dictated by project needs; nevertheless, it is useful to understand the effects of salience when designing both surveys and cover messages. If viewed as important to the respondent, response has been shown to rise between 12 and 14 percent.
Salience is an important factor in respondent behavior; unfortunately, it is also one aspect of a survey that is difficult to alter. At a minimum, salience should be emphasized in the messages accompanying a survey.
Requests for Help
If people tend to follow a norm of social responsibility, they may be more likely to comply with a survey request couched in terms of asking for help. Some evidence indicates that this is indeed the case. A recent study found an 18 percentage point increase by including the phrase “it would really help us out” in communications.
Putting It into Action: Four Tactics to Increase Participation
Now that we understand why people are willing to take surveys, let’s explore how to get as much of your target audience to participate as possible.
1. Choose an Appropriate Survey Length for Your Audience
Many researchers view a survey that is too long as an inhibitor to response, because longer surveys take more time to complete and thus increase the costs to the respondent. Respondents may fill out only part of the survey, or they may reject very long surveys outright.
In general, the experimental research on mail surveys indicates that shorter surveys do elicit higher response rates, but many of the differences are small. How long is too long? Generally, surveys that take longer than 15 minutes to complete are considered “too long” in most instances.
2. Make Sure the Survey Is Easy to Take and Return
One of the easiest ways to increase response rates in electronic, phone, and paper formats is thoughtful design. A well-designed, attractive survey that is easy to complete will improve response rates as well as data accuracy. In general, by making surveys easy to complete, you increase the likelihood that respondents will participate.Many institutions that conduct frequent or extensive surveys get caught in the trap of making survey administration easy for the organization and overlooking the needs of the respondent.
||…instead of this…
- Choose the most effective and reliable delivery method to reach participants (see Choose the Right Survey Format later in this article for recommendations).
- Include at least a personalized greeting on the survey form.
- Reduce the number of questions asked.
- Vary the delivery formats to capture interest.
- Keep telephone survey questions short and to the point.
- Sending only printed and mailed questionnaires by bulk rate to save money.
- Neglecting to personalize mailings.
- Squeezing more questions onto a page to save printing costs.
- Using identical formatting in every mailing to save mailer costs.
- Designing long-winded questions for telephone surveys.
To increase participation, avoid falling into the traps listed above. It may make it easier for you to process the responses, but making it harder for your participants defeats the purpose of your survey—to gather enough appropriate information to make effective decisions. Design every survey to make the respondents’ job of completing it easier.
3. Contact Participants Multiple Times
One of the most successful techniques to increase response rates is the use of multiple contacts with members of the sample. This technique is now considered standard methodology for any survey.
Studies suggest that to get full benefit from multiple contacts, do the following:
- Use a pre-survey notification message.
- Follow the pre-notification with a copy of the survey including a cover message.
- Contact non-respondents using combination of messages and surveys.
Studies using samples of the general population found that pre-notification letters or emails increased response rates by 4 to 29 percentage points. Reminder postcards or emails are also effective and have been shown to increase response rates from 3 to 8 percent.
Research also shows that increasing the number of surveys sent to respondents increases response rates. Researchers found that sending a fourth survey increased the response rate by more than 30 percent. One study showed that two survey mailings versus one survey mailing increased response rates 12 to 20 percentage points.
A recent example of multiple contacts with a web survey was administered at a major university regarding student housing. After the first e-mail notification, the response rate was leveling off at around 44 percent. After an e-mail reminder was sent to non-respondents, the response rate increased to 67 percent, and a final reminder to non-respondents notifying them of the deadline for the survey resulted in a final response rate of almost 72 percent, substantially higher than the rate after the first e-mail notification.
While multiple contacts can increase costs, re-contacting respondents is one of the best ways to ensure a good response rate. This is one reason that web surveys are growing in popularity: three or four contacts with respondents can be costless, while three or four paper mailings can be quite expensive, especially if postage is required.
4. Choose the Right Delivery Method
The right delivery method can make or break a survey’s response rate. Knowing your audience and their capabilities and preferences has a strong effect on response rates. The following sections discuss several common survey delivery methods.
But First: A Note about Using Multiple Survey Methods
Survey professionals have long recognized that some respondents prefer being surveyed by one mode, while others prefer another. A recent survey reported that among respondents to a telephone survey, 39.4 percent indicated that they preferred being surveyed by telephone, 22.7 percent preferred face-to-face interviews, and 28.1 percent preferred mail or email.
Many organizations are not only offering multiple-modes at the start of a survey, but are choosing an alternative mode to surveys for non-responders. Both approaches make sense when you consider that about 80 percent of the US population has access to the Internet (US Department of Commerce, 2011), over 96 percent of homes have a telephone, and many respondents are difficult to locate by mail due to moving or socio-economic status. No one method is guaranteed to reach everyone.
In a study of female veterinarians, the surveyors employed a mixed-mode survey design in targeting women graduating from all US veterinary colleges during an 11-year period (1970-80). The questionnaire elicited information on a variety of health and occupational factors and required 35 minutes on average to complete.
In the first stage a mailed, self-administered questionnaires were employed, yielding a response rate of 82.9 percent. In the second stage, a telephone interview of all mail non-respondents was attempted, yielding a response rate of only 30.1 percent, but increasing the overall response rate among those contacted to 90.2 percent. The ultimate response rate, using both methods, resulted in highly reliable data.
Web surveys can offer several advantages, such as shorter administration time, lower costs, and fewer data-entry errors. Yet some researchers question the validity of data obtained from web surveys; a common finding, for example, is that responses from web surveys tend to show more positive outcomes for computer- and technology-related items.
- Fast. Many people who will respond to an email invitation to take a web survey will do so the first day, and most will do so within a few days.
- Logos and other specialized graphics, fonts and multimedia options are available.
- You can adjust which questions participants see based on their responses to certain questions.
- Reduced data entry costs.
- Evidence suggests you will get more accurate answers to sensitive questions.
- Many people dislike unsolicited email even more than unsolicited regular mail. Be sure to adhere to anti-spam regulations.
- As with email, web surveys do not reflect the population as a whole.
- People can easily abandon in the middle of a questionnaire.
- As with mail surveys, electronic interviews may have serious response rate problems in populations of lower educational and general or computer literacy levels.
The most important factor to keep in mind with respect to web surveys is that a web survey will be successful only if the population has easy access to the Internet and is comfortable with using the web, and if the survey administrator has an accurate e-mail address datafile.
To increase online survey response rates:
- Keep it simple and user friendly. Make the first page simple in order to get the survey started easily.
- Make participation voluntary, anonymous, and confidential.
- Be relevant. Ensure that the survey topics and questions are of interest to the participants.
- Develop a value proposition for participants.
- Communicate in advance. Alert participants that a survey is coming.
- Communicate aggressively to track responses and send reminders to stimulate participation.
- Use graphics sparingly and strategically. Surveys with extensive graphical treatments have lower response rates than plain surveys.
- When possible, publish your results online to participants.
Another important aspect to consider when administering online surveys is access. Surveys that require the respondent to key an identification number and password into the survey have lower response rates than surveys that automatically log in the respondent.
Paper surveys are a tried and true format, and can reach a larger sample than the electronic format.
- Mail surveys are among the least expensive, compared to phone surveys.
- The questionnaire can include diagrams, graphics, etc.
- Mail surveys allow the respondent to answer at their leisure and are not considered as intrusive as other kinds of interviews.
- High accuracy available when forms are scanned instead of manually entered.
- Response time is usually longer than other methods.
- Response rates are often low or are unpredictable and may result in biased results.
Critical Activities and Inserts for Paper Surveys
For the best response rates, consider the following activities while planning a paper survey:
- Pre-Notification and Commitment Cards
Pre-notification usually involves a postcard or letter explaining the impending survey delivery and requesting participation. Commitment cards ask the potential respondent to return a postcard indicating they will participate in the survey. Results of a recent study indicate that pre-notification cards provided a response rate of 32.3 percent while the commitment card resulted in a response rate of 20.4 percent.
- Reminder Postcards
Reminder postcards are sent to survey recipients roughly one week after the initial survey mailing. This card serves as a thank-you for those who responded and a reminder for those who haven’t. Our experience with these cards suggests they are very helpful. Findings range from a 10 percent increase to a response nearly equaling that of the initial mailing.
- Return Postage
Rather than requiring the respondent to provide their own postage, return postage has been shown to increase response rates significantly. In addition, using stamps on return envelopes has been shown to be better than using business reply mail. However, depending on volume, the incremental return gained by stamping the return envelopes may be more than outweighed by the cost.
- Cover Letter
A strong cover letter is also a key motivator for respondents. It should explain the importance of the survey and the potential benefits to the respondent for the most direct and influential approach.
- Re-Mail Surveys
Re-mailing surveys entails distributing a second survey to either the entire respondent base or to non-responders only. Two studies provide considerable evidence of the benefits of using this approach. In one, a re-mail improved response rates from 39.5 to 50.0 percent, and the other showed an increase from 38.8 to 52.9 percent.
Types of Mail
Survey professionals have found that what type of mail you use to distribute and receive the survey has an impact on response rates. Research has examined both different mail types and postage types with results indicating they do have an impact on response rates (although there are frequently other variables involved).
- Certified Mail™ seems to be helpful for a lengthy survey. In a study on the impact of Certified Mail, varying questionnaire lengths were tested in combination with a certified mail process. The results indicated certified mail had little effect with short surveys. However, the response rate for certified mail was nearly doubles that of regular mail for long surveys.
- Express Mail® has also been found to positively affect response rates, especially for executives and business respondents. A recent study demonstrated that when Express Mail was used for both the mailing and return of the survey a response rate of 52 percent was achieved, compared to a response rate of 26 percent for the group receiving and returning their survey via regular US Mail. Express Mail may also convey the importance of the survey to the potential respondent.
- Critical Mail™ is a variety of Priority Mail that ensures mailed material gets to the intended recipient quickly via expedited delivery and a standard “signature required” delivery option. Studies have shown that when participants feel “invited” to respond, their active participation increases.
NOTE: Although there are no direct statistics available, our experience shows that sending surveys via pre-sorted bulk rate results in a lower response. Recipients do not view mail received via bulk rate as a personal invitation, but as “junk mail.”
Surveying by telephone is a very popular method, made possible by nearly universal phone coverage (96 percent of homes have a telephone, as cited earlier). Telephone surveys can capture nuances by permitting open-ended questions. This medium is powerful because it permits follow-up questions, but it’s expensive, and the interviewer’s skill is critical for success. Inexperienced interviewers can bias the results, commit transcription errors, and anger respondents.
- People can usually be contacted faster over the telephone than with other methods.
- Skilled interviewers can often elicit longer or more complete answers and can also ask for clarification of unclear responses.
- 96 percent of homes have a telephone
- Many people are reluctant to answer phone interviews and use their answering machines to screen calls.
- A limited calling “window” of about 6-9 p.m. exists for many households (when you can be sure not to interrupt dinner or a favorite TV program).
- Product samples or demonstrations are not an option over the telephone.
To increase response rates with telephone surveys
Spanish is the most common language spoken by non-English speakers in the US. These percentages vary a great deal by state, with a high percentage of the population in states like California and Texas speaking Spanish at home and a much smaller percentage in states like South Carolina. These variations imply that surveys may have to be prepared to locate and interview respondents in languages other than English. Moreover, language barriers are greater among low-income households, and low-income households are more likely to be isolated linguistically (where no one in the household speaks English).
The need for bilingual staff as well as Spanish versions (and perhaps other languages) of all questionnaires and materials is critical, particularly in some states.
- Interviewer Materials and Training
Interviewer experience can be crucial to obtaining high respondent cooperation. The theory is that experience makes interviewers familiar with many questions reluctant respondents may have about cooperating and allows them to respond in a quick and confident manner. Showing any type of hesitation or lack of confidence is correlated with high refusal rates. Conversely, being too aggressive can be seen as “bullying”; interviewers need to seek a respectful middle position.
Intensely training your interviewers on how to handle reluctant respondents may provide them with increased confidence, as well as the necessary skills, to survive difficult situations. A recent study shows significant improvement in cooperation rates once interviewers are provided with detailed training on how to handle reluctant respondents. This training consisted of drilling interviewers, through a series of role plays, on providing quick responses to respondent concerns about participating in the study.
The interviewers must also have the skills and knowledge to answer the subject’s questions, to overcome objections, and to establish the necessary rapport to conduct the interview. Training in these areas is crucial if you want to avoid refusals. You should prepare answers to frequently asked questions (FAQs) and make interviewers practice so that the answers sound natural, rather than like a script is being read.
You must also train interviewers to know when to accept a refusal, while leaving the door open for future conversion by a different interviewer who might have more success.
Telephone surveys can occur any time during the day, but the incredible growth of the telemarketing industry has led many people to screen their calls, especially during the dinner hour. Avoid calling on holidays, at meal-times, on weekdays, or during other times when respondents may be unwilling to participate.
A survey must have a good response rate in order to produce accurate, useful results. We have outlined several ways you can improve your response rate. The response rate for a survey is critical and you shouldn’t just leave it to chance. If you care about the data, you need to care about getting the best response rate possible.
How Can Scantron Help?
Scantron has 40 years of experience in building solutions that allow companies to measure various areas of their organizations—customer satisfaction, employee training and enrollment, and quality initiatives. We can provide all of the pieces of your program, using our flexible survey application: SurveyTracker Plus.
To be successful, you must consider and manage the distinct aspects and phases of a survey:
An intuitive and effectively designed survey is a critical building block for success. Well-designed surveys, regardless of format, help increase response rates, minimize survey costs and improve data accuracy. You can design surveys for the web with e-mail invitations, on scannable forms and as kiosk campaigns. The goal is to ensure that, regardless of collection method, you can consolidate the responses into a common database.
When administering a survey, flexibility is key to that survey’s effectiveness and efficiency. Surveys are most effective when they’re administered using multiple distribution methods. In other words, there's no single delivery method that serves as a magic bullet to produce high response rates. For real data with real meaning, a survey must reach the audience in a format that is both convenient to your responders and cost-intelligent to your company.
- Data Collection
Data collection is one of the most complex aspects of survey administration. Gathering responses from survey participants can appear daunting to even the most seasoned professionals. Participants may be responding by paper, kiosk, disc or web. All of this data has to be collected—quickly and accurately—into a common database for reporting and analysis. Scantron recognizes this challenge, and offers the tools necessary to make the data collection phase as straightforward and reliable as possible, while saving time and money otherwise spent on data entry personnel.
- Reporting & Analysis
Surveys are conducted for the response data. If you’re like most survey professionals, you inevitably want to look at data from several angles. Being able to filter the data, then display it in a variety of clear and informative reports allows for more thorough analysis. Scantron understands that the more quickly you can put reports in the hands of an organization's influencers and decision makers, the more quickly change can be affected.
Further, Scantron provides a full-service print and mailshop that can produce and send your surveys. Our printed forms are guaranteed to scan, and our mailshop offers a variety of services including pre-sort and USPS Critical Mail™ to ensure your surveys are delivered to the right recipient at the right time. We can also help you design the form and scan the responses, freeing your staff to focus on the survey results.
With Scantron, you can choose exactly the package of software, hardware, services, and forms to give you a total survey solution. We provide you with powerful functionality, depth, and breadth—notable attributes in the survey industry.
Contact us today to find out how the power of Scantron SurveyTracker Plus can help!
Back to SurveyTracker Plus
© 2014 Scantron Corporation. All rights reserved.