Category: Marketing

One of the more important aspects of designing online market research surveys is the length of the survey. Survey length should be discussed at the beginning of the survey process as the time it takes for respondents to complete the survey can impact sample costs and the fee for survey hosting services.

Many survey companies adopt a general policy of three survey questions equating to one-minute of online research. However, in the field, the timing can be far different depending on the number of words in the survey, the “thinking time” required by respondents, the manner in which the questions are programmed, and the speed of the survey technology. For example, a question group formatted as a radio table with column headings consisting of a 5-point scale and simple rows of questions, containing several words each, can equate to 10 questions a minute. On the other hand, message or image displays with lengthy statements or open-ended comment questions can equate to only one question per minute.

The professional staff at most survey companies should be able to estimate the length prior to survey launch. Online survey companies that offer respondent timers can use those timers during the survey preview period to estimate response times while also using the timers in the field. Keep in mind for cost purposes that the median response time is not necessarily the statistic used for sample pricing. For example, if the median survey completion time is 15 minutes, but 25% of the respondents take 22 minutes or longer, those respondents will expect to receive a survey incentive commensurate with their response times. This could increase the CPI for the entire project.

As a general rule, longer survey length in online research is more expensive in B2B surveys than in consumer market research – – although there are exceptions (such as high net worth households). Keeping B2B market research at 10 minutes or less can often provide a big assist in limiting sample costs. Survey length of 20 minutes or more can dramatically increase costs, with sample incentives rising significantly with each 5-minute increase. Clients should also keep in mind the validity of the survey data as survey length exceeds 20 minutes. It might not be possible to ask every question in a single survey instrument. Survey respondents, like everyone else, can get distracted or become tired after a while and their responses can be less thoughtful.

In summary, the time it takes for respondents to complete a survey can have a significant impact on the cost of online market research – – particularly in B2B surveys as compared to consumer research. Most online market research firms can provide pre-launch advice on survey length, and survey hosting vendors with survey timers can use this tool to help pinpoint the length. In addition, the programming structure of the online market research survey as well as the speed of page views and data inserts can impact survey length. Online market research firms should be able to offer helpful advice on programming options and offer suggestions on best use of skip logic, radio tables, and other features to reduce survey length.

Marc I. Tillman is a member of the professional services staff at Amplitude Research, Inc., a full-service market research firm headquartered in Boca Raton, Florida.

Do you want to drive your business forward? Do you really know what your customers think about your business? In order to ensure that you achieve both of these things you may wish to consider conducting a customer feedback survey. With the help of such a survey not only are you improving your customer retention and profit but you are also advancing satisfaction levels and prompting interest in new products and services that you offer. Basically, with the help of a feedback survey from your customers, you are building up a relationship with them, which will enable you to tailor your business more towards their needs and enhance the chances of increasing your own sales.

A customer feedback survey isn’t just about asking a few simple questions, you need to ensure you focus on the aspects that matter. This includes the underlying purpose of the survey and the best methodology of reaching your customers. Depending on the type of business you run, there are a number of options available to you including mail and telephone surveys. However one of the most effective methods that we are seeing today involves online surveys, or combined online and mail surveys.

Before you actually deploy your survey, make sure you read through it very carefully a number of times, or get someone else to do it as a means of checking that you are asking the right questions and also that you are asking them in the correct way. What this will do is help you not only construct the survey but also ensure that you are getting the best feedback from it.

One of the most important aspects of these surveys is analyzing the answers that you receive. This should actually help you decide what sort of questions to be asking in the first place. For example, closed-ended questions are a lot easier to analyze than open ones, so keep your customer feedback survey simple but informative. It is also helpful when you are analyzing the results to achieve as many completed surveys as possible. So don’t dive in too eager to examine the results. Let you survey gather up responses over a matter of weeks and then collectively analyze these results.

Another great feature that you may not be aware of where customer feedback surveys are concerned is the fact that you can also use them as a means of publicizing aspects of your service that your customers, until now, have been unaware of. So not only are you gathering the information needed to improve your service but you are also promoting your business in the process.

No matter what type of business you run or for that matter what size your business is, you will benefit from the help of a customer feedback survey. It will help you manage your business initiatives and push your business to improve in all areas. Also whether you receive negative or positive feedback it is important that you take the points on board and use them to make any necessary changes. Although it is hard to face up to criticism, what would you rather do, admit you are doing something wrong and fix it or watch your business fail because you did nothing about it? I think we all know the answer to this question and I think this alone is enough to demonstrate just how important customer feedback surveys can actually be.

Marc I. Tillman is a member of the professional services staff at Amplitude Research, Inc., a full-service mail and online survey company headquartered in Boca Raton, Florida. Amplitude is an industry leader in the design, programming, survey hosting, and reporting of customer feedback surveys.

The current economic climate has caused some companies to defer or cancel consumer market research surveys. In many cases, a company may have correctly determined that the cost of completing a survey within the parameters required for actionable data is no longer possible given its current budget. However, defining exactly what those parameters should be is not always clear, and that is what determines the project cost. This article focuses on the concept of incidence rates, which can be one of the most important cost considerations in designing market research surveys.

Pricing Components for B2B and Consumer Market Research Surveys

While online market research companies will each have their own formula for pricing business and consumer surveys, there are certain factors that almost all vendors take into account. These include:

• Survey Length (the longer the questionnaire, the higher the cost)
• Survey Programming and Hosting Services (the more complicated the programming, the higher the cost)
• Respondent Incidence Rates (the lower the incidence, the higher the cost)
• Number of Completed Surveys (the more completes, the higher the cost)

Other potential cost considerations are questionnaire design assistance (generally a low-cost item in relation to the overall cost of the study), and any additional reporting requirements such as banner tabulations, verbatim coding, or reporting (many vendors will include a raw data file, collated verbatim and a top-line frequency report at no charge or a minor charge, but there are many situations in which a customized written report involving significant professional time and additional cost is needed).

Impact of Incidence Rates

The term “incidence rate” refers to the percentage of respondents who qualify to complete a research study. Most vendors incorporate the incidence rate and cost of awards into a quoted cost per complete, referred to as the CPI (“cost per interview”) or CPC (“cost per complete”). The lower the incidence rate, the higher the cost; with more pronounced increases as the incidence drops below 20%. For example, a B2B market research study with a 70% incidence might have a CPI of $7.00; while a study targeting high-level executives with a 3% incidence might have a CPI of $50.00. The CPI is then multiplied by the number of completes to determine the cost of sample for the project. As the number of completes increase, there may be a volume discount in the quoted CPI, but it is generally a small reduction (such as $7.00 for 350 completes and $6.50 for 600 completes).

From a cost perspective, online market research surveys are different from RDD (random digit dialing) phone surveys where the general population is randomly called and the incidence rate is simply the percentage of respondents who qualify. Most survey panels have 100’s of pre-identified information cells (“selects”) of panelists and can target in advance participants who qualify for participation. This can result in significant cost savings, particularly in lower incidence studies.

Yet, when vendors quote an incidence rate for an online study and provide alternative CPIs for different incidence rates (such as a $12.00 CPI for a 25% incidence and a $16.00 CPI for a 15% incidence), the client might not fully understand how the vendor is defining “incidence”. For example, if the targeted select for an online consumer survey is “Owners of 2008 New Ford Mustang Convertibles”, and the vendor has “Owners of Ford Mustangs” pre-identified in its database, in theory the quoted incidence should be based on the percentage of those pre-identified Owners of Ford Mustangs who own a New 2008 Convertible. Yet, some vendors quote incidence based on the percentage of Owners of 2008 New Ford Mustang Convertibles within the general population, which makes little sense in the field since the general population is not targeted to participate in the survey.

SUGGESTION: For lower incidence rate studies, ask the vendor if the targeted select is pre-identified. Keep in mind that some vendors will list a select as being identified not because it is pre-identified, but because the vendor knows it can reach that select through a general population blast to its survey panel. However, the CPI on a pre-identified select is almost always less expensive in a low incidence study because there is less risk to the vendor and no need to blast out emails to a large portion of its database. The variation from vendor to vendor in the CPI could be $30.00 or higher depending on whether the select is pre-identified. Wide variations in quoted CPIs are particularly evident in B2B market research surveys.

Questionnaire Design and Incidence Rates

Selection of the specific criteria for respondent qualification, and the wording on the questionnaire of the associated screening questions, is a critical element of any project, particularly lower incidence studies. Differences in qualification requirements such as “final decision maker” versus “influencer”, “C-level executive” versus “Manager, Director, VP or higher”, or “Owner of 2008 New Ford Mustang Convertible” versus “Owner/Lessee of New or Used 2006-2008 Ford Mustang Convertible”, can often have a significant cost impact. Even a screener such as “How often do you travel out-of-town on business… ” can have very different pricing depending on whether the screening requirement is twice a year or four times a year.

While the goal of a survey should always be to reach the targeted audience that can best provide the desired feedback; there is often some flexibility in defining the screening criteria. Is there a good reason why a 2008 model year is the only qualifier for an auto study? Does a decision to purchase a product or service really need to be limited to a final decision maker and not include input from others in the decision-making process? Those determinations directly impact the incidence rate and can quickly move a project from too expensive to affordability, or vice versa.

SUGGESTION: Review the specific goals of the study with the vendor’s statistician or other questionnaire design expert to determine what options are available for defining respondent criteria that can reduce costs but still produce actionable data. Keep in mind, for example, that the cost difference between surveying “final decision makers” and “final decision makers or influencers” can be as much as $20.00 a complete or higher.

Survey Hosting and Incidence Rate Testing

Some online market research companies provide survey hosting services in addition to survey sample. In such cases, they may offer incidence rate testing at no charge or for a small fee. An “incidence rate test” is a small study using the screening questions only which is conducted in advance of the actual project. With this information, a prediction can be made as to the actual incidence rate and its cost in the field. The client can then determine how many completes will fit within budget, what changes might be needed in respondent qualifications to increase the incidence rate, or if the project still makes sense from a cost perspective.

SUGGESTION: Take advantage of this option if the study is low incidence and the incidence rate is uncertain. In many cases, the incidence rate established in the test might be higher than expected, resulting in a lower project cost. Or, if not, an informed decision can be made on what adjustments are needed in other parts of the project to lower the overall cost.

Working with a full-service market research supplier that is involved in the different phases of the project including questionnaire design, sample, hosting, and analytics brings the parties together with the common goal of providing high-quality, actionable data in a cost-effective manner. Take advantage of the vendor’s expertise in each of these phases and discuss the scope of the project with the vendor’s statistician or other questionnaire design expert. Respondent criteria, respondent targeting, number of completes, and other cost sensitive items are legitimate topics of discussion on which the vendor should be able to provide expert advice. There is rarely only one approach to a survey project.

Marc I. Tillman is a member of the professional services staff at Amplitude Research, Inc., a full-service survey company headquartered in Boca Raton, Florida. Amplitude is an industry leader in conducting business and consumer market research surveys, providing questionnaire design, survey programming, survey hosting, and reporting services using its proprietary software technologies and web survey panels.

Periodic measurement of customer loyalty can be an excellent way to gauge the success or failure of ongoing customer relationship management. Typically, online customer loyalty surveys are administered as part of a program of client satisfaction surveys. This allows for simultaneous measurement of both aspects of the customer relationship, which are in many cases (but not always) interrelated items.

A well-designed client survey will not exceed 10 to 12 minutes in length and ask customers to rate overall satisfaction with the company, provide specific reasons for overall satisfaction or dissatisfaction, inquire as to customer loyalty, and include a number of targeted questions regarding satisfaction with specific products or services and aspect of the customer experience. Developing the final list of customer experience attributes is an important part of the design process and needs to cover the key issues that are potentially related to overall satisfaction and customer loyalty. Most importantly, the attributes should be fine-tuned for the specific product or service offerings of the company and use wording that is easy for the average customer to understand and interpret as intended.

As part of the design process, it is also important to examine where the company offers its products via different channels or has distinct products or service offerings requiring separate sets of questions. In such cases, the use of online customer surveys provide the opportunity to use skip logic so that different customer segments can view different sets of questions based on their specific profiles. These questions should be appear as supplements to the main survey questions, which should be applied across all segments (if possible) to facilitate overall customer satisfaction and loyalty measurement.

In designing customer loyalty surveys, it can also be important to collect information concerning customer demographics, other characteristics, and/or past behavior or usage of the company’s products and services, or the use of similar products and services in the general market place. A key advantage of online customer feedback surveys is that it can often circumvent the need to ask many of these questions on the survey instrument. Instead, the questionnaire can be pre-populated with relevant profiles or the information can be merged with the post-survey data to facilitate reporting analysis. This reduces the length of the survey questionnaire.

At this point, it is important to comment on several key aspects of the survey design process that are often over-looked when conducting a customer loyalty survey. First, correct sample selection is critical. This phase is easy if all customers are invited to participate in the survey. However, if only select customers are invited or the number of available email addresses is limited, sample selection needs to represent the overall customer population or the collected feedback may not correctly represent the overall perspective of the customer population. Furthermore, even if the sample is selected in the correct manner, or all customer sample is used, the mix of respondents may differ from the company’s survey population and distort the results. In the end, statistical techniques might be needed to accurately measure overall customer feedback and provide correct weight to each segment.

A second aspect of customer loyalty and client satisfaction surveys that is often over-looked is the use of existing customer information. This might include specific details on the type of service (such as a special bank transaction), type of product, number of support requests, number of years as a customer, and a long list of other metrics. Not only can this information be used to trigger skip logic on the online survey, but also variable inserts can be used to create respondent-specific questions. For example, “what is your overall rating of the [variable insert] you received on [variable insert] from the Credit Union?”

In summary, effective survey design of online client satisfaction and customer loyalty surveys can be a complex process requiring extensive review of existing customer profiles, identifying key attributes of the customer relationship, scoping channels or segments of product and service offerings, and making sure that both sample selection and the actual respondent pool is representative of the overall customer survey population. Although company budgets can sometimes limit the ability to engage in extensive data analysis of the results, it is not a good idea to move forward with a customer feedback survey without careful questionnaire design and sampling techniques.

Marc Tillman is a member of the professional services staff at Amplitude Research, Inc., headquartered in Boca Raton, Florida. Amplitude Research is one of the top customer survey firms specializing in client satisfaction surveys and customer loyalty surveys.

This Article is Part 3 of a series of articles on the subject of client satisfaction surveys. Part 1 covered the early planning stages including project goals and reporting deliverables. Part 2 examined selection of a survey methodology, with emphasis on the availability and quality of the records list. Part 3 focuses on the use of online customer surveys.

Once it is established that there is an adequate email records list for conducting client satisfaction surveys online, it must be determined whether an online customer survey is still the best choice over other methodologies such as mail or telephone surveys. This decision will often rest on a choice between telephone and online customer surveys, although mail surveys can the best choice in certain cases. The survey vendor should be able to provide helpful advice in making this selection. Key decision factors include project budget and customer relations.

Generally, client satisfaction surveys that focus on b2c relationships have similar pricing for telephone and online customer surveys. These types of surveys might include retail customers or service providers such as accounting, personal banking, auto repair, or mortgage lending. The common thread is a non-enterprise product or service where the end user is typically an easy-to-reach survey participant. For these kinds of client satisfaction surveys, the best choice can often be a telephone survey due to higher response rates.

Cost Considerations and Online Customer Surveys

But where client satisfaction surveys are of a b2b nature (such as enterprise products, services or software), and/or the end user is a corporate executive, online customer surveys can often result in lower pricing than telephone surveys, and similar or better response rates given the difficulty of reaching the respondent by phone. In addition, given the ongoing business relationship with most enterprise products or services, the customer might have a strong interest in timely participation in online customer surveys. In the end, each business relationship is different, and should be examined based on its own set of circumstances.

Customer Relations and Client Satisfaction Survey

Cost issues aside, another key consideration that is often overlooked in designing client satisfaction surveys is the overall impact on client relations. At its most basic level, this means considering the type of methodology most customers are comfortable using. For example, if there are ongoing email exchanges between the company and its customers, online customer surveys might be a natural choice. On the other hand, if customers rarely communicate via email, then blanketing them with email invitations to online customer surveys might not be the best choice even if email addresses are readily available.

Another element that is often overlooked is whether reaching ALL customers should be considered for client relations’ reasons, even though sampling of a portion of the customers is less costly and/or adequate from a statistical standpoint. Put a different way, it can sometimes benefit client relations to ask customers for their feedback on how you are performing. In some cases, such as alumni surveys, it might be politically incorrect to survey only certain alumni to the exclusion others. If that is the case, online customer surveys can often provide the least expensive method of reaching all of the participants while placing your corporate brand in front of them.

Determining the best survey methodology is not always an easy decision. The starting point is typically the availability and quality of the records list. However, this alone is not necessarily the determining factor. Cost considerations and client relations must also be considered. For many client satisfaction surveys, this will boil down to a choice between telephone surveys and online customer surveys. Frequently, the price points are similar when the product or service is delivered b2c instead of b2b. But where enterprise products or difficult-to-reach executives are involved, online customer surveys are often the least expensive option for conducting client satisfaction surveys provided the participants are comfortable responding online. Lastly, and often overlooked, are client relations’ considerations, such as allowing all customers to participate instead of a select group.

Marc Tillman is a member of the professional services staff at Amplitude Research, Inc., a full-service survey company located in Boca Raton, Florida. Amplitude Research specializes in client satisfaction surveys including, mail, telephone, and online customer surveys.

This Article is Part 2 of a series of articles on the subject of customer and client satisfaction surveys. Part 1 covered the early planning stages for customer satisfaction surveys including project goals, reporting deliverables, and initial discussions on survey methodologies.

The next step for an effective customer satisfaction survey is gaining a better understanding of the best survey methodology given the client’s project goals and budgetary requirements. As much time as needed should be spent discussing the characteristics of the target survey population. For example, if the customers are other businesses, what are their specific job titles and/or job functions? What is the typical client revenue size buying the products or services? If buyers are primarily comprised of consumers rather than businesses, what are the specific demographics of the customer – such as age, gender, household income, region of the country, etc.?

Once the customer demographics are fully identified, the most effective survey methodology can be explored. This will require a full and complete understanding of the scope and quality of the customer records. Most clients will have this information at hand and can provide estimates on the availability of email addresses, phone numbers or mailing records, as well as the quality / reliability of each. Others will need to check and get back to you with “best guesses” of list quality. The provided records information can then be examined in concert with the customer characteristics to determine the best approach for gathering feedback.

For example, a client might advise that the records list includes highly accurate phone numbers of almost every customer. However, upon further inquiry, the records do not include the direct phone numbers or extensions of the targeted respondents at each customer location and/or the individual names of the key contacts are unreliable. This can result in significant cost issues for a telephone survey methodology where interviewers face the prospect of “networking” at each business to find the correct person.

A second example might be a large list of email records – which is always good news – with the warning that most of the records are updated sporadically. In this situation, a well-designed approach might begin with an online customer satisfaction survey followed by mail or telephone interviewing. Or, if the client sells to a wide group of customers, it might be possible to reach qualifying participants through a web panel.

In the end, however, most clients maintain lists of customers that include multiple fields of information. This means that budgetary elements, as well as survey sampling issues, often play an important role in selection of the best methodology for conducting a customer satisfaction survey. A skilled research supplier should be able to analyze this information and come up with a plan of action based on the quality and types of records maintained by the client, the availability of web sample as a possible supplement, and the objectives of the survey. The use of multiple survey methodologies can often provide a nice balance between cost and effective sampling in the realm of customer satisfaction surveys and should be considered, as needed, to meet the goals of the client.

Marc I. Tillman is a member of the professional services staff at Amplitude Research, Inc., a full-service mail and online survey company specializing in market research surveys and client satisfaction surveys. Please visit Amplitude’s customer satisfaction surveys page to learn more about its service offerings.