What IP Attorneys Should Know About Expectations and Costs for Survey Research
TASA ID: 961
In likelihood of confusion, trade dress, and secondary meaning cases, I am often called upon to develop surveys based on personal interviews at shopping malls, research centers, or via telephone. Most attorneys who have never personally been involved in such research generally ask a number of key questions such as:
1. How many interviews or responses are needed for significant results in such IP litigation surveys?
2. How many research venues are needed?
3. How do you gauge the effectiveness of results?
4. What can my client expect to pay for this research?
How Many Interviews And Venues
While different circuits rely on different precedents, a good rule of thumb is a minimum of between 200 and 300 interviews. Fred W. Morgan, presently a professor of marketing at the University of Kentucky, analyzed sample size in his article entitled "Judicial Standards for Survey Research: An Update and Guidelines," in the Journal of Marketing when he was a professor at Wayne State University.
Sample Size must be intuitively justifiable. In addition to concerns about the effect of sample size on confidence intervals, survey researchers must be able to justify sample with non-statistical arguments. InKentucky v. Alabama, telephone sample of 100 persons was ruled insufficient to determine whether the trial atmosphere within a city of 60,000 was prejudicial to Kentucky. The court stated no reasons for deeming the sample too small, but was simply unwilling to accept the notion that a random sample of 100 persons could adequately represent the views of 60,000 persons. Though sample size can always be criticized, minimum samples of 200 to 300 respondents seem to achieve a certain amount of face validity in the courts (Fred W. Morgan, "Judicial Standards for Survey Research: An Update and Guidelines," Journal of Marketing, Vol. 54 (January 1990) pp. 59-70).
In terms of "margin of error," a sample of 200 (to the 95 percent level of confidence) has an error factor of plus or minus 7 percent, while a sample of 300 has an error factor of 6 percent. Error factors change dramatically with smaller samples. For example, a sample size of 50 has a 14 percent error factor, and a sample size of 100 has a 10 percent error factor. On the other hand, beyond 300 respondents there is a "sliding scale" in effect. A sample size of 400 has a 5 percent error factor; a sample size of 500 has a 4 percent error factor. You need a sample size of 800 to achieve a 3 percent error factor, and you need 2,000 respondents to obtain a 2 percent error factor.
As for the number of venues, J. Thomas McCarthy, a recognized expert on the subject of trademarks, points out in McCarthy on Trademarks that four venues are usually required: "It has been said that an informal rule of thumb is that four testing sites are a minimum number to ensure a reasonable degree of projection to universe of a larger area" (J. Thomas McCarthy, McCarthy on Trademarks and Unfair Competition, 4thedition, West Group, St. Paul, MN, 1996).
Percentages Needed for Significance
McCarthy also cites quite a bit of information on the percentages needed to establish significance. For example:
The courts have been unwilling to be pinned down as to whether an"appreciable" number of customers is to be measured quantitatively (in percentage figures) or quantitatively (by the actual number of persons). The distinction may be crucial where a survey shows a low percentage but can be extrapolated over a large relevant "universe" of potential customers. For example, one court indicated that even 11 percent of a national market of millions of consumers constitutes a very large number of confused consumers (McCarthy on Trademarks, 6/04, p. 32-310).
What is considered a large or significant percentage of measured consumers? McCarthy writes, "Percentages over 50 percent are usually viewed as persuasive evidence of likely confusion, with the Seventh Court remarking that a result of over 50 percent is strongly probative of a likelihood of confusion as 'far in excess' of the needed figure"(McCarthy on Trademarks, p. 32-315).
McCarthy continues with his analysis:
Generally, figures in the range of 25 percent to 50 percent have been nviewed as solid support for a finding of likelihood of confusion. The Ninth Circuit has said that a survey showing a 27.7% level of confusion is alone sufficient evidence to prevent a summary judgment that there is no likelihood of confusion....the Second Court found that a 15-20 percent rate corroborates a finding of likely confusion (McCarthy on Trademarks, 6/04, pp. 32-326-327).
On the other hand, there are results that courts have considered to be too low. When the percentage results of a confusion survey dip below 10 percent, they can become evidence which will indicate confusion is not likely. The Seventh Circuit reviewing prior cases involving low percentage results found that 7.6 percent is "a factor weighing against the infringement." Similarly low percentage figures have been relied upon to support a finding of no likelihood of confusion and no infringement (McCarthy on Trademarks, 6/04, p. 32-318).
What Should A Survey Cost?
The cost of doing a survey is a function of a number of things. The paramount factor is to make sure you are interviewing members of the relevant universe. For example, if the product in question is a package good available on supermarket or drug store shelves, one can safely develop a questionnaire directed to members of the general population. The shopping mall is the ideal venue for such research, and it can be done relatively quickly and inexpensively.
A Word of Caution:
Cost estimates expressed by this author are based on his personal experience. Actual costs may vary depending on who is selected as the survey expert to develop and manage the survey, the specific problem or opportunity being studied, and geographic locations. Estimates referenced here are not meant to be quotations of the costs that the expert charges through TASA. Instead, these are averages ranges for marketing survey costs.
If the product is a specialty good targeted to specific consumers, a fast and relatively inexpensive telephone interview protocol can be employed.
On the other hand, if the product in question is targeted to a specific market segment and a face-to-face interview is required, the researcher and his/her research firm will be forced to recruit appropriate members of this target and to make arrangements for them to come into a research center at a specific time. Targeting this type of subject can be time-consuming and expensive.
In other cases, the face-to-face in-person interview is not feasible. Here, creative solutions that involve scheduling times for an Internet-type protocol or the air-expressing of materials that the subject will review while on the telephone with an interviewer will have to be employed.
Costs Of Doing Consumer Products Mall Surveys
Here recruiters from research companies within the mall screen and recruit suitable subjects. When they enter the mall research center, they are signed in and taken to a room where the interview takes place. There is generally some small incentive involved. Depending on the length of the interview and who is being targeted and the time constraints, a good rule of thumb is $5.00. Most mall research centers will charge anywhere from $20 to $30 per interview in addition to the incentive fee. So, an attorney can figure a four- or five-venue survey of 250 people at from $6,250 to $8,750. In addition, there is the cost of the research professional. When I contract to perform such studies, I like to be on hand to make sure that all the protocols are being followed. I observe the recruiting and the interviewing so that when I issue a report, I can do so with the assurance that I witnessed the execution. You can count on daily fees for a research/survey professional at anywhere from $3,000 to $5,000 per day plus the travel-related expenses that usually run about $500 per venue.
Totaling this up, one can safely figure a four-or-five venue mall study encompassing 250 participants to run from $20,250 to $30,750. Add to these costs the time involved in creating the questionnaire and tabulating and reporting on the results. Most questionnaires can be developed in approximately 10 hours of the expert's time ($3,000 to $5,000). Tabulation costs will run around $1,000. The cost of developing the expert's report will be in the range of 20 hours ($6,000 to $10,000.)
Totaling all these variables, one can figure the costs of creating, executing and reporting on a mall survey to be from $31,000 to $46,000 depending primarily on the hourly rate of the survey expert.
Costs Of Doing Consumer Products Telephone Surveys
This is perhaps the least expensive research alternative. Research companies operate call centers where they can interview randomly-selected consumers or consumers whose phone numbers and/or addresses are generated by mailing lists. There is no incentive needed, and a good rule of thumb is $20 per call. There is no need for the survey expert to be present at the call centers.
Therefore, one can figure a telephone survey of 250 to cost $5,000 plus the cost of the expert creating the survey ($3,000 to $5,000), the tabulations ($1,000) and the cost of developing the report ($6,000 to $10,000). Thus the total cost for this survey alternative will range from $15,000 to $21,000, depending on the hourly rate of the survey expert.
Specialty Surveys
Occasionally, I am called upon to develop surveys where specific respondents have to be specially recruited. For example, in doing a survey for a law firm representing a power tools company, it was necessary to interview professional tradesmen. The problem here is that these people are not available during the workday, so the interviews had to be scheduled on weekends or evenings during the week. Recruiting these people is time-consuming, and a rather large incentive is needed to bring them into the research center. The costs for recruiting can run from $30 to $40, and the incentive costs can run from $40 to $60.
All the other costs would be the same as for the mall study. So, one can figure for a four-or-five venue study of 250 participants, where members of a specific target market are brought into a research center, the costs can range from $17,500 to $25,000 for the basic threshold research costs. Adding the additional expert daily fees, travel expenses, plus the preparation of the questionnaire, tabulations and developing the report, the total cost of such a survey can range from $41,500 to $63,000.
In the final scenario, when you have to do interviews and cannot bring people into a research facility, there is no basic way to figure costs. The incentives would be a function of the target market. For example, if you were to survey heart surgeons, what kind of incentive would you provide? It probably would have to be a substantial donation to a charity. Then, you would have to figure a way to communicate - especially if you want to ask specific questions about some products or brand names. One way would be through a customized Web site that the participant could access only when instructed by the interviewer on the phone. Another way would be through an air-expressed package, which the respondent would be instructed not to open in advance of the interview. These scenarios become extremely complex and costly.
Final Concerns
Even the least expensive of these alternatives is costly. Depending on the nature of the litigation and the characteristics of the target market, the costs can be enormous. Before embarking on any of these journeys, the smart IP attorney will do an inexpensive local test to determine (1) the merits of the questionnaire; (2) the time and expenses that will be required to recruit respondents and (3) most importantly, the approximate results that the survey can be expected to generate.
This article discusses issues of general interest and does not give any specific legal or business advice pertaining to any specific circumstances. Before acting upon any of its information, you should obtain appropriate advice from a lawyer or other qualified professional.
This article may not be duplicated, altered, distributed, saved, incorporated into another document or website, or otherwise modified without the permission of TASA.