If like me, you have previously worked in consumer research, in the first few weeks of working in the pharmaceutical industry, you may well have raised the odd eyebrow when presented with a quantitative sample size of 30 respondents. And whilst you may have initially scoffed at such a paltry base size, after only a few months you will find yourself actively defending these sample sizes and bandying about terms such as ‘robust’, ‘statistically significant’ and ‘conclusive evidence’. So where does that leave us? Should we be worried about researching with such small base sizes? Or is this just a fact of life in pharmaceutical market research? Firstly, I firmly believe that we should always aim for the largest achievable sample size. Recent projects we have conducted have included sample sizes of over 1000 respondents, and with this number of respondents we can all feel confident that the results we obtain deserve the moniker ‘robust’. A larger sample size provides us with a much richer data set within which we can explore subgroups, test hypotheses and run more complex statistical analysis. Where we struggle is when researching small population sizes from which it is neither practical nor appropriate to draw a large sample of respondents. This is when we are forced to take a more pragmatic approach. Let’s face it, budgetary and time constraints nearly always impact upon the feasibility of a quantitative research study. But does that mean we are just making do? No, I don’t believe it does. However, there are a number of pitfalls which we need to avoid. Over-analysing and over-interpreting the data One of the key considerations when working with a small sample is simply to remind yourself of its limitations. Complex statistical analysis may not be appropriate and researchers need to remember that subgroup analysis can be misleading. This doesn’t mean that you can’t interrogate the data, but it does mean that you need to approach the analysis with a critical mind. Analysing subgroups of less than 30 respondents is clearly suboptimal, so if you find yourself doing so, think again. Indeed, it is all too easy to fall into the trap of analysing a small sample in exactly the same way as you would approach a larger sample. It is either negligent or disingenuous to gloss over the fact that we are dealing with low numbers so we need to ensure that our debrief document gives a fair and honest representation of the data. Over reliance of statistical testing Statistical significance tests provide us with an essential tool for analysing market research data. Without these tests, we would need to rely solely on our own judgement – a definite risk. However, there is a tendency to rely on significance testing above all else, and this is particularly problematic when dealing with a small base size. Yes, significance testing provides us with a great deal of reassurance that our findings are genuine. The issue comes when we rely solely on statistical tests and fail to take note of the overall pattern of response. Even with a small sample size, statistical tests will undoubtedly pick up some differences, but how do we know that these differences are real? In my opinion, we know this only by looking at the data set as a whole. Checking the consistency of response and themes within the data is a good start. When a small sample is exactly what you need Another factor we need to consider is the overall population you are researching. Within small populations, you would expect a proportionately small sample size. For example, a very large sample size of 10,000 respondents amongst the total UK population accounts for only about 0.16% of the population. Compare this to a sample size of 100 GPs in the UK, which accounts for almost 2.5% of the population. This starts to put the numbers into perspective. In addition, even with a very small base of respondents, the degree of consensus is important in defining how confident we can be in the results. If 18 out of 20 people agree on a particular point, this is a powerful result regardless of the small number of people questioned. Overall, in pharmaceutical market research small sample sizes are unavoidable but worthy of consideration nonetheless. The views of even a few people add to our knowledge and understanding and help us to make sound business decisions. What we often forget, is that small sample sizes need to be handled differently. Respect the statistics but don’t overplay them, be honest in the way you present data and don’t try to hide the fact that the data has limitations. Clients respect our opinions and expertise, so a discussion around the sample size is entirely appropriate. Apply Now!