Conducting mixed-methods research has become very popular over the past decade especially in the health research field.^1-4 This development ties in with the growth in inter-disciplinary and multi-disciplinary research. Many grant applications, PhD project and the resulting papers especially in the health field apply a mixed-methods approach, where in the past a single approach would have dominated.   This interest in combining methods seems to be the case even in the more traditional quantitative field of clinical effectiveness and randomised controlled trials. Whilst I find this development encouraging as a mixed-methods social scientist, it also makes me wonder whether the applicants putting forward a mixed-methods project have thought about the disadvantages or at least the opportunity costs of using such approach.

A mixed-methods approach is ‘simply’ combining two or more research methods to address a research question, i.e. what the label suggests. It is often perceived as the combining of qualitative with quantitative methods, but it can technically also be a mix of quantitative methods or a combination of qualitative methods. The advantage of a mixed-methods approach is that the different methods in the mix address different aspects of the research question and that combining these methods offers a synergetic effect. So what are the possible limitations of or barriers to mixed-methods research?

First, using a mixed-methods approach means you need an understanding of two different philosophies and how to bring the findings of these two different methods together.^4-6   One requires expertise in two different research approaches, either as individual or in the team as well as someone who can do the combining of the findings. For the latter you really need someone in the team who understand the pragmatic approach commonly used in mixed-methods approaches. Otherwise there is a great risk that the original mixed-methods study will be analysed and reported as two or more separate papers each based on data from one of the methods applied in the mixed-methods study.

Secondly, you can spend your money only once, hence there are opportunity costs. Thus if the maximum grant is £200,000 or £300,000 you can’t spend the full amount on the designing a large-scale quantitative study/survey, as you need to spend a proportion of your money and your attention and time on your qualitative study.

Thirdly, and related the above, both quantitative and qualitative methods have ‘rules’ about sampling and sample-size.⁵ Just because you have two methods this does not mean you can necessarily do a study with a smaller sample. The sample size calculations will still say you need at least xxx participants. Similarly, although perhaps not so rigidly you need a certain number of interviews or focus groups to do you qualitative study appropriately.

Fourthly, a common mistake seems to be to add a bit of qualitative research to a larger quantitative study, perhaps a bit tokenistic.⁷ Often it is so obvious in a grant application that the qualitative research is an add-on, an afterthought perhaps from a reviewer in the previous failed grant application.

Finally, not all mixed-methods studies are the same, in fact each mixed-methods study is more or less unique in the way in the way it mixes and matched individual research methods.³ So although mixed-methods may be the best way to address a particular research question, your particular proposed mixed of quantitative and qualitative research might not be the most appropriate to answer the overall research question.⁸

As with all research methods and research proposals my recommendation is if in doubt go and find an expert for advice.⁶ If necessary get an expert on your team of researchers to strengthen your application.

Professor Edwin van Teijlingen

CMMPH

References:

1.  Barbour, R.S. (1999) The case of combining qualitative and quantitative approaches in health services research. Journal of Health Services Research Policy, 4(1): 39-43.
2. Simkhada, P., van Teijlingen, E., Wasti, S.P., Sathian, B. (2014) Mixed-methods approaches in health research in Nepal, Nepal Journal of Epidemiology 4(5): 415-416.
3. Plano Clark, V.L., Anderson, N., Wertz, J.A., Zhou, Y., Schumacher, K., Miaskowski, C. (2015) Conceptualizing Longitudinal Mixed Methods Designs: A Methodological Review of Health Sciences Research, Journal of Mixed Methods Research, 9: 297-319.
4. MacKenzie Bryers, H., van Teijlingen, E. Pitchforth, E. (2014) Advocating mixed-methods approaches in health research, Nepal Journal of Epidemiology 4(5): 417-422. http://www.nepjol.info/index.php/NJE/article/view/12018/9768
5. Bryman, A. (1988) Quality and Quantity in Social Research, London: Routledge
6. Bazeley, P. (2003) Teaching mixed methods. Qualitative Research Journal, 4: 117-126.
7. Maxwell, J.A. (2016) Expanding the History and Range of Mixed Methods Research, Journal of Mixed Methods Research, 10: 12-27.
8. Brannen, J. (2005) Mixing methods: The entry of qualitative & quantitative approaches into the research process. International Journal of Social Research Methodology 8(3): 173-85.