Tagged / multidisciplinary

Mixed methods: not without its downside?

REF Subjects

Prof Edwin van Teijlingen

Conducting mixed-methods research has become very popular over the past decade especially in the health research field.1-4 This development ties in with the growth in inter-disciplinary and multi-disciplinary research. Many grant applications, PhD project and the resulting papers especially in the health field apply a mixed-methods approach, where in the past a single approach would have dominated.   This interest in combining methods seems to be the case even in the more traditional quantitative field of clinical effectiveness and randomised controlled trials. Whilst I find this development encouraging as a mixed-methods social scientist, it also makes me wonder whether the applicants putting forward a mixed-methods project have thought about the disadvantages or at least the opportunity costs of using such approach.

A mixed-methods approach is ‘simply’ combining two or more research methods to address a research question, i.e. what the label suggests. It is often perceived as the combining of qualitative with quantitative methods, but it can technically also be a mix of quantitative methods or a combination of qualitative methods. The advantage of a mixed-methods approach is that the different methods in the mix address different aspects of the research question and that combining these methods offers a synergetic effect. So what are the possible limitations of or barriers to mixed-methods research?

First, using a mixed-methods approach means you need an understanding of two different philosophies and how to bring the findings of these two different methods together.4-6   One requires expertise in two different research approaches, either as individual or in the team as well as someone who can do the combining of the findings. For the latter you really need someone in the team who understand the pragmatic approach commonly used in mixed-methods approaches. Otherwise there is a great risk that the original mixed-methods study will be analysed and reported as two or more separate papers each based on data from one of the methods applied in the mixed-methods study.

Secondly, you can spend your money only once, hence there are opportunity costs. Thus if the maximum grant is £200,000 or £300,000 you can’t spend the full amount on the designing a large-scale quantitative study/survey, as you need to spend a proportion of your money and your attention and time on your qualitative study.

Thirdly, and related the above, both quantitative and qualitative methods have ‘rules’ about sampling and sample-size.5 Just because you have two methods this does not mean you can necessarily do a study with a smaller sample. The sample size calculations will still say you need at least xxx participants. Similarly, although perhaps not so rigidly you need a certain number of interviews or focus groups to do you qualitative study appropriately.

Fourthly, a common mistake seems to be to add a bit of qualitative research to a larger quantitative study, perhaps a bit tokenistic.7 Often it is so obvious in a grant application that the qualitative research is an add-on, an afterthought perhaps from a reviewer in the previous failed grant application.

Finally, not all mixed-methods studies are the same, in fact each mixed-methods study is more or less unique in the way in the way it mixes and matched individual research methods.3 So although mixed-methods may be the best way to address a particular research question, your particular proposed mixed of quantitative and qualitative research might not be the most appropriate to answer the overall research question.8

As with all research methods and research proposals my recommendation is if in doubt go and find an expert for advice.6 If necessary get an expert on your team of researchers to strengthen your application.

 

Professor Edwin van Teijlingen

CMMPH

 

References:

  1.  Barbour, R.S. (1999) The case of combining qualitative and quantitative approaches in health services research. Journal of Health Services Research Policy, 4(1): 39-43.
  2. Simkhada, P., van Teijlingen, E., Wasti, S.P., Sathian, B. (2014) Mixed-methods approaches in health research in Nepal, Nepal Journal of Epidemiology 4(5): 415-416.
  3. Plano Clark, V.L., Anderson, N., Wertz, J.A., Zhou, Y., Schumacher, K., Miaskowski, C. (2015) Conceptualizing Longitudinal Mixed Methods Designs: A Methodological Review of Health Sciences Research, Journal of Mixed Methods Research, 9: 297-319.
  4. MacKenzie Bryers, H., van Teijlingen, E. Pitchforth, E. (2014) Advocating mixed-methods approaches in health research, Nepal Journal of Epidemiology 4(5): 417-422. http://www.nepjol.info/index.php/NJE/article/view/12018/9768
  5. Bryman, A. (1988) Quality and Quantity in Social Research, London: Routledge
  6. Bazeley, P. (2003) Teaching mixed methods. Qualitative Research Journal, 4: 117-126.
  7. Maxwell, J.A. (2016) Expanding the History and Range of Mixed Methods Research, Journal of Mixed Methods Research, 10: 12-27.
  8. Brannen, J. (2005) Mixing methods: The entry of qualitative & quantitative approaches into the research process. International Journal of Social Research Methodology 8(3): 173-85.

 

REF update: HEFCE’s REFlections event, 25 March 2015

Research assessment

I went to HEFCE’s (rather cleverly named) REFlections event on Wednesday to hear about the review of REF 2014 and plans for the future of research assessment.

The key points were:

  • Collaboration and multi-/interdisciplinary research are likely to be important for the next REF
  • HEFCE have commissioned Elsevier to undertake a project on measuring multidisciplinary research to inform the next REF
  • The REF impact case studies database went live yesterday and is an excellent resource
  • Dual support system is likely to stay
  • Impact case studies are likely to stay, however, the impact template may change/become obsolete
  • Peer review will stay, informed by metrics in some disciplines (akin to REF 2014)
  • Metrics are not yet robust enough to have a metrics-driven REF. In particular, this is not yet possible for the assessment of outputs or impact. It is possible, however, to rely more heavily on metrics for the environment assessment and there could be changes to this part for the next REF.

 

HEFCE plan to consult with the sector on future plans for the REF this coming autumn.

 

Further information:

Search for a Champion? Or BU Research Themes

Thank you to all those people who responded to the online questionnaire about the BU Research Themes.  There were 273 responses!  On this basis we can reduce the short list of twelve societal themes down to ten as set out in the briefing paper (available from  – I:\CRKT\Public\RDU\Research themes\Paper.docx).  The ten are:

  • Health and wellbeing
  • Recreation and leisure
  • Culture and society or Society & Social Change
  • Environmental change and biodiversity
  • Green economy and sustainability
  • Creative and digital economies
  • Aging
  • Learning and public engagement
  • Entrepreneurship and economic growth
  • Technology and design

We urgently need some champions to ‘flesh-out’ these themes to help scope them and allow us to ‘road test’ them further.  I am looking for as many views as possible for each theme; just fill out the template (available from  – I:\CRKT\Public\Research themes\Form.doc) and e-mail it back to me.  The idea is to then distill these views and produce a scope for each; if there are no champions forthcoming then we may be able to reduce the list further.  I need the templates back by the 27 May if possible; thank you!

Matthew Bennett

PVC (Research, Enterprise & Internationalisation)

(The documents are saved to the I-drive. If you are on-campus then you need to copy and paste the file pathway into an internet browser. If you are unable to access them please contact Julie Northam who will email you the documents).

Researchers of the next generation

Guidance

Prof Holger Schutkowski, Deputy Dean in the School of Applied Sciences, joined BU in January. Here he provides his thoughts on training the next generation of researchers.

In a week’s time I will have the great pleasure to open the School of Applied Sciences’ Postgraduate Research Conference. I was delighted that I was asked to give a keynote, not only because it is a nice way of making myself known to students, since I only arrived at BU in January, but also to share some thoughts about the way we do research. Whilst universities require some original research in the final year undergraduate dissertation, and to a much greater extent in Master’s programmes, it is at PhD level where we expect the clear evidence of intellectual independence, of playful recombination of knowledge, which will allow candidates to go beyond current established borders of thought, and to push scientific progress, something that is always happening at the fringes.

Are we able to give advice? Should we? How can we make this happen? The latest deliberations about the future of current PhD systems and their ever-increasing production of graduates is beginning to raise serious concerns and to elicit calls for reforms, quite drastically, indeed (http://www.nature.com/news/2011/110420/full/472261a.html). Interestingly enough, they emphasise co-operation between institutions (consortia) and cross-disciplinary work.

While the former is already part of the RCUK agenda and is likely to shape the future of PhD funding and recruitment in the UK over the next years, the latter maybe can do with further encouragement. Certainly, one way is to encourage our PhD students to decidedly undertake an interdisciplinary project, which essentially means to embrace the boldness of crossing borders, to work across disciplines and to become acquainted with, adopt and modify interpretive frameworks of other, related or cognate subject areas. Often claimed, rarely done, though; unfortunately a recurring problem and certainly not confined to the fledgling researcher.  Another approach is to ensure the research is firmly embedded into a wider and meaningful context, so that these connections across disciplinary borders can be made and the outcomes of the research become accessible and meaningful to the related or cognate subject areas we are trying to include in the first place.

As it happens, this is what I am going to talk about in my presentation, with case studies that necessitate contextualised analysis to demonstrate how we can bring past societies to life through the study of their skeletal remains. But to come back to the state of PhD systems: cross-disciplinary thinking and context awareness is what we want to instil in the researchers of the next generation. But this also requires the intellectual capacity and preparedness for doing so. The students we graduate need to be skilled and prepared for an increasingly competitive job market.

Prof Holger Schutkowski

Deputy Dean

School of Applied Sciences