What makes good research? Some reflections

Some ideas concerning the features of good research in education developed through dialogue with colleagues and international masters students:

  • a focus on a definable issue or problem. Research needs to be focused and have a clear area for exploration. If it is too broad it becomes too unwieldy and difficult to collect meaningful data. In attempting to develop a coherent focus for research the use of research questions is extremely important;
  • the need for an ethical approach. All research in education should be developed with an explicit understanding that it should be an ethical process. The vast majority of research in this field includes human participants in some way. Our research should always protect the well-being and dignity of both the participants and researchers. This is often the stated purpose of research ethics, the ‘legal’ aspects which are often the focus of review panels. However, we also stress that ethical research should also focus on the need for honest and transparent reporting so that the work completed can be read critically and fairly by peers. This includes the reporting of research approaches, any conflicts of interest and the context of the research. It also requires that when we rely on the work of others we reference them fully so that they are given due recognition for their work;
  • give a clear outline of the context of research. The process of education is highly complex. Therefore, when writing about research it is always important to give readers a clear context (albeit anonymised) for the research. If a small-scale study is completed with a class of 12 and 13 year olds, in an inner-city school, composed predominantly of more able students then it is important the reader has this information so that they can understand the context of the research data gained. This also allows the reader to consider the degree of relevance of the research to their own situation. It is a central part of honest and transparent educational reporting and debate;
  • making use of research literature to inform the research design. The vast majority of research builds on work already done. It is important to begin to gain an understanding of the research which has been published previously in an area of interest. We need to be good at reading and assessing research so that we can judge the degree of evidence on which we might build our own work;
  • gives a clear outline/discussion of the methodology and methods which have been used to collect data. Ethical research should make the methodology and methods which have been used to collect data transparent. Readers need to know how our research has been carried out as this is crucial to being able to interpret data, and therefore engage critically with any claims which are made. Decisions concerning preferred methodologies gives an insight into the way the research is positioned and the nature of claims made.An account of the data collection tools (methods) used are equally important for the same reasons. If a study has used interviews, are the questions reported so that we can judge the level of neutrality? Where observations are used, is the focus and method of data capture explained? If these issues are not thought through and reported then a considered, critical reading of the research cannot be achieved. Where research occurs at a meta-level, through the use of literature reviews for example, it should include a methodology outlining search criteria, filtering processes and how publications have been analysed. If the literature review merely presents an area of research with no methodology, it needs to be read with caution as we have no way of assessing its validity;
  • uses appropriate methods which clearly link back to the initial issues/problems and research questions. Well-conceived research will be able to make clear as to where particular methods help in investigating the chosen issues/research questions; this gives the research coherence;
  • analyses collected data in a transparent way. In the same way as it is important to carefully consider the reporting of methodology and methods, so it is the case with analysing the data which has been collected. Analysis is often not considered to the same level of detail as methodology and data collection, but it is crucial in ensuring a reasoned and valid consideration of the data, particularly in trying to minimise biases and selective use of data. To make the process transparent it is again important to report how data has been analysed;
  • develops explanations and discussion derived from the data. Good research develops a clear discussion of the data which has been collected. This is at the centre of reporting research as it is where the interpretation of the project is developed. It is crucial that explanations emerge from the data provided and is not dissonant with the evidence provided. In addition the discussion of the data should be related to the literature which you have engaged with and which is the foundation upon which the research study rests;
  • offers measured insights/conclusions. Finally, good research is measured in the claims made. Small-scale research cannot easily make claims which can be scaled up to a large scale, in other words an analysis of one cycle of action research focusing on improving questioning practices in one class cannot act as the basis for national policy. However, small-scale research can still provide extremely important insights for further study and for practitioners by providing useful information as to where good practice might be found. Where conclusions include polemic and assertive language, it can often be the first sign we need to explore the study and its messages further. Many large-scale research projects rely on quantitative analyses. Insights are often based on statistical manipulations and offer a great deal of useful exploration of patterns and trends. However, in-depth explanations are sometimes more problematic as this type of research is often much stronger in providing answers to the ‘what’ rather than the ‘why’. All research has potential shortcomings as no approach is perfect or has all of the answers to an area of interest. Often, deep insights occur through a long-term application of a number of both qualitative and quantitative approaches, used to augment understanding, and giving progressively fuller and more critical perspectives on the issue of interest.

Book Review: An Introduction to Qualitative Research Synthesis

Major, C.H. & Savin-Baden, M. (2010) An Introduction to Qualitative Research Synthesis: Managing the Information Explosion in Social Science Research. Abingdon: Routeldge.

Over recent years there has been a valourisation of large-scale, quantitative research from some quarters. In some ways this is no surprise as there is a ready appeal to see ‘generalised’ patterns in data which can then be used for decision-making and policy formation. However, in this scramble for the use of ‘big data’ there has also been some criticism of qualitative research as being ‘anecdotal’, too focused on the particular, and therefore of little use when it comes to decision-making and policy generation; this has unfortunately also led to a shift in educational research funding which often appears to follow this logic. This book instead focuses on qualitative research and provides a very well argued case for the synthesis of qualitative studies as an additional route to providing insights for practitioners and policy-makers.

The book has three parts, the first two of which (‘Arguing for qualitative research synthesis’ and ‘Doing qualitative research synthesis’) outline and discussion the approach, whilst the third offers examples and frameworks for carrying out qualitative research synthesises (QRS). The first section includes a very clear argument for the use of QRS as an approach for combining and interpreting qualitative research studies. A refreshing element of the first chapter is a clear engagement with the possible problems and restrictions of QRS under the title ‘Top ten criticisms of the approach, point and counterpoint’. This helps develop an honest debate about the potential limitations of the approach whilst making transparent the philosophical and methodological foundations of the approach. This critical voice is retained throughout the book and I think provides an excellent example to those new to educational research in how to build arguments whilst being transparent about both approach and possible restrictions and problems. As the authors state at one point in the book, no methodology is perfect and honest discussion of the limitations and problems which occur as research is undertaken is important. The second chapter in this first section then goes on to locate QRS within the wider field of research syntheses, discussing how it is linked to, but different from, traditional literature reviews and structured reviews such as those developed by EPPI.

The second section goes on to outline the stages in carrying out a QRS, from development of a question around which the synthesis is structured, through designing and completing a search, analysing, synthesising and interpreting the data to presenting the outcomes. The overall explanations are very clear and with the examples provided in section three, there is a good overall explanation of the approach. Importantly, the outlining of QRS is not given in the form of a ‘recipe’, as the process involves a lot of reflexivity, and therefore it is only possible to outline principles and general structures rather than giving a step-by-step ‘how to’ guide. I particularly liked the sections on establishing plausibility, including validity and trustworthiness, which stress the need for qualitative research to be clear in developing contextual information as well as clear explanations of methodology, data collection and approaches to analysis so that the coherence and quality of evidence can be transparently assessed. One table which is provided to highlight this need is given on page 61, based on a template from the Joanna Briggs Institute and could act as a more general starting point for qualitative researchers as they develop reports of their research.

A rating schedule to establish rigour in findings

  • Unequivocal

Findings supported with clear and compelling evidence

  • Credible

Findings that are plausible given the weight of evidence

  • Unsupported

Findings that are suggested but not supported by data

Reading this section, I was struck by the degree to which it would act as a useful starting point for discussion into considering quality research writing not only in QRS, but in qualitative research writing more generally.

Section three goes on to provide some interesting and useful examples of QRS studies which can be considered alongside the earlier sections to see how the process can be understood through the final product.

This book is a very useful introduction to QRS and offers both a critical and clear overview of how qualitative research reports can be synthesised and interpreted to provide broader insights into educational problems and issues. In my opinion, it sits well alongside Pope et al’s (2007) book focusing on the synthesis of qualitative and quantitative evidence in health. The approaches are different, a point which is stressed by Major and Savin-Baden, but together these two books offer critical approaches to bringing together evidence from across the research spectrum to offer new and interesting insights into educational issues.

References 

Pope, C.; Mays, N & Popay, J. (2007) Synthesizing Qualitative and Quantitative Health Evidence: A Guide to Methods. Maidenhead: Open University Press.