some essential research knowledge.
As far as critique frameworks are concerned, there are a number of these, all aimed at providing some kind of systematic checklist of important areas to consider within a research report. The Webinar will use the one developed by Rees (2003), as this comes in two versions depending on the type of research being examined. However, the important principle is to use one that covers relevant issues and that you find easy to apply. Without a system, it is easy to overlook important elements that might be missing.
In terms of research knowledge, this is important if we are to assess if the research has been carried out soundly, and can be compared to the best of its type. In other words, we need to be confident that the researcher has been rigorous in the way they pay attention to good research practice. If they have, the result will be robust findings; that is, we will be able to apply the research safely in a variety of appropriate settings.
Although research is a vast subject area, we can go a long way with some essential considerations that could affect our evaluation of an article. Some of these are highlighted below as essential pointers to remember, and can be reinforced with the help of a research textbook.
Essential pointers to remember
1. One research study rarely provides the definitive answer. That is why replication studies, that is repeating an original study under the same or slightly different conditions, are important to ensure that the results of a single study weren’t a one-off fluke. Research is rarely undertaken under perfect circumstances, so it’s not going to be flawless. Although we must remember that there are possible limits to knowledge derived from research, it is certainly better than blind faith in interventions that have not been subject to enquiry and testing. The implication of this is that you should not feel that published research is beyond criticism.
2. Research can take different forms. Two broad classifications are those of quantitative and qualitative approaches. Although some academics do not like this distinction, if you are new to critiquing it certainly helps to understand some important differences in study designs. Quantitative research focuses on measurement and the relationship between things of interest, for example the level of continuity provided by a midwife to a woman, and the duration of breastfeeding. It is relatively easy to spot this approach as the results are in the form of numbers, for instance in tables, bar charts, pie charts or similar.
Qualitative research is not about ‘quality’ or simply subjective views. It is about experiences, understandings and interpretations. It explores human situations and events through the eyes of those concerned, using their language and descriptions of things. The findings sections are in the form of words rather than numbers.
3. The most important aspect to identify when critiquing is the aim of a study. The Webinar will tell you where to find it, but it usually begins with the word ‘to’ and will be written something like this: ‘the aim of this study is to determine/examine/identify/describe’. This, and the rest of the words in the sentence that comes after it, form the aim of the study. There may be more than one part to this, and if that is the case, it is worthwhile naming them as part 1, part 2, etc. As you read through the study, you should ensure that each of these parts is followed up in the results section and the conclusions. It is easy for the researcher to start off with an impressive list of aims and not address them all without you noticing.
The importance of the aim is that it influences the form and nature of many other aspects of a study, such as the broad research design, the tool of data collection, and method of analysis and sample. At the end of the study, part of your judgement will be how well the author has achieved the aim.
4. Ethical considerations are one of the elements it is crucial to look for in a study. An ethics committee should have approved studies involving women and their babies, family members or midwifery staff. In the UK this committee will be called an LREC, or even REC. In the US it will be called the IRB, which stands for Institutional Review Body. You should also ensure that informed consent has been given by those involved, and that they were treated with dignity throughout.
5. The use of numbers, and statistical processes in research articles intimidates many midwifery readers. Although we should not always assume they must be right, if it is a ‘peer reviewed’ journal, it usually means that someone has considered whether the appropriate kind of statistical techniques have been used and may have checked their accuracy.
However, do avoid ignoring anything to do with numbers or charts. Do look at the figures and try to work out what they show. Usually the writer will explain in the article what the numbers represent and their interpretation of them. A regular reading of the comments attached to the MIDIRS abstracts of research will confirm for you that it is not what the researcher found that is important, it is their interpretation of what they found that is crucial. MIDIRS abstract reviewers will often point out that there may be another way of interpreting the results. That is why with tables and figures you should think about your interpretation of the results, and then compare that with the researcher’s interpretation.
6. Similarly, don’t be put off by the language of research. When in doubt of the meaning of words, you can quickly find out the meaning by using ‘Wikipedia’ on the web (www.wikipedia.com).
7. In most research studies, there are a number of different research tools that can be used to collect data. You need to consider if their choice of tool was a reasonable one, given the aim of the study and the target sample group. Do they justify their choice to you? If not, do you think an alternative tool may have been more appropriate?
Every method of collecting data will have its advantages and disadvantages, so when you consider their tool of data collection think about why it was a good choice for this study, and whether there are any limitations that are a natural part of that tool.
In the case of a quantitative study, the tool should either already have been used in previous research, and therefore known to be accurate, or piloted to test how consistently and accurately the tool worked. If the researchers have demonstrated one of these alternatives, then they have attempted to consider the reliability of the tool. A related issue is that of validity, which relates to how they were sure the tool was measuring what they hoped it would. Often in the discussion section, the results of similar studies using the same tool will be compared in order to suggest that validity has been demonstrated.
Qualitative research will not have a standardised tool. In fact, one feature of qualitative tools and approaches to data collection is that they are very fluid. This makes piloting difficult, as they are not going for standardisation. Although many readers are uncomfortable with this flexibility, qualitative research is carried out in the same rigorous way, especially during data analysis.
8. Look closely at the size of the sample, and how this compares to other studies mentioned in the article. In qualitative research, the sample can be quite small, even under a dozen. This is still relevant because of the depth of data produced. In quantitative studies, the general rule is that the bigger the sample the better, as this makes generalisations more acceptable. However, this does not apply in qualitative studies, it is whether the sample have experience in whatever the researcher is interested.
The way the sample was selected for inclusion in the study can vary enormously. The main consideration in quantitative research is whether those in the sample are typical of those they represent. As with the tool of data collection, each sampling method will have its advantages and disadvantages. Research texts need to be consulted for more information on this.
9. The crux of a study will be what the researchers found as the answer to the question they set out to explore. These are the main findings of the study. Read not only what the authors say they found, but look at the results yourself in the tables, charts or quotes they present.
10. The discussion section will raise the issues arising from the results. It may start out with the limitations of the study. Do take this as a positive inclusion, as they are trying to help you determine how much emphasis you should put on the results.
11. The final section of a research paper will include the conclusion and recommendations. These round off the study, and should answer the aim. In some cases, the conclusion is really the recommendations and you have to look to the discussion for the answer to their aim. The recommendations should say who should do what now, and how, based on the researchers interpretation of the results.
12. The most important part of the critique is whether it is fit for purpose. You need to consider if you feel the results can reasonably be applied to practice. In some cases, articles will suggest this element themselves, often in a box at the end of the article.
Finally
Critiquing is a core midwifery professional skill that will inform clinical practice. It is important not to dismiss research as being too ‘scientific’ for the art of midwifery, as women and their families deserve the best form of care based on the best knowledge available. However, do not accept all research simply because it has been published. Part of the responsibility of the researcher is to share their findings so that they can be debated and weak areas identified and strengthened.
The most important point is to consider how the research can make a contribution to practice.
When you have critiqued a study you should be able to explain verbally or in writing the following main areas, and these can form the summary of your critique:
The aim of this study was to (aim)…