5. Lesson 2 Methods of evaluation

5.1. The problem of evaluating impact

Lesson 1 showed how evidence of the impact of patient and public involvement in research is mainly demonstrated through case studies. These are very useful, but not generalisable, so building a body of evidence remains somewhat elusive.

Patient and public involvement might be considered as a complex intervention, where there is much in-built variability and many interacting component parts. According to the MRC, this presents a number of extra problems in evaluation, relating to β€œthe difficulty of standardising the design and delivery of the interventions, their sensitivity to features of the local context, the organisational and logistical difficulty of applying experimental methods to service or policy change, and the length and complexity of the causal chains linking intervention with outcome.”

A recently published paper challenges this idea and proposes (though with little detail) some alternative methods of evaluation. - (Reference: Edeman N, Barron D. Evaluation of public involvement in research: time for a major re-think? Journal of Health Services and Research and Policy. 2015)

The gold standard of evidence, the RCT, is not likely to be helpful here. So what is valid evidence of impact? We will look at one method of evaluating the impact of patient and public involvement in research: realist evaluation, which can help to establish whether it is effective and how. Then we will look at a framework designed to provide a standard method for researchers to plan and evaluate the impact of patient and public involvement in their research.