PPI 2015 Unit III

1. Introduction

Unit III Impact of patient and public involvement in research

You have seen how patient and public involvement is now enshrined in research policy in the UK and is increasingly an essential requirement to gaining a grant. You have also seen how much effort is needed to make it effective. A key challenge is to demonstrate that involving patients and the public makes a positive difference to research.  In this unit, you will critically examine the evidence to support Professor Dame Sally Davies’ assertion that patient and public involvement improves the quality, relevance and cost-effectiveness of research (see Unit 1, Lesson 1). We will look at how impact might be measured and why it seems to be problematic. We will critically appraise some real-life case studies where researchers claim to demonstrate impact from involving patients in their research. We will also look at impact from the patient and researcher perspective, including a tutorial with Marilyn Kendall, a highly experienced researcher with an impressive track record of involving people affected by cancer and other long-term conditions in her research.

Aims

To critically examine the nature of evidence of the impact of involving patients and the public in research.

Learning outcomes

1. Critically appraise the theory and key principles of patient and public involvement in clinical research
2. Critically assess their own practice/place of work in relation to the level of patient and public involvement in research.
3. Evaluate the barriers to effective patient involvement and strategies and techniques to overcome such barriers.

2. Overview

Features

Unit III runs over two weeks (Feb 8th - 28th) and is organised into the following 3 lessons:

• Lesson 1 Determining the impact of involving patients and the public in research
• Lesson 2 Methods of evaluation
• Lesson 3 Tutorial with Dr Marilyn Kendall on the impact of patient involvement on her research (week 6)
  

Each lesson is comprised of a selection of partner readings, written content and independent activities. There is a Unit III discussion board which includes 2 Thought questions for collaborative discourse.

As you may know from the course planner, there are a number of key dates and action points for you to be aware of in weeks 5 & 6.

Key dates

• Friday, Feb 12th: Due date Individual Project I 

Adobe Connect

• Tuesday, Feb 23rd (6 - 7 pm): Online expert guest lecture with Dr Marilyn Kendall
  • Topic: The researcher’s experience: the impact of patient and public involvement

Adobe Connect URL: http://edinburghcrf.adobeconnect.com/ppi/

Your tasks

Please refer to the Unit III checklist

[Checklist pops up in a separate window]

3. Core readings & resources

For core readings and resources, please refer to the Resource list on the Moodle course homepage (from the menu list on the left-hand side of the page).

4. Lesson 1 Determining the impact

Partner readings

For Unit III partner readings, please refer to the Resource list on the Moodle course homepage (from the menu list on the left-hand side of the page).

Topics

• What is impact?
• Why is it difficult to measure?

Introduction

We have received a great deal of activity around patient and public involvement in research. Now, we will look at the impact of all this activity. Does it make a difference to research? We'll look at case studies to consider this question.

4.1. What does impact mean?

‘Impact’ addresses these questions:

1. Does involving patients and the public in research make a difference to the research process and outcomes?
2. What impact does involving patients and the public have on the researchers and the patients/members of the public who get involved?

Impact can be positive or negative and is often a mixture of both.

4.2. Why is it difficult to quantify the impact?

Robust evidence of the impact of patient and public involvement in research, it is often said, is scant. Challenges to evaluating impact include: inadequate theoretical underpinnings for involving patients and the public in research; variability of methods and levels of involvement (process variability); the variety of contexts in which involvement is conducted; lack of clarity about purpose, process and outcomes of involving people; lack of standard methods of evaluation. Patient and public involvement is just one aspect of research design and it is difficult to pin down what action directly influenced any particular outcome. Even if methods were standardised, it is unlikely that public involvement carried out in the same way would achieve the same impacts across different research projects. It is difficult to generalise from methods used in one impact assessment to another or to generalise about the findings of different studies of impacts. The diversity of the patients/members of the public who are involved and the aims of and context for involvement make it hard to predict where involvement will have the greatest impact. Patient and public involvement is also a relatively new and fast-growing area of practice, so evaluation methodology is still developing.

We will look at the nature of ‘evidence’ in Lesson 2; for now, we will consider the reported evidence of the impact of patient and public involvement on research. Each of these case studies of individual clinical trials and research programmes claim to show the impact of involving patients.

4.3. Case study 1

The PLEASANT trial

The PLEASANT trial is the first example in the INVOLVE report ‘Exploring the impact of public involvement on the quality of research: examples’

The trial aims to reduce asthma exacerbations in school children at the start of a new school term by a simple intervention in primary care. The researchers involved children with asthma and their parents prior to submitting their grant application, so they could demonstrate impact on the research design as follows:

1. There was a lot of support for the idea for the project, which the researchers could use in the application to show that it was endorsed by people with the condition.
2. Discussion with parents and children resulted in agreement that the invitation letter should go to the parent or guardian rather than the child.
3. The wording of the letter was significantly changed. The original was quite long and complicated – it was refined and simplified at the meeting. Of particular importance was the opening paragraph which now starts ‘Please read this important letter about your child’s asthma’ in bold to get their attention.
4. The timing of the intervention was discussed and it was agreed that the letter should go to parents two weeks after the beginning of the holidays.
5. The children chose the logo for the project.

A second meeting was held when the funding was awarded to update the parents, to discuss changes to the letter, and to contribute to the lay summary of the Research Ethics Committee submission. Two parents were invited on to the Trial Steering Committee. Another consultation event was held at the end of the study to present the findings, to discuss the interpretation of the data and to produce a lay summary of findings.

4.4. Case study 2

Consumer involvement in the design of a randomized controlled trial of routine oxygen supplement after acute stroke (Ali et al)

In this study, people affected by stroke were consulted about the design of a clinical trial that aimed to test the effects of oxygen supplementation following an acute stroke. The patients were asked to comment on consent issues, the relevance and acceptability of the outcome measures and the preferred method of follow-up. Following the consultation, one of the main changes that was made to the trial was to introduce new outcome measures. The most commonly used stroke assessment scales focus on physical health and physical recovery. However, the patients pointed out that quality of life after stroke is determined more by cognitive and emotional problems than by the level of disability. They therefore wanted the trial to assess the impact of treatment on communication, mood, cognitive function, tiredness and sleep. As these are not well represented in the standard assessment tools, the researchers added their own questions to the assessment questionnaires. This helped to make the results of the study more relevant to people affected by stroke.

4.5. Case study 3

Patient and Public Involvement in The Asthma UK Centre for Applied Research (AUKCAR)

Asthma UK, a charity supporting asthma research, decided to fund a new research centre with the aims of reducing asthma exacerbations and deaths and improving asthma management.  The University of Edinburgh successfully applied to run the Centre and built in patient and public involvement from the start (declaration of interest – Allison Worth led the patient and public involvement work). We asked Asthma UK to email all their members in Scotland asking what they thought our research priorities should be. 52 people with asthma and parents of children with asthma took part in a priority-setting exercise by completing questionnaires, or taking part in focus groups and telephone interviews. They identified six priority research areas, which we built into the grant application. Three people affected by asthma were co-applicants, came to our funding ‘interview’ and now sit on our Centre Management Committee, so they are involved in all the strategic decision-making for the Centre. The Centre now has a Patient Advisory Group of 81 people across the UK who advise on all aspects of our asthma research. A Children and Young People’s Group has also been set up. Every project run by the Centre must have patient and public involvement throughout. The Patient Advisory Group advises on every aspect of project design and conduct. Patient and public involvement in AUKCAR has an impact on:

• research topics: a funded PhD studentship is based on an idea generated by one of the group
• training for researchers and students: patients contribute to face-to-face and webinar training sessions on topics such as living with asthma and writing for a lay audience
• study design: patients contribute to study materials (lay summaries, patient information leaflets and consent forms), influence interventions and outcome measures
• funding decisions: a patient panel reviews all PhD studentship applications
• facilitation: patients co-facilitate the children and young people’s group and occasional research prioritisation groups
• interviewing: patients participate in interview panels for new researchers
• dissemination: patients help to promote the Centre and its research through blogs, public speaking and charity events and co-author publications

Olivia’s blog, which you looked at in Unit 2, emphasises the impact involvement in AUKCAR has on her personally. Here is another comment from one of the Patient Advisory Group:

“As the working mother of a young child with asthma and living far from the Centre, I get involved mostly through telephone discussions with the researchers. I am particularly interested in improving care and treatment for children and have been involved in some research studies which focus on that. I have also been part of a ‘patient panel’ to review PhD studentship applications for the Centre. This was a fascinating experience as I developed an even greater understanding of the facts surrounding asthma and the impact it has on the lives of sufferers and their families. It was reassuring to read so many innovative ideas for studies that could make such a huge difference to people. It is comforting to know that there are so many enthusiastic students who are dedicated to improving the lives of asthma sufferers in a variety of ways. I find it interesting to hear about all the research that’s going on in the Centre and like the fact that I can choose what to get involved with. I feel really encouraged that so much exciting research is going on that will help children with asthma lead better lives and I feel privileged to have the chance to play a small part in it.”

4.6. Your turn: Thought question 1

Task I. Thought discussion

Please join us on the Unit III Thought discussion for Thought question 1.

Is the evidence of impact we have looked at sufficiently impressive? Can you identify:

• Evidence of impact on the research process?
• Evidence of impact on research outcomes?
• Evidence of impact on patients and members of the public?
• Evidence of impact on researchers?

Do you think there are gaps in the evidence and if so where?

We look forward to your thoughts!

5. Lesson 2 Methods of evaluation

Partner readings

For partner readings, please refer to the Resource list on the Moodle course homepage (from the menu list on the left-hand side of the page).

Topics

• How to approach evaluation of patient and public involvement
• Methods of evaluation

Introduction

It should be clear to you by now that evaluating the impact of patient and public involvement in research is rather difficult. Currently "realist evaluation" seems to be the favourite method to address this, so we will explore it and see if it helps.

5.1. The problem of evaluating impact

Lesson 1 showed how evidence of the impact of patient and public involvement in research is mainly demonstrated through case studies. These are very useful, but not generalisable, so building a body of evidence remains somewhat elusive.

Patient and public involvement might be considered as a complex intervention, where there is much in-built variability and many interacting component parts. According to the MRC, this presents a number of extra problems in evaluation, relating to “the difficulty of standardising the design and delivery of the interventions, their sensitivity to features of the local context, the organisational and logistical difficulty of applying experimental methods to service or policy change, and the length and complexity of the causal chains linking intervention with outcome.”

A recently published paper challenges this idea and proposes (though with little detail) some alternative methods of evaluation. - (Reference: Edeman N, Barron D. Evaluation of public involvement in research: time for a major re-think? Journal of Health Services and Research and Policy. 2015)

The gold standard of evidence, the RCT, is not likely to be helpful here. So what is valid evidence of impact? We will look at one method of evaluating the impact of patient and public involvement in research: realist evaluation, which can help to establish whether it is effective and how. Then we will look at a framework designed to provide a standard method for researchers to plan and evaluate the impact of patient and public involvement in their research.

5.2. Realistic evaluation

This method of evaluation asks the question: ‘What works for whom in what circumstances . . . and why?’ It analyses a number of case studies, comparing the process and outcomes from a range of perspectives (such as patients, researchers, managers, funders) to identify the links between context, mechanisms and outcomes across studies. It is theory-driven, in that it starts with a theory about how a particular mechanism operates in a specific context to produce a defined outcome. The theory is used to generate a hypothesis about what aspects of the mechanism might produce change, which subgroups might benefit most readily and what resources are necessary to sustain the changes. The evaluation or study then adopts a design and methods of data collection and analysis to test these hypotheses.

Evans et al. (2014) conducted a qualitative realist evaluation and found that public involvement was associated with improvements in research design and delivery, particularly recruitment strategies and materials, and data collection tools. They identified the importance of principal investigator leadership as a key contextual factor leading to the impact of public involvement. In terms of the mechanisms of involvement, allocating staff time to facilitate involvement appeared more important than formal budgeting. Many research proposals significantly under-costed public involvement. Nurturing good interpersonal relationships was crucial to effective involvement. Payment for patients’ time and formal training appeared more significant for some types of public involvement than others. Feedback to patients on the value of their contribution was important in maintaining motivation and confidence.

5.3. Public Involvement Impact Assessment Framework (PiiAF)

This custom-made framework for evaluating patient and public involvement aims to get researchers thinking about impact when they plan their research, rather than retrospectively. The authors identify these aspects as influencing impact:

• Approach i.e. the ways in which members of the public are involved in the study
• The values associated with public involvement by members of the research team
• The focus of the research and study design
• A range of practical issues including human and material resources.

You can see a worked-through example of how this is recorded on their website http://piiaf.org.uk/

5.4. Your turn: Thought question 2

Task I. Thought discussion

Please join us on the Unit III Thought discussion for Thought question 2.

Revisit the thought question from lesson 1 Unit III Thought question 1

Does realist evaluation offer a way forward to improve the evidence base for patient and public involvement in research? What more is needed to develop the evidence base for the impact of patient and public involvement in research?

We look forward to your thoughts!

6. Lesson 3 The impact of patient and public involvement on the researcher

Partner readings

For partner readings, please refer to the Resource list on the Moodle course homepage (from the menu list on the left-hand side of the page).

Topic

• The researcher's experience of involving people in research.

Introduction

In this lesson we will look at Dr Kendall's experiences of involving people in health service research using a variety of approaches. We will discuss the strengths and weaknesses of each approach and the general challenges there have been for both the researchers and the service users. We will finish by considering what has been learned along the way and how we might improve the process in the future.

Your turn activity: Expert guest lecture

Task I. Join online guest lecture

In this tutorial, you will have the opportunity to talk to Marilyn Kendall, a researcher with extensive experience of patient and public involvement in research . You can ask any questions about the process of involving people, the benefits she perceives, the problems she has faced and the impact it has had on her personally and on her research.

Presenter: Dr Marilyn Kendall Topic: The researchers experience: the impact of patient and public involvement Date: Tuesday February 23rd 2016 Time: 6 - 7 pm (GMT) Location: http://edinburghcrf.adobeconnect.com/ppi/ This session will be recorded.

6.1. Profile: Dr Marilyn Kendall

Dr Marilyn Kendall

Dr Marilyn Kendall has a long-standing interest in lay involvement in health and social care. She was for many years a member and then Chair of the Northumberland Community Health, before commencing a career in health service research. She is now Senior Research Fellow with the Primary Palliative Care Research Group at the University of Edinburgh and undertakes qualitative research with people living with a variety of life-limiting illnesses.

Dr Marilyn Kendall

Senior Research Fellow , Primary Palliative Care Research Group,  University of Edinburgh

6.2. Your turn activity

Task I. Reflection

1. What is your view of the evidence of impact (positive and negative) of patient and public involvement in research? Did your discussion with Marilyn influence your view? Do you have any additional suggestions about how the impact of patient and public involvement can be demonstrated?
2. Revisit your reflection from Unit 1, lesson 1 when you recorded your initial thoughts on patient and public involvement. Has your thinking changed, and if so, how?

7. Unit III summary

Think back on the Unit III lessons and activities - what have we learned?

Consider the Unit III learning outcomes - have you achieved them? Congratulations! You have completed Unit III. Unit IV will be released in week 7.