Case from a Pharma Company (“Company”)
Company has been integrally involved in the cross-sectoral (industry/academia/regulators/patients/HCPs) initiatives coordinated by [a non-profit organisation facilitating the implementation of solutions in the R&D phase] (hereafter, referred to as the R&D Organisation) and [US based resource center] (hereafter, referred to as Resource Center), aimed to improve communication of study results to participants and the wider community through the development and dissemination of plain language summaries (PLSs). They also served on the Health Research Authority Task Force, a multi-stakeholder working group responsible for developing the EU Regulatory Guidance on Layperson Summaries (release August, 2017). These initiatives will address the requirements of the upcoming EU regulation (2018) to publish a PLS within 12 months after LSLV (last subject last visit) of a study and support efforts to provide study result to participants.
In 2014, Company started its internal work to ensure good preparedness and smooth delivery of this initiative within the Company, reach the PLS quality criteria in line with the requirements, and address patients’ expectation to have results communicated as early as possible after a study completion. This early work, well ahead of the applicability of the EU CT Regulatory requirement (EU Regulation No 536/2014 and EU recommendations for development of Summaries of Clinical Trial Results for Laypersons), enabled Company to understand the challenges and identify pragmatic solution to help inform external work in this area. Company also wanted to ensure that the developed PLS would be understandable and clear for patients and the general public (all of whom are patients as well).
The following are two examples of feedback sprints authors conducted:
- Pilot PLS (COPD) reviewed by EUPATI trainees (2015) and
- 2 PLSs review (COPD study 113108 and study 115151 SLS) via Crowd sourcing Amazon Turk online platform (2017).
Detail on the methodology:
2) Layperson Summaries of Clinical Trials: An Implementation Guide (Download here)
1- Shared purpose
The Resource Center workgroup was co-led by a patient advocate. The multi-stakeholder discussions and ultimately the guidance and toolkit released, represented a balanced and comprehensive approach based on significant input and discussions among the many stakeholders. This work then formed the basis of the HRA Task Force’s guidance which was further vetted and refined specifically for alignment and clarity to enable compliance with the EU Clinical Trial Regulation as to Layperson/Plain Language Summaries (PLSs) shared with patients and community representatives to ensure good understanding of key study points and addressing all unclear stuff prior final PLS approval and publication.
The first activity in 2015 helped Company refine formatting and content to improve clarity. This activity also provided further feedback as to the importance of providing clinical trial results in plain language from the patient perspective.
Pilot 2016: EUPATI interviews
Feedback on version 1 helped refine content and improve quality.
Initial Patient feedback
“It is essential and respectful to volunteers and patients who have taken part in any clinical trial”.
“Surely it will build up good relationships between patients, the public, and researchers”.
“A graph would be useful... I’m very visual”.
“A patient who spoke English very well as a second language misunderstood the distinction between “”inflammation” and “infection” ”.
“Feeling of partnership (not) merely a patient to test on a guinea pig”.
“There are three really important sentences, but they are hard to search out. (Maybe) if there were headings?”
The second activity in 2017 helped Company to understand patients/wider community preferences and expectations from a communication on study results as well as common understanding of the pilot PLS:
“Really can’t stress enough - this was incredibly well compiled and easy to process. I started this under the impression that it would ultimately be mind-bendingly tedious given the subject matter, and it was very pleasantly surprised to find the exact opposite. Thanks!”
“I enjoyed this study very much. I’m always interested in finding out whether the medicines big pharma companies churn out are worth the price or side effects”.
“It was very interesting. Especially so since I may have the beginnings of COPD but have not yet been diagnosed”.
“This is important if the manufacturer wants it to be read by this group (lay public) which apparently it does”.
“I think it’s a great idea because far too much information provided to patients (especially package inserts) is practically unreadable unless the patient has a medical background”.
“As someone who has a relative who suffers from COPD, this is a very interesting trial to read”.
“Thank you, this is important research because it is important that people understand this information and that it is not written in a way that scares people away...”
“It was understandable plus; plus, it was interesting to me since I have COPD”.
“It’s a shame it wasn’t effective. Regardless, this is why drugs are so expensive by the time they get into the consumer market”.
2- Respect and accessibility
At the pilot stage, Company invited the EUPATI trainees (patients who completed the EUPATI course on R&D procedures and medicine development continuum) to have a preliminary feedback on the approach used. At the stage of the review via the Crowd-sourcing Amazon Turk online platform Company assessed feedback on understandability, likeability and areas for improvement from patients and other stakeholders: advocates, media and treatment activists, caregivers and others (118 participants for the COPD Study 113108 and 115 for the SLS study 115151). Company focused on the US geography, three age brackets, varying education levels and all had English as their primary language.
Overall, the PLS development process was highly appreciated by patients and community representatives; the response rates exceeded our expectations.
3- Representativeness of stakeholders
The feedback received on the developed PLS drafts has reflected the interests of several stakeholder categories: first and foremost – patients, caregivers, media and treatment activists, members of patient organisations and other community representatives. The only limitation was around the English-speaking audience in Europe (for the pilot stage) and in the US (for the Amazon Turk platform), as all drafts were written in English.They envision opportunities to obtain feedback on PLS in other therapy areas, clinical trial phases and on discrete issues (e.g. whether a particular graph or chart is understandable etc.).
The diversity of views/opinions was reflected in the following statements from participants regarding possible PLS improvement:
“I would like to see a little more explanation about a few points such as pulse wave velocity and the m/sec value”.
“Improving it would make it longer more complicated, and then it would lose its ease of use. I’d leave this way”.
“No, the document seems to provide the minimum amount of information while still serving its intended purpose, which appears to me to be the goal.”
“If the study was a success, not a success or if it needed more data to be determined successful or not successful. It showed results but never quite stated that I saw if it was worth it”.
“I thought the paper could be longer and more involved, but then it would be complex and not as readable to the average person”.
“Add a glossary to explain terms used in the summary”.
“I would like to see the results for smokers vs. non-smokers”.
“It would have been better if the medicines tested were useful”.
4- Roles and responsibilities
All participants were instructed prior to PLSs review and accepted terms and conditions (at the pilot stage and prior starting the Amazon questionnaire). Participants from Amazon Turk crowdsourcing were included if they were rated as high-quality responders within the Amazon Turk environment. Results for Amazon Turk were assessed across 3 discrete age ranges to get acceptable representation as well as for two different PLS (Company internally rated as “easier” to understand and the other “harder”).
Internally questionnaire was developed that assessed understanding and how well the PLS are received. Company assessed understanding by asking responders to provide 3-4 main points from the PLS in open response and then scored as to whether or not statement was accurate. Thus, Company allowed for individual difference is what stood out as important to each but could nonetheless measure if accurate or not. Company found that respondents were 94-96% accurate on the 2 PLS selected for feedback (one was simpler and one more complex). Company also measured whether there was anything they liked or disliked about PLS and if so, what, and whether there was anything confusing and if so, what.
Feedback received was assessed for quality of responses and completeness of questionnaire and found to be acceptable by internal team for the intended purpose.
5- Capacity and capability for engagement
Having in mind the key target audiences for PLSs as study participants (patients, including expert patients) and community representatives (caregivers, advocates and activists) Company asked them to review the prepared drafts and provide a feedback on wording/terminology, visibility, format, understandability and the ways data presented.
PLS development/review projects have made them understood that patient and general public input is needed to assess understandability to them. Extrapolation and repeated assessment in different populations will likely be needed to assess how well-received the PLS are for a given population. Company strive for understandability at a 12-year old reading level and use health literacy and numeracy principles but in addition, stakeholder feedback is critical to ensure Company is achieving a quality PLS that is fit for purpose.
6- Transparency in communication and documentation
The received feedbacks were documented and presented internally. As Company takes part in the cross-sectoral initiative as an industry representative including at the Resource Center and its findings were presented and discussed with broader stakeholder groups including patient and patient representatives such as at the EFGCP-EFPIA Workshop, Brussels on May 2. 2017. This case study was also presented at the PFMD Task Force meeting on May 9, 2017.
Due to the tight timelines for developing and disseminating PLS, Company would not recommend obtaining individual feedback before dissemination. However, obtaining feedback generally and on specific issues and incorporating lessons learned and best practices moving forward is quite important. Crowdsourcing is useful for feedback from a general audience. Feedback from study participants is also important, keeping in mind that perspectives can differ by TA, geography, phase of study, age of participants, etc.
Written materials, documents and records are available internally and/or externally.
7- Continuity and sustainability
Company has developed a standard operating procedure (SOP) for development, translation and distribution of PLS with particular consideration of a mechanism for getting feedback from patients (study participants) and wider community on developed PLS to improve their quality.
The PLS development strategy as an essential part of Company’s commitment to R&D transparency and disclosure of study results has been substantiated and approved for the period of 2018-2021.
Building in quality by strategic writing and review process which help establish roles and responsibilities as well as actual process for writing and reviewing by each involved stakeholder. This well-defined writing and review provided clarity around who is reviewing for plain language, who for scientific accuracy and who for technical review, for example. Often there is a need to go back and forth between plain language and subject matter experts to assure accuracy remains, however, subject matter experts are often not adept at writing in plain language. Another learning has been the benefit of our internal work in developing PLS template and detailed instructions so that Company can hand to its selected external vendor now in 2018 and help develop their capability and understanding of what Company wants.
Rollout plans are to post PLS to Company’s Study Register along with translation to local languages. Company is scaling up this capability and will expand beyond the EU CT Regulatory requirements for PLS.
Positive impact for specific medicines development phases:
Although this case study reflected the post-phase IIIb experience, PLS may be reviewed after any phase study/clinical development milestone (of course, with the biggest consideration of phase II-IV interventional studies) and Company is currently seeking feedback from study participants. This approach contributes to Company’s values:
- Be focused on the patient;
- Respect for people;
- Act with integrity;
- Operate with transparency.
Direct or indirect positive impact for patients:
- Explore and/or pilot co-creation of PLS to ensure clarity and consistent understanding by several groups of patients and community representatives;
- Awareness of study results as early as possible;
- Satisfaction of study participants (if selected as PLS reviewers) making them proud of any to have contributed to the study and any knowledge gained as a result.
Direct or indirect positive impact for stakeholders involved in the project (other than patients):
- PLS team within the Company: lessons learned and areas for improvement;
- Cross-sectoral project teams (other companies, (in the development of an Implementation Guide and Recommendations for Drafting
- Non-Promotional Layperson Summaries – both through the R&D Organisation, regulators contributed input toward an aligned PLS guidance from FDA, EFPIA/industry associations (Reflections paper), HCPs and investigators);
- Public health benefits: increase transparency and openness within healthcare systems. The public (as patients) may feel empowered with increased knowledge and understanding of clinical trials and are better able to discuss their condition. Also, they may develop greater trust in the drug development process.
After concluding the Amazon Turk second sprint, the assessments are following:
- There is a right direction;
- This Amazon Turk system offers a fast way to see if specific portions of a PLS are understandable;
- Great method for quick check where needed;
- Could be used to test for specific issues (e.g. are more graphics/wording/glossary preferable?)
- Is there benefit in considering a head-to-head comparison with scientific summary or publication abstract?) or clarity/confusion of part of a PLS as an adjunct?
- Significant expertise has been gained, which can help with vendor selection and onboarding
Methodology, involvement of patients and the general public for feedback.
To seek feedback from study participants as a specific target audience.