Sponsored by GSK

Description:

With the recent emphasis on equity, diversity and inclusion, it is more important than ever to capture the perspectives of all members of a given population. It is no longer acceptable to simply include the easily accessible majority while ignoring important under-represented groups. This plenary session focused on strategies to engage populations who, for a variety of reasons, traditionally are resistant to complete PROMs and/or enroll in research. In addition, it highlighted cultural differences in the way that questions should be asked and how the results should be used and interpreted.

Speakers:

Nancy Young, BScPT MSc PhD, CHEO Research Institute, Ottawa, Ontario, Canada

Dr. Young is a Senior Scientist at the CHEO Research Institute and specializes in child health measurement science. She leverages research to elevate the voices of children in health assessments. She has collaborated with many different populations, including children with complex disabilities and bleeding disorders. Her current research focuses on innovative methods to measure quality of life among children with rare disorders and on creating culturally-relevant approaches to health measurement with and for Indigenous children and youth. Through collaboration, she has improved access to high-quality local data that inform health services planning within rural and remote Indigenous communities.

Jennifer Stinson, RN-EC PhD CPNP FAAN, The Hospital for Sick Children, Toronto, Ontario, Canada

Dr. Jennifer Stinson is the Mary Jo Haddad Nursing Chair, Nurse Practitioner, and Co-Director of the Pain Centre at the Hospital for Sick Children. She is a Professor at the Lawrence S. Bloomberg Faculty of Nursing and Institute of Health Policy, Management and Evaluation at the University of Toronto. Her research program focuses on co-design and evaluation of digital therapeutics for youth with painful chronic conditions as well as patient reported outcome measures. She also focuses on interprofessional pain education such as Pediatric Project ECHO that provides virtual mentorship to community healthcare providers on management of complex pediatric pain patients.

Mitchell Lunn, MD, Stanford University School of Medicine, Palo Alto, California, United States

Dr. Mitchell R. Lunn (he/him) is an Associate Professor of Medicine (Nephrology) and of Epidemiology and Population Health at Stanford University School of Medicine. He studies sexual and gender minority (SGM) health and utilizes existing and emerging technologies to characterize the health and well-being of underrepresented and vulnerable populations that face numerous health and healthcare disparities. He focuses on improving understanding of the factors that positively and negatively influence SGM health including research on SGM health disparities, SGM societal experiences, provider education about SGM health, and institutional climate towards SGM people.

Moderator:

Ligia M. Chavez, PhD, University of Puerto Rico, San Juan, Puerto Rico

Sponsored by AstraZeneca

Description:

The Cutting Edge Research plenary session featured some of the highest-ranked, innovative research from ISOQOL abstract submissions. In particular, these abstracts reflected research that truly “pushes the ISOQOL envelope” in providing new and different ways to look at quality of life.

Speakers:

Andrea Pusic, MD MHS, Brigham and Women’s Hospital, Boston, Massachusetts, United States

Scaling and Spreading Electronic Capture of Patient-Reported Outcomes through the American College of Surgeons National Surgical Quality Improvement Program for Continuous Surgical Quality Improvement

Andrea Pusic, MD, MHS, FACS, FRCSC, is Chief of Plastic and Reconstructive Surgery at Brigham and Women’s hospital and the Joseph E. Murray Professor of Surgery at Harvard Medical School. She is Director of the Patient-Reported Outcomes, Value & Experience (PROVE) Center and Medical Director of the Mass General Brigham PROMs Program, which is one of the largest PROMs collection program in the country. The patient-reported outcomes measures that she and her team have developed have been widely adopted for research and clinical care (www.qportfolio.org). Dr. Pusic has been funded by the NIH, PCORI and AHRQ. She has authored and co-authored over 300 peer reviewed papers.

Corneliu Bolbocean, University of Oxford, Oxford, United Kingdom

Being Born Early, Small or Both and Health-Related Quality of Life in Adulthood: Evidence from an Individual Participant Data Meta-analysis of Very Preterm and Very Low Birthweight Adults

Dr. Bolbocean main research areas are: patient reported outcomes measures, the economics of children’s health and trial based economic evaluation. His current research projects include:

• RECAP: This project, funded by the European Commission Horizon 2020 Programme, explores the effects of preterm birth throughout adulthood.
• Miscarriage, Self-harm and Mental Health Outcomes: This research program aims to evaluate the impact of a first miscarriage on the adverse mental health and self-harm outcomes from over 600 general practices across the UK.
• COVID-19 and HRQoL in Autistic Children and their caregivers.

Dr. Bolbocean holds academic appointments at Oxford University, and UMAS.

Manraj Kaur, PhD, Brigham and Women’s Hospital, Boston, Massachusetts, United States

How much do social determinants of health-related needs account for ethnoracial inequities in patient-reported mental health?

Dr. Manraj Kaur (she/her) is a Canadian Institutes of Health Research (CIHR) funded Postdoctoral Fellow within the Department of Surgery at Brigham and Women’s Hospital, Harvard Medical School, with expertise in the development and implementation of patient-reported outcome measures for clinical care, research and quality improvement efforts.

Carolyn Schwartz, ScD, DeltaQuest Foundation and Tufts University School of Medicine, Concord, Massachusetts, United States

When better is the enemy of good: Two cautionary tales of conceptual validity versus parsimony in clinical psychometric research

Dr. Carolyn Schwartz is President & Chief Scientist at the not-for-profit DeltaQuest Foundation; and Adjunct Research Professor of Medicine and Orthopaedic Surgery at the Tufts University School of Medicine. The 2016 recipient of the ISOQOL President’s Award, her interdisciplinary and methodological research focuses on understanding what patients can do to have an impact on the course of their disease and their well-being. In addition to development of theory and methods for response-shift research in quality of life, her work has spanned a number of diseases and conditions, and she has developed over 20 PRO measures.

Moderator:

Katrina Borowiec, MA, Boston College, Massachusetts, United States

Sponsored by Merck

Description:

In recent years, there has been a proliferation of quality of life measures. These may be novel tools to address new clinical areas, or ones which were developed to improve upon existing legacy tools. In this panel debate/discussion, presenters highlighted the good, bad, and ugly of legacy tools. Using examples, they also explored the pros and cons of moving from legacy tools to more modern ones. Regardless of which camp you may align with, all attendees were treated to a lively discussion and question and answer period.

Speakers:

Andrew Finlay, FRCP, Cardiff University, Cardiff, United Kingdom

Professor Andrew Finlay has led the development of quality of life measures in dermatology, creating the most widely used outcome measures in dermatology worldwide, including the Dermatology Life Quality Index. His current research focuses on measuring the impact of disease on patients’ family members. Professor Finlay was Head of Dermatology, Cardiff University and was President of the British Association of Dermatologists (BAD). Author of >450 publications, he co-authored the undergraduate textbook “Dermatology at a Glance” and co-edited in 2022 “Dermatology Training: the Essentials”. He was appointed CBE in 2010 and awarded the 2020 Sir Archibald Gray medal by the BAD.

John E. Ware, Jr., PhD, John Ware Research Group, Portsmouth, Rhode Island, United States

Dr. Ware is a research psychologist specializing in psychometrics who’s career began developing patient-reported outcome (PRO) measures in the 5-year randomized RAND Health Insurance Experiment and 4-year quasi-experimental Medical Outcomes Study (MOS), leading to development the MOS 36-item Health Survey (SF-36), and industry-sponsored International Quality of Life Assessment (IQOLA) Project translations of SF-36 used in multinational population surveys and clinical trials worldwide. He is an elected member of the US National Academy of Medicine and one of the most frequently cited scientists. His current emphasis is on more practical and more useful integrations of disease-specific and generic PRO results.

Moderator:

Caroline B. Terwee, PhD, Amsterdam UMC, Amsterdam, Netherlands

Sponsored by RWS

Description:

Despite the recognized importance, need, and value of patient-reported outcome measures, their implementation within health services remains challenging. This plenary session featured examples of successful implementation of PROs at the micro (patient), meso (health system), and macro (policy) levels. In addition to presenting how PROs are used, panelists highlighted the strategies that researchers, clinicians, and administrators have utilized to address barriers they encountered, as well as lessons learned along the way.

Speakers:

Stirling Bryan, PhD, Health Research BC / UBC, Vancouver, British Columbia, Canada

Stirling Bryan is a health economist with extensive experience of engagement with health policy and decision-making worlds. He is Chief Scientific Officer at Michael Smith Health Research BC, professor in UBC’s School of Population & Public Health, and senior scientist at Vancouver Coastal Health Research Institute. Before emigrating to Canada in 2008, Stirling held academic positions in the U.K. at St Thomas’ Hospital, Brunel University, and the University of Birmingham. He was a Commonwealth Fund Harkness Fellow in 2005/06, spending a year at Stanford University. In 2020 he was elected as a Fellow of the Canadian Academy of Health Sciences.

Judith Baumhauer, MD MPH, University of Rochester Medical Center, Rochester, New York, United States

Dr. Baumhauer is a tenured Professor and serves as the Dean of Academic Affairs for the University of Rochester School of Medicine and Dentistry. She is a clinically active orthopaedic surgeon. She is the Director of the Clinical Health Informatics Core for the UR Health Care System. Dr. Baumhauer is the past president of the ABOS, AOFAS, EOA and PROMIS Health Organization. Her research focuses on patient reported outcomes (PROs) in clinical decision-making. She has studied how collecting and sharing PROs affect patient engagement, patient satisfaction and clinical efficiency.

Brenda Jones, BA B.Ed, Independent – Patient Partner, Surrey, British Columbia, Canada

Brenda Jones is a retired teacher who specialized in educating students with special needs. After receiving care for health concerns, she volunteered for public speaking opportunities to share her experience. She participated, as well, on local and provincial health advisory committees and joined the Patient Voices Network in B.C.. Brenda became active as a Patient Partner in 2019 when she participated in a project to improve the use of patient-reported outcome measurement data by healthcare providers at the point-of-care. She is currently a Patient Partner in the BCIT School of Health Sciences Programs.

Moderator:

Angela Wolff, PhD RN, Associate Professor, School of Nursing, Trinity Western University, Langley, British Columbia, Canada

Description:

PROMs are widely used in mental health contexts, and they have proven to be an effective vehicle for innovative translational research and care concepts. Participants were encouraged to share experience and questions, (i) to explore successes and challenges within this area; (ii) but also to explore the innovation potential that developments in other health and service contexts could bring.

Host:

Jan R. Boehnke, PhD
University of Dundee,
Dundee, United Kingdom

Description:

Data-driven methods, such as machine learning methods and unsupervised latent variable models, are advantageous for the analysis of patient-reported outcomes (PROs) in diverse population. At this roundtable, we invited participants to discuss the potential opportunities and challenges to apply machine learning algorithms to evaluate response shift in the context of longitudinal analysis of patient-reported outcomes.

Host:

Lisa Lix
University of Manitoba,
Winnipeg, Manitoba, Canada

Description:

The use PROs in clinical practice might support more flexible and individualized follow-up activities. In remote care, PRO can be used to support clinical decisions based on PRO-based algorithms and/or assessment by a clinician. In this roundtable session, we discussed use of PRO-based algorithms to support clinical decisions, how threshold can be defined and evaluated, and benefits and challenges by using PROs in remote care. Roundtable participants were also invited to provide their perspectives on this topic.

Host:

Liv Marit Valen Schougaard
AmbuFlex – Centre for Patient-reported Outcomes, Gødstrup Hospital, Herning, and Department of Public Health, Aarhus University,
Aarhus, Denmark

Description:

This interactive roundtable invited participants to discuss ways to improve the quality of PRO utilization in clinical trials, from protocol development and PRO selection through analysis, reporting and dissemination. Existing documents that guide and support quality PROs were discussed, as were resources from the PROTEUS Consortium that are designed to facilitate using these guides.

Host:

Michael Brundage, MD MSc
Queen’s University,
Kingston, Ontario, Canada

Description:

Involving patient advocates in the research process is recognised as a central component of health research and an important step towards person-centered care. One of the key principles of creating meaningful partnerships is to avoid just ticking the box and in achieving this, understanding the misconception that one size fits all. This roundtable discussed practical strategies and lessons learned in engaging well with patient advocates.

Host:

Leonie Young
Australia & New Zealand Urogenital & Prostate Cancer Trials Group,
Brisbane, Australia

​Description:

In this interactive roundtable session we discussed several topics related to being a New Investigator at ISOQOL. Attendees were encouraged to ask questions and share their thoughts about for example; how they heard about ISOQOL, how you can get the most out of your ISOQOL membership, or how you get involved in committees and Special Interest Groups. And of course there was plenty of room for attendees to share their experiences with being an ISOQOL member and performing Quality of Life research as a New Investigator.

Host:

Maud van Muilekom, PhD
Emma Children’s Hospital, Amsterdam UMC, University of Amsterdam, Department of Child and Adolescent Psychiatry & Psychosocial Care,
Amsterdam, Netherlands

Description:

Quality of life in Developing Nations is relative, varying from person to person, culture to culture. Participants were encouraged (i) to share experiences and questions, and (ii) to explore challenges and opportunities within this area. In terms of QOL measurements in Developing Nations, is what is good for the goose, good for the gander?

Host:

Ezinne Ekediegwu, B.M.R (Hons.) M.Sc(Hons.)
Nnamdi Azikiwe University, Nigeria.
McGill University, Montreal, Quebec, Canada

Sponsored by Merck

Moderator and Discussant:

Leah Mcclimans, University of South Carolina, Columbia, South Carolina, United States

Symposium Synopsis:

To be impactful, HRQoL measurement must be relevant and meaningful, as this year’s conference theme suggests. But relevant to whom? And for what? Even when limited to the considerations of day-to-day patient care, tensions emerge. Most agree that HRQoL measures should be relevant to patients. But they also need to be relevant to the service providers who use them and the clinical and research contexts where they are applied. In this panel we explored these tensions, which often are presented as barriers to relevance, and offered some practical solutions for research, practice and policy.

HRQoL research is full of philosophical questions and relevance is one of them. This panel brought together three philosophers and two HRQoL researchers to present an interdisciplinary dialogue addressing conceptual questions using quantitative, qualitative, philosophical and historical approaches. Many HRQoL researchers have philosophical questions, but don’t know how or where to ask them. The papers in this panel provided a rare opportunity where ISOQoL members can think about conceptual questions of relevance, for example, how can clinicians ascertain the clinical relevance of measures for their practice? How do questions of relevance intersect with questions of validity? What should be done about the often idiosyncratic relevance judgments of patients? The four presentations in this panel addressed these questions.

The first presentation drew on a philosopher’s observations during the development of a HRQoL measure of older adults living actively in the community (OPAL). It explored the iterative development of this measure as experts strived to make it relevant to the measurement population and context of use. The second presentation specifically addressed the meaning of scores to patients. It suggested a tension between patients’ meanings and administrators standardized interpretation and inferences. Our third presentation focused on the relevance of HRQoL measures to the clinicians who use them. It presented a training program designed for clinicians who seek to measure youth mental health. Finally, our fourth presentation addressed the relevance of reconceptualizing HRQoL measures in terms of clinimetrics or psychometrics.

Individual Presentations:

Embedded philosophy in the design of an HRQoL instrument: What is ‘Fitness for Purpose’?

Sebastian Rodriguez Duque, PhD Candidate, McGill University, Montreal, Quebec, Canada

“I feel 4 out of 5 depressed” – Meaningful measurement of mental health

Femke Truijens, Dr, Erasmus University Rotterdam, Rotterdam, Netherlands

Addressing barriers to effective measurement of youth mental health in clinical settings

Eran Tal, PhD, McGill University, Montreal, Quebec, Canada
Skye Barbic, PhD, University of British Columbia, Vancouver, British Columbia, Canada

“Clinimetrics” or “Clinical Psychometrics”?: Reexamining disciplinary boundaries and methodological commitments

Rebecca Jackson, MA, Indiana University – Bloomington, Bloomington, Indiana, United States

Moderator:

Daniel Serrano, The Psychometrics Team, Sheridan, Wyoming, United States

Discussant:

M. Chris, Runken, PharmD, Grifols Inc., Durham, North Carolina, United States

Symposium Synopsis:

Alzheimer’s Disease (AD) is a neurodegenerative disorder progressing from mild cognitive impairment (MCI) to mild, moderate, and severe AD. Each stage is characterized by differential cognitive impairments and functional consequences. Notably, in the past decade every Phase 3 AD trial has “failed.” Explanations for “failure” include COA endpoints being insufficiently sensitive to detect change in early AD stages (where Phase 3 trials increasingly focus); and efficacy analyses not accounting for inter- and intra-individual heterogeneity in cognitive and functional performance. These factors directly influence the probability of detecting treatment benefit when one exists and the valuation of an efficacious therapy.

The recent divide between industry expectations and CMMS adjudicated reimbursement for Aducanumab reveals the need for valuation methods that use objective and empirical parameters. Valuation differences may be reconciled by linking AD therapy value to avoided costs that would otherwise occur in the absence of therapy from more rapid disease progression.

This symposium addressed measurement and modelling theory specific to AD progression with supporting evidence relevant to detection of treatment efficacy (presentation 1 and 2) and valuation of effective therapies (presentation 3).

Presentation 1 examined heterogeneity in sensitivity to detecting change in primary outcomes for progression between and within AD stages that can impact detection of treatment efficacy.

1. Variation in sensitivity of stage-specific COA endpoints were examined with methods to mitigate this effect in a power-preserving paradigm.
2. Impact: Change in cognitive and functional outcomes from non-significant to significant.

Presentation 2 addressed impact of unobserved heterogeneity in COA outcomes on estimated treatment efficacy, demonstrating:

1. Evidence for inter- and intra-individual AD heterogeneity, grounded in the literature, and use of mixture models to mitigate the effect of unobserved heterogeneity in AD.
2. Impact: Change in cognitive and functional outcomes from non-significant to significant.

Presentation 3 presented a paradigm for empirical valuation of efficacious AD therapies, focusing on:

1. COA-based time-to-event evaluation of the time to cognitive and functional endpoint thresholds associated with substantial home care needs and care facility admission.
2. Impact: Change in cognitive but not functional outcomes from non-significant to significant.

Individual Presentations:

The application of a novel endpoint staging framework to demonstrate plasma exchange with albumin (PE-A) treatment efficacy in Alzheimer’s Disease: a post-hoc evaluation

Lauren Podger, MSc, OPEN Health Group, London, United Kingdom

The use of latent growth mixture models to demonstrate plasma exchange with albumin (PE-A) treatment efficacy in Alzheimer’s Disease

Nicolai D. Ayasse, PhD, OPEN Health Group (at the time the research was conducted), Parsippany, New Jersey

A time-to-event analysis to highlight potential cost-saving benefits of plasma exchange with albumin replacement (PE-A) treatment in Alzheimer’s Disease

Shauna McManus, OPEN Health Group, Parsippany, New Jersey, United States

Moderator:

Felix Fischer, PhD, Charite – Universitätsmedizin Berlin, Berlin, Germany

Symposium Synopsis:

In the past decade, Bayesian methods have been increasingly utilized to analyze patient reported outcome data. The reasons for this trend include: 1) Bayesian modeling is extremely flexible and versatile (e.g., one can fit models not supported by standard statistics software); 2) available evidence can be updated with new data in a straightforward way; and 3) it can accommodate complex data structures (e.g., patients nested within doctors); and 4) Bayesian inference is less prone to problems in conventional statistical inference.

The goal of this symposium was to highlight the aforementioned advantages of Bayesian methods for applications in quality of life research. Following a short introduction to the Bayesian framework, we presented recent works using Bayesian approaches to overcome limitations of classical statistical approaches, with an emphasis on their relevance in quality of life research.

The symposium started with a short introduction highlighting the key concepts of Bayesian data analysis provided by the discussant (Andrew Trigg). The first presentation (Aaron Kaat) covered strategies to model multidimensional, correlated outcomes in a random effects model. In the second talk we presented a Bayesian approach to assess meaningful change over time (Yuelin Li). In the third talk, we discussed approaches to estimate prevalence accounting for imperfect diagnostic accuracy of questionnaires often used in population-based studies (Felix Fischer). We concluded the session with an open Q&A with all three symposium presenters and the discussant, where we aimed to specifically discuss the relevance of the proposed methods for quality of life research and analysis of patient reported outcomes.

Individual Presentations:

Fundamentals of Bayesian analysis

Contributor: Andrew Trigg, MSc, Bayer Plc, Reading, United Kingdom

Multilevel Multivariate Bayesian Regression

Aaron Kaat, PhD, Northwestern University, Chicago, Illinois, United States

Estimating prevalence with depression screening tools

Felix Fischer, PhD, Charite – Universitätsmedizin Berlin, Berlin, Germany

Meaningful Change in Quality Of Life using Bayesian Posterior Predictive Distribution

Yuelin Li, PhD, Department of Psychiatry & Behavioral Sciences; Department of Epidemiology and Biostatistics, Memorial Sloan Kettering Cancer Center, New York, New York, United States

Moderators:

Jeanette Jackson, Health Quality Council of Alberta, Calgary, Alberta, Canada
Chyloe Healy, Blackfoot Confederacy Tribal Council, Calgary, Alberta, Canada
Kristina Watkins, Health Quality Council of Alberta, Calgary, Alberta, Canada
Rhonda Pouliot, MA BSc MLT, Health Quality Council of Alberta, Calgary, Alberta, Canada
Richard Sawatzky, PhD, Trinity Western University, Langley, British Columbia, Canada

Symposium Synopsis:

Ongoing colonialism continues to impact the wellness of Indigenous Peoples and constructs barriers to accessing traditional health practices. Further, the lack of diverse Indigenous voices and methodologies in mainstream measurement contribute to health inequities. This symposium aimed to share four distinct Nation and place-based perspectives and approaches to measuring the unique health and wellbeing needs of Indigenous Peoples. The symposium exposed gaps in measurement and measurement methodologies that challenge the assumptions of the dominant systems perspectives. Recent calls for eradicating Indigenous-specific racism and discrimination in the health system have created space for Indigenous people to join the conversation on providing culturally safe measurement. Indigenous led and lens measurement highlights measures and indicators relevant to Indigenous Peoples’ wellness providing contextual strengths-based solutions.

The four invited presenters shared their perspective (15 minutes each):

1. Wise Practices – from Indigenous lens to Indigenous led
2. Co-endeavour, building Māori Health Equity Report of New Zealand Health System Quality Dashboard
3. Decolonizing Birth Research: Indigenous-led research centering lived expertise and equity in perinatal services)
4. Supporting the Trajectory of our Spirit: Living the Cree Pimatisiwin and Blackfoot Kipaitaipiiwahsinnooni – “Our Spiritual Way of Life”

The symposium moderators then guided a robust discussion to inspire ongoing conversations about thinking and doing measurement differently. It is paramount to self-reflect and keep in mind, who is telling the story to initiate more responsive changes for those who have been historically excluded from the dominant systems perspectives. Both, Indigenous led and Indigenous lens measurement approaches offer a more wholistic story that is currently missed.

At a critical time in our Reconciliation journey with our Indigenous people here in Canada, we saw this as a unique opportunity to welcome our international peers in a good way to this topic and create a safer space for conversation. To enable a fulsome experience for everyone joining the symposium and increased appreciation for the context of this topic, we brought in ceremony, as well as allocated more time for discussion.

Individual Presentations:

Wise Practices – Moving Patient-Centred Measurement from Indigenous-lens to Indigenous-led

Mark Matthew, BBA, British Columbia Patient Safety & Quality Council, Kamloops, British Columbia, Canada

Co-endeavour, building Māori Health Equity Report of New Zealand Health System Quality Dashboard

Ying Li, MSc. major in Applied Statistics, Health Quality & Safety Commission New Zealand, Wellington, New Zealand
Patricia Keelan, BA Public Policy, Te Pinakitanga ki te ao Kairangi, Executive MBA, Current PhD Student – Indigenous Futures, Ngāti Porou, Te Aupouri, Tūranga/Gisborne, New Zealand

Decolonizing Birth Research: Indigenous-led research centering lived expertise and equity in peri-natal services

Wanda Phillips-Beck, PhD, First Nation Health and Social Secretariat of Manitoba, Winnipeg, Manitoba, Canada
Contributor: Saraswathi Vedam, RM PhD FACNM Sci D (hc),  Birth Place Lab, Vancouver, British Colombia, Canada

Supporting the Trajectory of our Spirit: Living the Cree Pimatisiwin and Blackfoot Kipaitaipiiwahsinnooni – “Our Spiritual Way of Life”

Chyloe Healy, Blackfoot Confederacy Tribal Council, Calgary, Alberta, Canada

Moderator:

Elizabeth Unni, Touro College of Pharmacy, New York, New York, United States

Symposium Synopsis:

Patient reported outcome measures (PROMs) assess a patient’s health-related quality of life. It is now clear that PROMs can improve the effectiveness of therapeutic interventions and the efficiency of the healthcare system. As a result of these research advances, the use of PROMs in clinical settings has proliferated. Central to the optimal use of PROMs is the completion of the PROMs by the patients. Many healthcare institutions offer training programs to clinicians to understand the use and meaning of PROMs. Less attention has been given to educating patients and their caregivers about PROMs, which can jeopardize optimal co-production of better health-related quality of life. Literature review demonstrates that more work is needed to educate patients about PROMs. Compared to the few patients who are involved in the PROM development, a much larger number will be asked to complete the PROMs on a regular basis. Educating patients in the use of PROMs in clinical care is the key to ensure that their use is relevant to patients. Training and education materials should be co-designed with them.

This symposium aimed to explore solutions and answers to the following questions:

1. What evidence exists regarding educating patients on PROMs?
2. How have patients been educated in PROMs completion?
3. How can we engage patients in creating education programs for patients regarding PROMs
4. How can patients be educated in using these outcomes in their discussions with the healthcare providers?
5. What are patient’s experiences using PROMs in clinical care?

Each presenter discussed these questions in a case study format from three countries, both adults and pediatrics. The symposium started with a short overview of the evidence, followed by case studies, and ended with presentations by patients and families describing their personal experiences in understanding the PROMs’ use in their routine clinical care. The symposium provided an opportunity for attendees to consider and discuss the best approaches in educating patients about PROMs in routine clinical care.

Individual Presentations:

Engaging Patient and Caregivers in Using Patient-reported Outcomes Measures in Paediatric Clinical Care

Maria Jose Santana, PhD MPharm, Associate Professor, Patient and Family-centred care Scientist, Departments of Paediatrics and Community Health Sciences, Cumming School of Medicine, University of Calgary and Provincial Lead, Patient Engagement, Alberta Strategy for Patient Oriented Research Unit
Calgary, Calgary, Alberta, Canada
Bishnu Bajgain, University of Calgary, Calgary, Alberta, Canada

Engaging and educating patients about PROMs in clinical practice: experiences from the Netherlands

Maud van Muilekom, PhD, Emma Children’s Hospital, Amsterdam UMC, University of Amsterdam, Department of Child and Adolescent Psychiatry & Psychosocial Care, Amsterdam, Netherlands

Educating patients in the use of PROMs in clinical practice: Experiences from the adult cancer setting

Kate Absolom, PhD, University Academic Fellow, Leeds Institute of Medical Research/Leeds Institute of Health Services Research, University of Leeds, Leeds, United Kingdom

Moderator:

Tom Willgoss, Roche, Welwyn Garden City, United Kingdom

Symposium Synopsis:

The past 10 years has seen a booming interest in the inclusion of qualitative methods within clinical trials. During this period our application of methods has evolved and improved, and the research questions and insights have broadened in scope. Today, in-trial interviews have become commonplace in interventional clinical trials and their value is of interest to a broad range of stakeholders.

Despite the growing interest, in-trial interviews are not without challenges. These include choosing inappropriate methods for the research question, practical barriers at clinical trial sites, and compliance hurdles. This symposium discussed the state-of-the-art of in-trial qualitative methods based on a decade of learning. In addition to showcasing innovative approaches and use cases from real-world examples, we also critically evaluated the value of in-trial methods in a range of settings.

Each presenter shared and reflected on their experiences of designing and implementing in-trial interviews, and shared recommendations on how to optimize this approach. We concluded the symposium with a panel discussion and Q&A to ensure maximum audience participation.

Individual Presentations:

When are in-trial interviews needed and not (only) exit interviews?

Helen Kitchen, MSc, Clarivate, London, United Kingdom

Qualitative estimates of within-person meaningful change on novel Clinical Outcome Assessment (COA) measures from exit interviews – is it really useful?

Robert Krupnick, PhD, IQVIA, Boston, Massachusetts, United States

Looking beyond exploring change in interviews alongside late phase clinical trials

Claire Burbridge, MSc, Clinical Outcomes Solutions, Folkestone, United Kingdom

Design Considerations for Successful In-Trial Interviews: The Who, What, When, Where, and How

Dana B. DiBenedetti, PhD, RTI Health Solutions, Research Triangle Park, North Carolina, United States

Moderator:

Arto Ohinmaa, University of Alberta, Edmonton, Alberta, Canada

Symposium Synopsis:

There is a considerable body of evidence and relative consensus on measuring and valuing health in adult population for various purposes including economic evaluation and applications in health technology assessment. However, such evidence and consensus are lacking in younger populations including children and adolescents. Measuring and valuing health in these population groups poses various challenges, starting with the conceptualization of health and health-related quality of life, to methods used to measure these constructs, as well as methods used to value health states. Children go through different developmental stages from newborns and infants up to adolescents and young adults and these changes need to be considered both in the development of the instrument questions, how the data is collected (proxy and self responses), and what valuation methodologies are used.

The EuroQol Group has led a significant amount of research for over a decade to examine methods of measuring and valuing health, which started with the launch of the EQ-5D-Y-3L in 2010. The group continues to develop and test new measures for pediatric use, including the EQ-5D-Y-5L, and a new tool aimed at measuring and valuing health in infants and toddlers, EQ-TIPs.  The group has also explored ways of valuing health in children: a valuation protocol for the EQ-5D-Y-3L published in 2020 has successfully been used to produce value sets around the world. Important methods issues remain e.g. how to interpret different characteristics of values for child and adult EQ-5D instruments? And how/whether to value HRQoL for very young children? That is how to incorporate pediatric HRQoL measurement and valuation in the lifespan measurement of EQ-5D.

In this symposium, the speakers provided the current state of play in measuring and valuing health using EuroQol instruments, discussed key challenges and opportunities in advancing the measurement and valuation of health and validation of these instruments in children across a range of diseases, and shared examples from large projects being conducted around the world. All the speakers have been extensively involved in the development and testing of methods to measure and value health in children and were currently leading international projects in this area.

Individual Presentations:

Measuring health in younger populations: EQ-5D-Y and beyond

Michael Herdman, MSc, Insight Consulting & Research, Mataró, Spain

Methods for valuing child HRQoL: value sets for EQ-5D-Y-3L and related methods challenges

Nancy Devlin, BA(Hons) PhD, University of Melbourne, Melbourne, Australia

Validating the EQ-5D-Y-5L in childhood arthritis: the UCAN precision health study

Deborah Marshall, PhD, University of Calgary, Calgary, Alberta, Canada

Entering the child’s mind: qualitative and quantitative evidence in valuing youth health in Canada

Feng Xie, PhD, McMaster University, Hamilton, Ontario, Canada

Sponsored by EORTC

Moderator:

Amylou Dueck, PhD, Mayo Clinic, Scottsdale, Arizona, United States

Contributor:

Madeline Pe, PhD, EORTC Quality of Life Department, Brussels, Belgium

Discussant:

Michael Brundage, Queen’s Cancer Research Institute, Kingston, Ontario, Canada

Symposium Synopsis:

The availability of reliable and valid HRQOL instruments marked a critical milestone in the cancer clinical trial field. These measures were developed with a strategy of having a core questionnaire that can be supplemented with disease-specific and/or treatment-specific modules. Although this approach advanced the use of HRQOL instruments in clinical trials, there were limitations to this approach. Because of their “static” nature, patients are asked the same set of questions regardless of their disease stage, therapies used or relevance of the items to their current state. The inability to tailor these “static” HRQOL questionnaires to the needs of specific cancer clinical trials and/or individual patients led to the development of PRO item libraries that allows investigators the flexibility of selecting the HRQOL domains of interest and the type of form to administer to assess treatment impact in terms of benefit of safety.  Item banks and computerized-adaptive testing (CAT) methods are considered part of the broader PRO Item Libraries.

This symposium provided an overview on how the development of HRQOL instruments in the oncology field evolved over time. We contrasted the traditional versus PRO item library when implementing HRQOL measures in cancer clinical trials. A practical example of a modern, balanced, approach to selecting HRQOL instruments in cancer clinical trials was provided.

Individual Presentations:

The development of HRQOL instruments in oncology: the static approach

Claire, Piccinin, MSc, Quality of Life Department, European Organisation for Research and Treatment of Cancer, Brussels, Belgium

Best Practices for Selection of HRQOL Item Libraries in Oncology Trials

Bryce Reeve, PhD, Duke University School of Medicine, Durham, North Carolina, United States

Real-world Example Demonstrating the Use of HRQOL Item Libraries in an Oncology Trial

Amylou Dueck, PhD, Mayo Clinic, Scottsdale, Arizona, United States

Moderators:

Sandra Zelinsky, University of Calgary, Canada
Sadia Ahmed, Master of Science, University of Calgary, Canada
Natasha Roberts, The University of Queensland, Australia

Speakers:

Natasha Roberts, University of Queensland, Australia
Sandra Zelinsky, University of Calgary, Canada
Sadia Ahmed, Master of Science, University of Calgary, Canada
Leonie Young, Australia & New Zealand Urogenital & Prostate Cancer Trials Group, Australia (Patient Research Partner)
John J. Librett, Survivor Healthcare, United States
Jessica Gentile, PharmD MPA, RPh PA-C, Survivor Healthcare, United States
Maria J. Santana, University of Calgary, Canada
Sumedh Bele, PhD MBBS MPH, University of Oxford, United Kingdom

The aim of this educational session was to explore patient engagement in quality of life research internationally. Presenters included clinicians, researchers, and patient research partners who shared their projects and perspectives on how patient engagement is carried out in quality of life research.

Moderator:

Donna Rowen, University of Sheffield, United Kingdom

Speakers:

Ava Mehdipour, McMaster University, Canada
Ana Moga, PT MSc PhD-c, McGill University, Canada
Philip Powell, PhD, University of Sheffield, United Kingdom
Sarah Knight, MSc, Clarivate, United Kingdom
Heather Gelhorn, PhD, Evidera, United States
Jiat Ling Poon, PhD, Eli Lilly and Company, United States
Mehmet Inceer, PhD, IQVIA, Canada

The role of health preference research (HPR) in PROM development, validation and refinement is emerging, and the use of HPR to enable PROMs to inform healthcare decision-making and clinical practice is rapidly expanding globally. This symposium provided case studies covering HPR in Quality of Life PROM development and testing and the use of HPR to enable PROMs to inform decision-making and clinical practice. The session was intended for anyone interested in HPR or curious to learn how HPR use can leverage and enhance the use of PROMs. The session combined presentations (8-9 minutes per speaker) from 8 speakers with audience discussion, and shared the international work of HPR SIG members from academia and industry.

Sponsored by Merck

Moderator:

Ana Maria Rodriguez-Leboeuf, PhD MSc PT, McGill University and IQVIA, Canada

Speakers:

Konrad Maruszczyk, University of Birmingham, United Kingdom
Norah L. Crossnohere, PhD, The Ohio State University College of Medicine, United States
Paul Williams, MPH, IQVIA, France
Ana Maria Rodriguez-Leboeuf, PhD MSc PT, McGill University and IQVIA, Canada

As real-world evidence (RWE) is increasingly used to inform decision making, the importance of guidance on the use, analysis, and interpretation of PRO data in RWE is growing. Guidance documents have been in circulation since 2005 to delineate the development, use, and interpretation of PRO data in the context of clinical trials for submission to regulatory authorities, but how this guidance adapts to a RW setting has not been explicated. This symposium provided case studies and considerations covering RWE uses of PRO, and differences with their use in clinical trials that may require special considerations to be used to inform decision-making and clinical practice. The session was intended for anyone with or without industry experience who may be interested in the transition of PRO evidence from clinical trial to RWE research and practice. The session combined presentations from 4 speakers with audience discussion.

Workshop level: Basic

Workshop Goals:

The recent proliferation of personal mobile technologies allows researchers and practitioners to collect objective patient behavioral and health data at increased granularity. That can be done in real-time and in context, longitudinally, and may also reduce patient burden. Collecting and acting upon these co-called Technology-Reported Outcomes (TechROs) will become a cornerstone of patient-centered care. TechROs can assess individual characteristics, biological and physiological variables, physical, psychological, social/role functioning, and environmental variables. Acquiring the data via the TechROs can be challenging as there are many options, each with advantages and disadvantages, in addition to the many operational and human factors influencing the collected TechROs data quality. The same applies to selecting the right methods for analysis to derive meaningful results from the collected data.

Intended Audience

• Health professionals
• Medical scientists who are not experts in the use of TechROs
• Consultants
• Pharmaceutical and medical device representatives
• New investigators and research students
• Policymakers
• Other associations and individuals who are interested in acquiring familiarity with the TechRO research terms and methodology
• Patients and caregivers

Presentation Format

This full-day, intensive and interactive educational workshop offered a curriculum that provided a basic level introduction to the why and how of using TechROs in research, which later can pave the way for clinical practice. Initial lectures presented how, when, and with which devices TechROs can be collected in order to empower their research in many domains. Attendees were also given the opportunity to apply their learning throughout the course (hands-on learning). Specifically, they first played with real-world collected TechROs data and assessed causality using a state-of-the-art method developed for the simulation of a randomized control trial from longitudinal observational data. After, they were given the opportunity to collect and analyze their own TechRO datasets, by wearing a wearable (provided by the workshops authors) throughout the conference duration (optional).

Overview/Outline

The workshop presented how, when, and with which devices TechROs can be collected to empower research. It did so by bringing together academic and industry experts to discuss TechROs future. Additionally, attendees learned how to assess causality from longitudinal observational data collected with those TechROs.

Learning Objectives

1. Participants learned about the rationale and value of incorporating TechROs into QoL assessment through an understanding of the definition, history, types, and different efficacy of technologies in assessing QoL. Additionally, they had an understanding of typical human and operational factors influencing the TechRO data quality.
2. Participants learned about the different quantitative and qualitative methods of analyzing data collected with TechROs, using powerful statistical and machine learning methods.
3. Participants were guided step-by-step on a hands-on basis on how to implement and interpret results acquired with a novel method for causality estimation of real-life collected longitudinal TechROs data. Furthermore, participants were given a wearable device for usage during the conference days to analyze their own data with that same method after the event ends.

Organizers:

Katarzyna Wac, Prof. Dr., University of Geneva, QoL Lab, Geneva, Switzerland

Igor Matias, MSc, University of Geneva, QoL Lab, Geneva, Switzerland

Clauirton Siebra, PhD, University of Geneva, QoL Lab, Geneva, Switzerland

Contributors:

• Eric J. Daza, Dr.P.H., Evidation, San Mateo, United States
• Paul De Villèle, PhD, Withings, Paris, France

Workshop level: Advanced

Workshop Goals:

The aim of the interactive workshop was to provide participants with foundational knowledge, skills, and resources to prepare and motivate healthcare providers (HCPs) to routinely use PROMs in direct care (i.e., preventive care, chronic care, population-based care). The Consolidated Framework for Implementation Research guided the workshop to illustrate the intersection between the “user” and “implementation process” domains.

Intended Audience

Are you currently planning the implementing PROMs in patient care areas? Have you implemented PROMs by require sustainability assistance? This workshop was for you! To get the most from this workshop, participants should have had an intermediate knowledge level about PROMs in clinical practice and the overall implementation process (e.g., selected PROMs and/or implementation experience). Having an implementation case/project to design implementation supports (e.g., training, practice supports, interpretation tools, and decision-making aids) was beneficial.

Presentation Format

50% lecture; 35% interactive activities; 15% admin (intro, breaks, conclusion).

Overview/Outline

HCPs are often the lynchpin to the successful implementation of PROMs to engage patients in their care. Although using PROMs in practice is becoming more prevalent the preparation of HCPs is not standardized and/or limited. Facilitating HCP adoption of PROMs is a complex behavior change, often necessitating flexible and tailored strategies to suit the local context and current workflow. To facilitate their incorporation into routine clinical care, several questions need to be answered: What knowledge do clinicians need to use PROMs on a regular basis? How can clinicians be motivated and engaged? What evidence-based strategies could be used to assist clinicians? How do the needs of clinicians vary for ongoing and sustained use? How do clinicians visualize PROMs data and interpret scores (e.g., initial scores and meaningful change)? How can clinicians use the PROM score data/feedback into inform patient care (e.g., shared decision making and planning care)? By answering these and many other questions you may have, we explored the multifaceted aspects of HCP behavior change by showcasing theory-informed approaches and best practice from current implementation projects in diverse clinical settings. This workshop provided participants with the opportunity to apply the theory/knowledge to their own context to address the needs of HPCs and specific barriers to PROMs implementation.

Learning Objectives

To implement patient-reported outcome measures (PROMs) into daily clinical care, this interactive workshop focused on the needs of healthcare providers (HCPs) for both initial and sustained. Moreover, we focused on the specific details to address that are often overlooked or unforeseen. The objectives were to:

1. Illustrate the knowledge needed to and attitude/beliefs required for optimal application of PROMs to meet the specific needs of clients for preventive care (screening), chronic care (monitory symptoms and functions), and population-based care (interventional care such as oncology and surgery).
2. Demonstrate the skills needed to (a) collect PROMs data (e.g., fill out the PROMs questionnaire), (b) discuss results with patients, (c) score and interpret PROMs, and (d) incorporate PROMs data into clinical reasoning and shared decision making with patients.
3. Evaluate the implementation of PROMs by HCPs (process and outcomes) and identify evidence-based strategies to provide ongoing support for adoption.

Organizers:

Angela Wolff, PhD RN, Associate Professor, School of Nursing, Trinity Western University, Langley, British Columbia, Canada

Elizabeth Austin, PhD MPH, Sr. Research Scientist, Department of Health Systems and Population Health, School of Public Health, University of Washington, Seattle, Washington, United States

Maria Jose Santana, PhD MPharm, Associate Professor, Patient and Family-centred care Scientist, Departments of Paediatrics and Community Health Sciences, Cumming School of Medicine, University of Calgary and Provincial Lead, Patient Engagement, Alberta Strategy for Patient Oriented Research Unit, Calgary, Alberta, Canada

Kate Absolom, PhD, University Academic Fellow, Leeds Institute of Medical Research/Leeds Institute of Health Services Research, University of Leeds, Leeds, United Kingdom

Sara Ahmed, PhD PT, Professor, McGill University, Faculty of Medicine and Health Sciences, McGill University Health Center, Clinical Epidemiology, Montreal, Quebec, Canada

Susan Bartlett, PhD, Professor, McGill University, Faculty of Medicine and Health Sciences; Senior Scientist McGill University Health Center, Montreal, Quebec, Canada

Contributor:

• Brenda Jones, BA B.Ed, Independent – Patient Partner, Surrey, British Columbia, Canada

Workshop level: Basic

Workshop Goals:

Health-related quality of life (HRQL) research has developed into a highly interdisciplinary field, embracing a wide variety of methodologies and applied contexts. It has therefore become increasingly difficult for researchers and practitioners to navigate this landscape when planning their own research agendas and projects. This workshop aimed to help investigators: (i) understand the core methodological standards for health outcomes research; (ii) outline common researcher pitfalls that delay or prevent their research from being published and their solutions; and iii) know about resources that support the development of such research and publications.

Intended Audience

The intended audience was people working with HRQL and patient reported outcomes (PROs) in evaluation or research contexts, developing their own research (programmes) and wanting to get their HRQL/PRO research published. The workshop was targeted at early to mid-career clinicians and researchers and any investigators new to HRQL/PROs.

Presentation Format

40% lecture, 40% hands-on learning, 20% question and answer.

Overview/Outline

The workshop provided an overview of key considerations for doing good HRQL/PRO research, exploring theoretical and practical issues such as attention to the objectives, research methods, measures, and statistical analysis during protocol development, considerations during study conduct to ensure quality PRO data, and clearly and effectively presenting results to aid interpretation.

The workshop relied on practical and interactive group work, utilising existing guidelines and resources developed to help navigate the use of PROs in clinical trials and clinical practice and to support good scientific practice. As an additional pragmatic perspective to help identify avoidable inappropriate conduct and dissemination of research, we introduced concepts from the “research waste” literature, covering formulating research questions, quality research methods, adequacy of publication practices, and quality of research reports.

The later part of the workshop was organised as a research clinic, drawing on methods used in research speed dating and for concept development. Participants each pitched an idea and concern area. Depending on where participants were at in the research process, we worked towards either turning ideas into a research study or answering specific questions.

Learning Objectives

1. Appreciate the value of patient reported outcomes (PROs) as outcome measures and the importance of collecting quality PRO data.
2. Understand key considerations for doing good HRQL/PRO research including accepted community standards and practices and approaches to support the planning, conduct and reporting of HRQL/PRO research.
3. Be aware of common researcher pitfalls that lead to research waste, questionable research practices, and failure to publish HRQL/PRO research and their solutions.

Organizers:

Jan R. Boehnke, PhD, School of Health Sciences, University of Dundee, Dundee, United Kingdom

Claudia Rutherford, PhD, Faculty of Medicine and Health, University of Sydney, Sydney, Australia

Workshop level: Basic

Workshop Goals:

This workshop built on a similar workshop presented at the 2022 ISOQOL conference. The overall workshop goal was to equip participants with information and skills about progress in modern validity testing practice in relation to patient-reported outcomes (PROs). The focus this year was on Kane’s argument-based approach to validation, which requires the development of a clear interpretive argument (a rationale) about the intended interpretation and use of an instrument’s scores, and evaluation of a range of evidence to assess the plausibility of the interpretive argument, specifically, the assumptions underpinning it. The 2014 Standards for Educational and Psychological Testing (the Standards) provides the framework of five sources of validity evidence for structured validation practice. Specific goals were to:

1. Describe Kane’s approach and the Standards’ framework
2. Contrast the notion of validation as a property of an instrument with the validation practice of accumulating evidence about an interpretive argument
3. Outline assumptions that are common to interpretive arguments in health research
4. Demonstrate when and why validity evidence should be included in a validation plan, including framing research questions, identifying relevant qualitative and quantitative methods, and reporting validity evidence

Intended Audience

All professionals involved in developing, choosing, and using PROMs.

Presentation Format

30% presentations, 40% hands-on learning, 30% group discussion and questions.

Overview/Outline

Short presentations provided background about the argument-based approach to validation and modern validity testing practice and addressed the imperative to advance validation practice in PRO measurement.

Descriptions of the five sources of validity evidence and interpretive arguments, including common assumptions, were provided with examples of research questions and evidence generation methods. Emphasis was on minimising measurement biases and ensuring accurate, useful (actionable), and equitable PRO measurements in diverse populations.

Based on a provided interpretive argument, small groups built validation plans by developing research questions for the common set of assumptions and identifying relevant sources of validity evidence, including methods for investigation. Existing case studies were used as examples.

A plenary group discussion focused on reporting validity evidence and considered how evidence synthesis informs the plausibility of interpretive arguments.

Learning Objectives

1. Be able to describe the argument-based approach to validation and the sources of validity evidence.
2. Be able to formulate a validation plan to answer research questions related to the kinds of evidence needed to investigate assumptions underpinning an interpretive argument.
3. Be able to choose appropriate qualitative and quantitative methods to generate evidence to ensure accurate, actionable, and equitable PRO measurements, and craft a reporting format for presenting the five sources of validity evidence.

Organizers:

Melanie Hawkins, PhD, Swinburne University of Technology, Melbourne, Australia

R.J. Wirth, PhD, Vector Psychometric Group, LLC, Chapel Hill, North Carolina, United States

Scott Gill, Swinburne University of Technology, Melbourne, Australia

Contributor:

• Sandra Nolte, Charité Universitätsmedizin Berlin, Berlin, Germany

Workshop level: Basic

Workshop Goals:

This workshop provided participants with knowledge on the latest research on ethical and equity, diversity and inclusion (EDI) issues in the context of PRO data collection, clinical research, drug development, and clinical practice and enabled:

1. Insights on ethical and EDI issues in PRO data collection, in clinical research, drug development, and clinical practice based on current evidence;
2. Understanding and discussion of minimal ethical and EDI considerations as well as rationale;
3. Application of best practices and practical case studies for inclusion of ethical and EDI considerations.

Intended Audience

A basic knowledge of PROs is assumed. The workshop was intended to academics, members of ethical review boards, researchers from industry and those stakeholders working in PRO data collection, clinical research, drug development, and clinical practice interested in improving the ethical and equity acceptability of PRO research.

Presentation Format

Initially, the workshop introduced the PRO ethics guidelines and EDI considerations in PRO data collection, clinical research, drug development, and clinical practice. This was supported by best practices and practical case studies (50%).

Two activities were planned 1) small group discussions focused on PROs data collection, clinical research, clinical practice and industry will be held to discuss specific PRO ethics and EDI considerations; and 2) a group discussion/exercise based on the Implementation Tool for PRO Researchers and Research Ethics Committess included in the PRO Ethics guidelines and EDI considerations (40%). Participants had the opportunity to discuss with other delegates and ask questions to the facilitators (10%).

Overview/Outline

PROs can inform clinical research and regulatory decisions, healthcare policy and provide timely care tailored to individual needs. However, the increased use of PROs has uncovered pre-existing ethical issues. For example, uncertainty among patients about why and how PRO data are being collected and used. Furthermore, the increased use of PRO data should match strategies to ensure greater equity, diversity and inclusion (EDI) in their collection and planning to reflect the perspectives of underserved groups and avoid increasing health disparities. Therefore, it is important to understand how to improve EDI in PRO collection, the implications for not doing so; and strategies for evaluating and addressing EDI issues.

Learning Objectives

1. Enable insights on ethical and EDI issues in PRO data collection, in clinical research, drug development, and clinical practice based on current evidence.
2. Understanding and discussion of minimal ethical and EDI considerations as well as rationale in PRO data collection, in clinical research, drug development, and clinical practice.
3. Application of best practices and practical case studies for inclusion of ethical and EDI considerations in PRO data collection, in clinical research, drug development, and clinical practice.

Organizers:

Samantha Cruz Rivera, Dr, Centre for Patient Reported Outcomes Research, University of Birmingham, Birmingham, United Kingdom

Ameeta Retzer, Dr, Centre for Patient Reported Outcomes Research, University of Birmingham, Birmingham, United Kingdom

Angela Stover, PhD, University of North Carolina, Chapel Hill, North Carolina, United States

Paul Kamudoni, Dr, Merck Healthcare KGaA, Darmstadt, Germany

Workshop level: Basic

Workshop Goals:

Increasingly, including the patient’s voice is essential in HRQOL research and PRO measurement. But how do we do this rigorously? The overarching goal of this workshop was to equip attendees to undertake methodologically sound qualitative research that advances the fields of HRQOL research and PRO measurement.

Intended Audience

Trainees, industry partners, and researchers.

Presentation Format

50% presentation; 35% individual/small group work; 15% large group discussion.

Attendees were invited to draw upon their proposed/current research project to create a qualitative purpose statement and research question by following established frameworks (in required pre-readings).

Overview/Outline

Presentation

What is qualitative research?

• Philosophical and methodological roots
• History of qualitative work in HRQOL research and PRO measurement

Why do qualitative research?

• Provide examples of qualitative work in HRQOL research and PRO measurement

What are qualitative methodologies?

• Choose of an appropriate methodology
• Introduction to qualitative description, interpretive description, grounded theory, phenomenology, Indigenous methodologies, and types of content analysis

What are qualitative methods?

• Introduce sampling strategies (participant number and type)
• Outline data collection methods: interviews [closed-ended, semi-structured, and open-ended] and focus groups
• Pros and cons of cognitive interviews and think-aloud in PRO measure development
• Approaches to analysis: content and thematic analysis
• Journal requirements for qualitative checklists

What are common pitfalls and misunderstandings?

• Not specifying an appropriate methodology
• Denoting narrative responses from a survey as “qualitative”
• Closing analysis prematurely
• Forgetting ethical considerations or power dynamics

How do you pose qualitative purpose statements and research questions?

• Description of established frameworks

Individual and group work

Attendees wrote a qualitative purpose statement and research question by drawing on established frameworks. They then identified an appropriate methodology and methods. In small groups, attendees presented this work to each other and provided their rationale. One facilitator was present in each group. Each group reported back on shared learnings and outstanding questions for full group discussion.

Learning Objectives

1. Comprehend key qualitative research methodologies and worldviews, and how these factors have influenced the use of qualitative approaches in the contexts of a) health-related quality of life (HRQOL) research, b) development of patient-reported outcome (PRO) measures, and c) interpretation and use of PRO scores.
2. Critically analyze “tips, tricks, and pitfalls” in qualitative methodologies and methods used in HRQOL research and PRO measurement.
3. Articulate a qualitative purpose statement and research question, and defend an appropriate qualitative research methodology and methods.

Organizers:

Kara Schick-Makaroff, PhD RN, University of Alberta, Edmonton, Alberta, Canada

Richard Sawatzky, PhD, Trinity Western University, Langley, British Columbia, Canada

Antoinette Davey, PhD, University of Exeter, Exeter, United Kingdom

Motolani Ogunsanya, DPh MSc PhD, University Of Oklahoma, Oklahoma City, Oklahoma, United States

Joakim Öhlén, RN PhD, Institute of Health and Care Sciences and Centre for Person-Centred Care at the University of Gothenburg, and Palliative Centre Sahlgrenska University Hospital, Gothenburg, Sweden

Workshop level: Basic

Workshop Goals:

To introduce methods that can be used to generate health utilities in complex patient populations such as young children or orphan/ ultra-orphan indications.

Intended Audience

Beginner. No previous knowledge of utility elicitation methods expected.

Presentation Format

A combination of lectures, hands-on exercises, and online polls/surveys.

Overview/Outline

A version of HTA is used in many countries to decide which drugs, devices, public health initiatives (etc.) are reimbursed. A key component of this process is often some form of economic evaluation with patient benefit expressed in terms of quality-adjusted life years (QALYs). However, to generate these QALYs, it is necessary to have health utilities built into the model on which the economic evaluation is based.

Whilst many generic preference-based instruments, such as the EQ-5D or SF-6D, can be used in general populations, these instruments often have difficulty generating meaningful results in certain populations (e.g., particular orphan/ ultra-orphan indications) due to small sample sizes and heterogenous disease presentation. Consequently, generic measures often lack content validity in such populations, failing to capture aspects of the disease that are important to patients. Additionally, data collection using generic instruments may be impractical, for instance, in very young paediatric populations.

This workshop provided the attendees with a practical overview of alternative methods for eliciting health utilities that can be used in the HTA of interventions targeting such patient groups or indications. The workshop began by providing an overview of health utilities before moving on to explain the difference between direct and indirect elicitation methods and the use of vignettes in the elicitation process. We also covered important aspects to consider when choosing which approach to use and while designing the vignettes.

Case studies based on practical examples were used throughout the workshop, and attendees also had the opportunity to undertake several tasks to help deepen their understanding of the learning provided in the taught part of the workshop.

Learning Objectives

1. Understand why the use of conventional generic PRO instruments such as the EQ-5D-3L or -5L and SF-36 in complex populations, such as young children or orphan/ ultra-orphan indications, is either not possible or highly sub-optimal.
2. Understand alternative approaches that can be used to elicit health utilities in these populations for use in economic models that will form the basis of HTA submissions. These alternative approaches included an extension of the general EQ-5D instruments and vignette-based approaches such as standard gamble/ time trade-off and discrete choice experiments.
3. Understand the strengths and limitations of these alternative approaches in order to be able to identify which would be the preferred option in a given situation.

Organizers:

Stuart Mealing, MSc, York Health Economics Consortium, York, United Kingdom

Adam Smith, PhD, York Health Economics Consortium, York, United Kingdom

Workshop level: Basic

Workshop Goals:

To optimize the impact of research in practice and policy, patient-centered research must involve different stakeholders, including researchers, clinicians, patients, and policymakers. A key role for this stakeholder group is to formulate relevant and researchable questions that can produce timely results that inform practice and policy. It is important that the voices of diverse stakeholders are heard, and they have the skills and framework to ask meaningful and robust research questions that can be operationalized.

A well-conceptualized and clear research question sets out the variables that are meaningful to each stakeholder and describes how the variables are linked together. Properly framed research questions set the stage, a priori, for a discussion on the research design, analysis, data mobilization strategy, and uptake of results in policy and practice. For applied researchers, a framework is needed to support diverse stakeholders to co-design robust research questions.

The goals of this workshop were to provide participants with 1) knowledge about different research question frameworks in the quantitative paradigm; and 2) the skills to co-design research questions with end-user stakeholders and operationalize the implementation of the research in applied clinical settings.

Intended Audience

Health outcomes researchers including early career investigators, decision-makers, graduate students, patient partners, and policy-oriented researchers.

Presentation Format

The workshop format was small group activities, lectures, and discussions

• Small group activities (60%) – to appraise existing research questions and craft strong research questions based on the lessons learned from the workshop. In small group activities, members played different stakeholder roles.
• Lectures (20%) – to introduce frameworks for different types of questions and a case presentation on how to co-design relevant and researchable questions
• Discussion and Q&A (20%)

Overview/Outline

1. Presentation of a patient engagement example identifying researchable questions from the perspective of the relevant stakeholders.
2. The research cycle: Event/outcome estimation, identification of causes, elucidation of mechanisms, and evaluation of deliberate interventions or programs
3. Introduction to different frameworks for formulating research questions according to the intent (e.g., PICOT, PECO).
4. Architecture and vocabulary for strong research questions
5. Going from the question to the research design, analysis, and data mobilization strategy.
6. Crafting tables to show the answer to the question

Learning Objectives

1. To recognize the principles and benefits of patient engagement in applied research topic selection and formulation of research objectives and describe potential strategies to improve or support continued patient involvement in research. To optimize the impact of research in practice and policy, patient-centred research must involve different stakeholders, including researchers, clinicians, patients, and policymakers. A key role for this stakeholder group is to formulate relevant and researchable questions that can produce timely results that inform practice and policy.
2. To identify the appropriate research question framework for different types of applied clinical research. A well-conceptualized and clear research question sets out the variables that are meaningful to each stakeholder and describes how the variables are linked together. Properly framed research questions set the stage, a priori, for a discussion on the research design, analysis, data mobilization strategy, and uptake of results in policy and practice.
3. To design a relevant client- or community-centred research objective using appropriate frameworks. For applied researchers, a framework is needed to support diverse stakeholders to co-design robust research questions.

Organizers:

Nikki Ow, PhD, The University of British Columbia, Vancouver, British Columbia, Canada

Maryam Mozafarinia, PhD, McGill University Health Center, Montreal, Quebec, Canada

Skye Barbic, PhD, The University of British Columbia, Vancouver, British Columbia, Canada

Nancy Mayo, PhD, McGill University, Montreal, British Columbia, Canada

Contributor:

• Dan Nixon, Foundry Central Office, Vancouver, British Columbia, Canada

Organized by the New Investigators SIG

Turning Challenges into opportunities – examples from the Field

The 30-year journey of Quality of Life research has not been without its share of hurdles. However, the pioneers of this field have transformed these trials into remarkable opportunities for future projects, research, and applications. In this session, four of these trailblazers shared their experiences and inspired us with their stories of overcoming adversity.

Speakers:

David Cella, PhD, Northwestern University, Chicago, Illinois, United States

Dr. Cella is Professor of Medical Social Sciences at Northwestern University Feinberg School of Medicine, and an elected member of the National Academy of Medicine (NAM) and the Academy of Behavioral Medicine Research. He led the development and validation of the FACIT Measurement System, PROMIS, Neuro-QoL, and the emotional health domain of the NIH Toolbox. His work has led to patient-centered improvements in the US healthcare system; for this he was awarded the NAM 2016 Gustav O. Lienhard Award for Advancement of Health Care.

Claire Snyder, PhD, Johns Hopkins School of Medicine, Baltimore, Maryland, United States

Dr. Claire Snyder is Professor of Medicine, Oncology, and Health Policy & Management at the Johns Hopkins Schools of Medicine and Public Health in Baltimore, Maryland. She is a past president of ISOQOL. Her research focuses on the quality of cancer care. She currently leads the PROTEUS Consortium, which helps navigate the use of PROs in clinical trials and clinical practice.

Nancy Mayo, PhD, McGill University, Montreal, Quebec, Canada

Lisa Lix, PhD P.Stat., University of Manitoba, Winnipeg, Manitoba, Canada

Devonian Gardens

The 2023 social event took place at the Devonian Gardens, a vibrant sanctuary amidst Calgary’s bustling core. Attendees joined us in this unique setting and enjoyed the botanical gardens, fish ponds, and water features that this place offered.

The Experience Calgary Social Event at the 30th Annual Conference was an exciting way to socialize with colleagues and build new connections. The ticket price included heavy appetizers, two drink tickets, musical entertainment, and dancing.

Come meet with us to discuss the challenges and solutions related to strategic and operational management of PROMs and other COAs, as well as the keys to increase the visibility of your COAs and improve the COA user experience (and satisfaction!) – all to ensure the increased use of COAs worldwide.

*This was an Ancillary Event hosted by Mapi Research Trust and was not an ISOQOL event. Pre-registration was required.

The Clarivate Clinical Outcomes Assessment (COA) team help sponsors to select, develop, and validate COAs (including PROs) for clinical trials. We were available for meetings throughout ISOQOL to discuss COA needs.

*This was an Ancillary Event hosted by Clarivate and was not an ISOQOL event.