Sponsored by Vector Psychometric Group

Description:

The Cutting Edge Research plenary session featured some of the highest-ranked, innovative research from ISOQOL abstract submissions. In particular, these abstracts reflected research that truly “pushes the ISOQOL envelope” in providing new and different ways to look at quality of life.

Speakers:

Chao Long, MD, The Johns Hopkins Hospital and The Curtis National Hand Center, Baltimore, Maryland, United States

Dr. Chao Long is a surgeon-scientist who conducts health services research aimed to increase access to high quality, patient-centered, and affordable surgical care in the US and abroad. Her current research centers on promoting equitable evaluation of patient-reported outcomes in low literacy and other disadvantaged populations. Her clinical focus is on plastic and reconstructive, hand, and musculoskeletal conditions. She is currently a PGY-4 plastic and reconstructive surgery resident in the Johns Hopkins/University of Maryland program. She previously served as the Research Fellow at The Curtis National Hand Center and the Donald R. Laub Fellow at ReSurge International.

Katrina Borowiec, MA, Boston College, Chestnut Hill, Massachusetts, United States

Katrina Borowiec is a doctoral candidate in Measurement, Evaluation, Statistics, and Assessment (MESA) at the Boston College Lynch School of Education and Human Development. Her dissertation research employs an intersectional approach to measurement invariance analyses in psychosocial scales for diverse college student populations. Katrina is also a psychometric consultant for the DeltaQuest Foundation, where she utilizes item response theory, structural equation modeling, and other approaches to analyze patient-reported quality-of-life outcome measures for individuals with disabilities and life-threatening medical conditions. She holds master’s degrees in higher education from Boston College and survey research methodology from the University of Connecticut.

Jacquelyn S. Pennings, PhD, Vanderbilt University Medical Center, Nashville, Tennessee, United States

Jacquelyn Pennings is a Research Associate Professor in the Department of Orthopaedic Surgery in the School of Medicine at Vanderbilt University. Jackie’s research interests include the development, validation, and use of patient reported outcome measures, particularly after orthopaedic surgery, the development and use of longitudinal registries to track patient reported outcomes over time, and the relationship between physical and mental health outcomes. She earned a Ph.D. and Master’s degree in experimental psychology from Texas Christian University after obtaining a Bachelor’s degree from Belmont University. Jacquelyn has also been accredited by the American Statistical Association as an Accredited Professional Statistician.

Yuelin Li, PhD, Memorial Sloan Kettering Cancer Center, New York, New York, United States

My current research focuses on Machine Learning (ML) and Bayesian nonparametric statistical methods. These methods offer new and powerful ways to solve problems that cannot be easily solved by conventional methods. That is in part because ML imitates intelligent human behaviors. For example, ML can recognize patterns in complex data without the need for human supervision. In this talk I will use Bayesian nonparametric clustering as an ML engine to identify personalized treatment rules using patient-reported symptoms. I will also show that personalized treatments tailored to co-occurring symptoms may be more effective than a conventional one-size-fits-all approach.

Sponsored by RWS

Description:

Person-centred care is a fundamental shift in the delivery of health care that personalizes it by placing the patient/client at the nucleus of this care. This plenary incorporated critical perspectives to explore the interconnections of person-centred care and PRO use at individual and aggregated levels, alongside patient engagement with patients as partners in healthcare and their own use of PROs.

Speakers:

Joakim Öhlén, RN PhD, Institute of Health and Care Sciences and Centre for Person-Centred Care at the University of Gothenburg, and Palliative Centre Sahlgrenska University Hospital, Gothenburg, Sweden

Joakim Öhlén, PhD, RN, is professor in nursing at the Sahlgrenska Academy, University of Gothenburg in Sweden, and has a clinical appointment to the Palliative Centre at Sahlgrenska University Hospital in Gothenburg. He is the Centre Director at the University of Gothenburg Centre for Person-Centred Care, and leads a research programme on person-centeredness in palliative care. He has contributed with knowledge about how to put person-centeredness in action including a practice model for communication in the context of palliative care, patients’ preparedness for recovery, and support to family members in palliative care. He has extensive teaching experience.

Kirstie Haywood, DPhil BSc(Hons), PGCertTLHE, Warwick Research in Nursing, Warwick University, Coventry, United Kingdom

Kirstie’s interest in PRO research has embraced many conditions, with a long-term interest in rheumatology/musculoskeletal and resuscitation research. She has led on the development, use and evaluation of PRO approaches in research and routine practice settings, and in core outcome set development. She is passionate about working collaboratively with patients as partners across these endeavours, co-founding and chairing ISOQOL’s Patient Engagement SIG. She is an invited member of the European Resuscitation Council guidelines and American Heart Association ‘Statement of Survival after Cardiac Arrest’ writing groups, raising the importance of capturing the voice of survivors and their families in resuscitation research.

Sponsored by EORTC

Description:

Modern perspectives of measurement validity focus on the inferences, actions and decisions based on PROs. This plenary addressed evidence-based foundations for comparing PROs of different people and assessing change over time to inform healthcare decisions.

Speakers:

Mirjam Sprangers, PhD, Amsterdam University Medical Centers, Amsterdam, Netherlands

Mirjam Sprangers, psychologist by training, is full professor at the Department of Medical Psychology of the Amsterdam University Medical Centers in the Netherlands. She coordinates a research line addressing the methodological, theoretical and clinically applied aspects of quality-of-life research in a range of chronic diseases. She is a long-standing member of the European Organisation for Research and Treatment of Cancer Study Group on Quality of Life. She was a Foreign Adjunct Professor of the Karolinska Institute in Stockholm, Sweden. She served as President of the board of ISOQOL and received the 2014 ISOQOL President’s award.

Bruno D. Zumbo, PhD, The University of British Columbia, Vancouver, British Columbia, Canada

Bruno D. Zumbo is a mathematical scientist working at the intersection of the behavioural, social and health sciences. He is Professor and Distinguished University Scholar, Tier 1 Canada Research Chair in Psychometrics and Measurement, and the Paragon UBC Professor of Psychometrics & Measurement at the University of British Columbia. As an internationally recognized psychometrician and measurement theorist, his research program has influenced psychometric and validity theories, measurement invariance (DIF & test bias), and reliability. His insights into validity theory and measurement invariance have been twice recognized as a Killam Laureate (Research & Teaching awards) and the Samuel Messick Award.

Sponsored by Mapi Research Trust

Description:

Patient reports on their health outcomes and healthcare experiences, including their narratives, are central to person-centred learning healthcare systems. The presenters discussed the state of the art in using patient reports to ensure that value-based healthcare is achieved and drives the person-centred learning health systems.

Speakers:

Charles P. Friedman, PhD, University of Michigan Medical School, Ann Arbor, MI, United States

Charles Friedman is the Josiah Macy Jr. Professor of Medical Education and Chair of the Department of Learning Health Sciences at the University of Michigan Medical School. In recent years, he has focused his academic interests and activities on the concept of Learning Health Systems, and the socio-technical infrastructure required to sustain them. He is editor-in-chief of the open-access journal Learning Health Systems and co-chair of the movement to Mobilize Computable Biomedical Knowledge.

Verna Yiu, MD FRCPC, Alberta Health Services, Edmonton, Alberta, Canada

An accomplished leader in the academic and healthcare sectors, Dr. Verna Yiu has been involved in public sector governance for over two decades. She is currently the President and CEO of Alberta Health Services (AHS), Canada’s first and largest province-wide health system providing integrated care to more than 4.2 million Albertans. AHS has been recognized as a national and international leader in many clinical and administrative areas including: being named as one of the top 5 most integrated healthcare systems out of 40 countries, having one of the top 3 best health supply chain management systems in the world, and awarded one of Canada’s Top 100 Employers for three consecutive years.

Dr. Yiu previously served as Vice President, Quality and Chief Medical Officer in AHS from 2012-2016. Prior to the role of Interim Dean in 2011, she was Vice-Dean for the Faculty of Medicine/Dentistry from 2008-2011 and Assistant Dean for Student Affairs from 2000-2008. She has won numerous awards for teaching and clinical excellence and also chaired the Specialty of Nephrology for the Royal College of Physicians and Surgeons of Canada for 6 years. Dr. Yiu is a Professor of Pediatrics at the University of Alberta and continues to practice as a pediatric nephrologist.

Brenda Hemmelgarn, MD PhD, University of Alberta, Edmonton, Alberta, Canada

Dr. Brenda Hemmelgarn is a nephrologist, health services researcher and Dean, Faculty of Medicine & Dentistry U of Alberta. Her research includes strategies to improve care and outcomes for patients with kidney disease and other chronic conditions. She is a strong advocate for patient-oriented research, patient engagement and integrated knowledge translation. She has more than $20 million in funding and more than 500 peer-reviewed publications. Among her awards Dr. Hemmelgarn received the Alberta Science and Technology Leadership (ASTech) Foundation Award for Outstanding Achievement in Health Innovation and is a fellow of the Canadian Academy of Health Sciences.

Workshop level: Basic

Workshop Goals:

Our goal was to provide an overview of the development, evaluation, and implementation of PROMs for screening in clinical practice and research. We emphasized key skills with workshop activities. Background: Patient-reported outcome measures (PROMs) used for screening purposes have score thresholds (cut-scores) that can be used to categorize patients into clinically-meaningful groups, which facilitates identifying patients who might benefit from additional intervention or supportive care. Score thresholds can also enhance the interpretability and actionability of PROMs in clinical practice and research.

Intended Audience

Individuals interested in developing and/or using PROMs for screening purposes in clinical practice or research. This workshop was designed to be applicable to any patient population and to learners of all levels with prior knowledge/experience with PROMs.

Overview/Outline

1. Introduction and workshop objectives (10 minutes) All presenters
2. Methods lecture (45 minutes) 3 presenters
   • Methods for establishing score thresholds for a PROM when a “gold standard” is available
   • Methods for establishing score thresholds for PROMs in the absence of a gold-standard,
3. Methods Activity (25 minutes) All presenters: break out groups moderated by presenters will apply the methods learned in the didactic lecture to example data.
   Break (5 minutes)
4. Clinical/research applications (45 minutes)
   • Example 1 – Successes and learnings from integrating a screening PROM in otolaryngology practices – 1 presenter
   • Example 2 – Consideration of the Needs of Clinicians to Successfully Implement PROMs Used for Screening Purposes – 1 presenter
   • Example 3 – Using self-reported cognitive concerns and cognitive test performance to screen for cognitive impairment in primary care – 1 presenter
5. Panel discussion – Workshop presenters will discuss lessons learned from the Clinical Examples in a Q&A format (10 minutes)
   Break (5 minutes)
6. Evaluation/Implementation Activity (25 minutes) All presenters: Break out groups will elicit and discuss desirable properties of PROMs when used for screening and key considerations for successful implementation based on pre-defined contextual scenarios
7. Discuss resources for further learning (10 minutes) – 1 presenter
   Closing Total time = 180 minutes (max = 180)

Learning Objectives

1. Describe the methods for developing or adapting PROMs for screening purposes in clinical practice and research. Through didactic presentations, participants learned the methods for developing new or adapting existing PROMs for screening purposes when a gold standard is available (sensitivity, specificity, negative and positive predictive value, etc.) versus when a gold standard is not available (standard setting by a panel of stakeholders, use of normative data, etc.), including the strengths and limitations of various approaches. Moderated break-out groups provided participants with the opportunity to apply the methods discussed.
2. Identify the properties of a useful patient-reported screener. Presenters described approaches for critically evaluating PROMs used for screening purposes. In a second break-out session, participants were presented with example PROMs for screening and clinical and/or research vignettes. For example, one important property for clinicians working directly with patients is the clinical relevance and integration of results into their clinical reasoning processes. Attendees applied their knowledge in discussions of selecting and evaluating the properties of PROMs for screening for specific applications.
3. Recognize opportunities to use PROMs as screening tools in clinical practice and research. Didactic presentations instructed attendees on how to identify situations where PROMs can be informative as well as situations where their use is to be avoided. Practical applications of PROMs for screening purposes were illustrated through a series of three case studies. Additionally, consideration of the needs and individual factors that impact clinicians’ behaviors to support the use of PROMs as screeners were illustrated. In the second moderated break-out session, participants used research vignettes to elicit and discuss attributes needed for successful implementation of PROM screening tools in clinical practice or research.

Organizers:

• Kathleen Yost, PhD, Mayo Clinic, Rochester, United States
• Theresa Coles, PhD, Duke University School of Medicine, Durham, United States
• Minji Lee, PhD, Mayo Clinic, Rochester, United States
• Angela Wolff, RN PhD, Trinity Western University, Langley, British Columbia, Canada
• Cindy Nowinski, MD PhD, Northwestern University, Chicago, United States
• Natasha Roberts, Royal Brisbane and Women’s Hospital, Herston, Australia

Workshop level: Advanced

Workshop Goals:

1. Illustrate the importance of the estimand framework for patient-reported endpoints
2. Demonstrate the link from the PRO objective to the interpretation through the estimand framework
3. Provide a clear and accessible demonstration of PRO estimands that can be applied in the regulatory setting

Intended Audience

The workshop was intended for academics, trialists and outcomes researchers who design clinical trials and plan and analyze patient-reported endpoints.

Overview/Outline

1. Background
 a. Regulatory and reimbursement authorities increasingly require that patient experience is captured in clinical trials
 b. PRO objectives are often vaguely defined, leading to interpretation challenges and exclusion of results from the drug label
 c. The estimand framework provides a clear process for defining and interpreting PRO endpoints
 d. Regulatory agencies are adopting the estimand framework and it is expected to be increasingly used in trials
2. Definitions of estimands, per ICH E9 addendum
 a. 5 components
 b. Strategies for evaluating estimands:
  • Treatment policy
  • Composite
  • Hypothetical
  • Principal Stratum
   1. When might this be helpful?
   2. Advantages and disadvantages
   3. User-friendly and more technical resources and examples
  • While on treatment
3. Sensitivity analyses for PRO endpoints
4. Demonstration of estimands on PRO endpoints
 a. The ACCENT trial evaluated the efficacy of N-acetylcysteine (NAC) in adults with cannabis use disorder;
 b. The Marijuana Cravings Questionnaire4 evaluated patient-reported marijuana cravings;
 c. Estimand 1: assessment of efficacy for NAC relative to placebo on patient-reported cravings
  i. Definition of intercurrent events
  ii. Strategies used:
   • a composite strategy for discontinuation
   • a treatment policy strategy for non-adherence
  iii. Sensitivity analyses
  iv. Communicating the findings
 d. Estimand 2: assessment of efficacy for NAC relative to placebo on patient-reported cravings while participants were medically adherent
  i. Definition of intercurrent events
  ii. Strategies used:
   1. a hypothetical strategy for discontinuation and for non-adherence
  iii. Sensitivity analyses
  iv. Communicating the findings
 e. Estimand 3: assessment of efficacy for NAC relative to placebo on patient-reported cravings while participants were receiving treatment
  i. Definition of intercurrent events
  ii. Strategies used:
   1. a while on treatment strategy for discontinuation
   2. a treatment policy strategy for non-adherence
  iii. Sensitivity analyses
  iv. Communicating the findings
5. Final remarks
 a. The goal of this workshop was to demonstrate a link from the PRO objective to the interpretation through the estimand framework

Learning Objectives

1. Understand how using the International Council for Harmonisation (ICH) addendum on estimands and sensitivity analysis in clinical trials provides a framework connecting the PRO objective, with the analytic approach, and interpretation.
2. Describe the 5 components of estimands for PROs and evaluate the strengths and limitations of different estimand strategies for clinical trials.
3. Given a PRO measure, design 3 evaluable estimands for a clinical trial and given a PRO estimand, define a corresponding sensitivity analysis and successfully communicate the interpretation.

Organizers:

• Libby Floden, PhD MPH, Clinical Outcomes Solutions, Tucson, United States
• Melanie Bell, PhD, University of Arizona, Tucson, United States
• Bellinda King-Kallimanis, PhD, LUNGevity, Washington, United States
• Jessica Roydhouse, PhD, University of Tasmania, Hobart, Australia

Workshop level: Basic

Workshop Goals:

There is a critical need to understand whether individual patients have changed on patient-reported outcome measures (PROs). The United States Food and Drug Administration (FDA) requires that PROs qualified as clinical outcome assessments have established thresholds for meaningful within person change and for determining when individual patients have benefited from treatment (responder definition). Clinicians need to be able to determine if their patients improve after a treatment modification. To date, group-based methods have often been inappropriately used to define individual change. In addition, there is confusion about what counts as important change at the individual level. This workshop clarified the conceptual basis for defining individual change on PROs, detailed the most appropriate methods for estimating individual level change, and provided a hands-on, demonstration-based instructional session on estimation of individual change in common statistical software.

Intended Audience

The intended audience was PRO researchers, clinical trialists focusing on PROs, practitioners interested in determining whether individual patients have changed, and researchers involved in regulatory applications of PROs.

Overview/Outline

1. Introduction (5 minutes)
2. Background on Key Concepts (60 minutes)
   • Change scores
   • Brief history of minimally important differences and change
   • Group change: Anchor methods vs. distribution methods
   • Group methods should not be used for individual change
   • Methods for individuals
     • Reliable change index and related statistics
     • Applications in Item Response Theory
     • Meaningful change at the individual level
3. Guided Group Discussion (15 minutes)
   Break (15 minutes)
4. Demonstration and Practice Estimating Individual Change in Common Statistical Software (60 min)
   • Set-up cloud-based access to practice data for all participants
   • Ensure each participant has appropriate software installed
   • Walk-through multiple examples
5. Guided Group Discussion (20 minutes)
6. Wrap-up (5 minutes)

Learning Objectives

1. Distinguish between appropriate methods for estimating group and individual change on PRO measures.
2. Be familiar with methods to identify statistically significant and meaningful change on PROs, and the advantages and limitations of applying different thresholds.
3. Estimate statistically significant individual change using common statistical software (e.g., SAS, R) and interpret the results.

Organizers:

• John Peipert, PhD, Northwestern University, Chicago, United States
• Ron Hays, PhD, UCLA Department of Medicine, Los Angeles, United States

Workshop level: Basic

Workshop Goals:

To support the adoption of high-quality, evidence-based, fit for purpose digital clinical measures in clinical trials, patient care, and public health.

Intended Audience

Professionals dedicated to the development and deployment of digital clinical measures using wearable sensor technology.
We are skilled at communicating with diverse audiences, and provided invaluable insights to everyone passionate about advancing remote patient monitoring.

Overview/Outline

The global COVID-19 pandemic driven a new urgency for devising a common framework for the development and deployment of high quality, patient centric, digital clinical measures. Multiple digital technologies for symptom tracking and contact tracing are currently in use and/or development, and clinical researchers are now designing new decentralized clinical trial strategies using telemedicine and remote patient monitoring to minimize pandemic-related disruption of the clinical research enterprise. Adoption of a comprehensive “how-to” roadmap to synthesize best practices in the digital health field is essential for guiding these efforts and safeguarding public health.

The Playbook synthesizes best practices from the digital health field into one comprehensive “how-to” document. A first draft, built by collaborators from the Digital Medicine Society, Elektra Labs, Genentech, Koneksa, Myokardia, Sage Bionetworks, Scripps Research, and the FDA. The Playbook is now being advanced for broad adoption by these organizations and new collaborators from ActiGraph; Activinsights; American Pharmacists Association; BlackThorne Rx; Center for Digital Health at Brown University; Covance; Digital.Health; Duke University; Eli Lilly and Company; the European Medicines Agency; Evidation Health; Merck.; Open mHealth; Pfizer; physIQ; Savvy Coop; Takeda; VivoSense; and Winterlight Labs.

This session was delivered as an interactive workshop, providing participants with the foundational elements of selecting, developing, and deploying high quality digital clinical measures that cut across use cases. Speakers are co-creators of The Playbook with extensive experience and expertise in measurement science, HEOR, the science of patient engagement, regulatory science, and — perhaps most importantly — education. This speaker panel demonstrates disciplinary diversity as well as effective pre-competitive collaboration. To support the audience in selecting, developing, and deploying fit-for-purpose, meaningful digital clinical measures across clinical research, patient care, and public health.

Learning Objectives

1. To support the audience in selecting, developing, and deploying fit-for-purpose, meaningful digital clinical measures across clinical research, patient care, and public health.
2. To support the audience in identifying ethical considerations associated with the introduction of digital clinical measures to clinical trials, patient care, and public health.
3. To support the audience in selecting the right partners for co-creating and implementing high quality, patient centric, digital measures of health.

Organizers:

• Jennifer Goldsak, MChem MA MBA, Digital Medicine Society (DiMe), Boston, United States
• Betsy Tschosik, Genentech, Chicago, United States
• Elizabeth Nicki Bush, Eli Lily and Company, Indianapolis, United States

Workshop level: Basic

Workshop Goals:

Integrating PRO measures into clinical practice requires a considered approach to ensure effectiveness and sustainability. There is much to take into account, including impacts from the health service, from staff, and from patients and their families.
The goal of this workshop was to explore the application of complexity theory to the use of PROs routine clinical care.

Intended Audience

This workshop was intended for individuals who have a basic knowledge of PROs in research and/or clinical practice.

Overview/Outline

Workshop outline [180 minutes]:
1. Overview of complex interventions [40 minutes]

• Welcome and agenda overview
• Introduction to complex interventions:
• What does it mean for an intervention to be “complex”?
• What tools and resources are available? What makes using PROs complex?
• Small group activity reflecting their experiences of complexity
• Facilitated discussion of what each group identified from their settings/contexts

2. Theoretical approaches to complexity [30 minutes]

• Overview of implementation science and PROs
• Complexity science theory
• PCORI proposed method standards for studies of complex interventions
• Process Measurement for the implementation of complex interventions

Break [15 minutes]

3. Case Study 1 [25 minutes]: Developing a strategy for introducing PROs using Complexity Theory: Using PROs to overcome the fragmentation in clinical care of individual patients

• Facilitated discussion and questions

4. Case Study 2 [25 minutes]: Development of a PRO intervention in clinical practice: How using phases of development can be used to build a robust complex intervention

• Facilitated discussion and questions

5. Case Study 3 [25 minutes]: Using a complexity lens to guide the design and implementation of PROs in clinical practice: A complexity-informed approach across multiple use cases and workflows within a health system

• Facilitated discussion on sustaining complex intervention and questions

6. Interactive Group Activity with Facilitators – Apply complexity theory to your context [15 minutes]
7. Share results and wrap up [5 minutes]

Learning Objectives

1. Describe the fundamentals of complexity theory and complex interventions in relation to healthcare
   Participants were able to classify theoretical approaches, understand what complexity is, and why a theoretical approach is needed. They were able to recognize the complexities of PRO use in individual patients, clinicians, specific subgroups, local institutions and broader health system and assist incorporation of PROs into the workflow. Through facilitated reflection and group discussion about complex interventions, participants’ were able to draw on their experiences. Participants were able to access tools and resources available to manage complexity in research and practice and know where to find further information.
2. Describe PRO use in clinical practice as a complex intervention
   Participants were able to identify why PROs are complex interventions. They were able to compare and contrast complexity theory in order to recognize complex interventions in their respective settings. This learning objective was achieved through presented content of both theory and applied international case studies, through facilitated reflection, and group discussion. As a result, participants were able to explore the value and potential for using PROs in clinical practice.
3. Apply understanding of complexity theory and complex intervention frameworks based on international case studies
   Participants were able to understand how to apply complexity theory to PROs in clinical practice. They were able to plan the design of a clinical implementation that integrates complex PRO interventions into clinical care. Participants were able to understand the complexities of using PROs at an individual patient and clinician level, complexities at the local level, and at a broader health service level. Additionally, real world examples enabled the application of complexity theory across a variety of disease types, patient and clinician groups and international health services.

Organizers:

• Natasha Roberts, PhD, University of Queensland, Herston, Australia
• Elizabeth Austin, MPH, University of Washington, Seattle, WA, United States
• Liv Marit Valen Schougaard, PhD, Regional Hospital West Jutland, Herning, Denmark
• Bernice Gulek, MS, University of Washington, Seattle, United States
• Angela Wolff, PhD, Trinity Western University, Langley, British Columbia, Canada

Workshop level: Basic

Workshop Goals:

1. Introduction to the method and practice of Digital Storytelling (DST)
2. Take part in viewing and discussion of digital stories
3. Discuss how stories can be used and shared in health-related quality-of-life research
4. Discuss the ethical considerations of DST

Intended Audience

Researchers, clinicians, patient-partners, and other stakeholders interested in learning about the methodology, process, and practice of storytelling in health research, and how digital stories complement patient-reported outcomes research.

Overview/Outline

Stories and storytelling help us to make sense of our thoughts, feelings, and experiences. They help us to better understand our environment, interactions with others and to formulate and convey our values, beliefs, and insights for ourselves and others.

DST is an innovative Community-Based Participatory Research (CBPR) method that increases the participation of people with lived experience in health research. Digital stories are short (3-5 minutes), first-person video narratives created with the combination of images, video, narration, and music. DST has been used by academics across disciplines, including health research for the past two decades.
In this workshop, we focused on DST methods, how DST works in practice as well as its benefits and challenges. This workshop introduced participants to a 7-step methodology developed by Story Center, US (Lambert, J., 2020, Berkeley, CA., USA). The 7-steps include owning your insights, owning your emotions, finding the moment, seeing your story, hearing your story, assembling your story, and sharing your story. As a CBPR method, DST can help to reach and elevate the voices of underrepresented populations and can be used as a tool to meaningfully engage populations who value storytelling as a way of knowing and understanding. It could also be used as a tool to understand what matters most to patients, to inform and personalize quantitative data, and as a powerful knowledge translation tool.

The workshop outline:

1. Introduction to DST: including the 7-steps involved to develop a digital story (60 minutes)
2. View and discuss the use of digital stories in health research (60 minutes)
3. Group discussion: digital stories and patient-reported outcomes research (30 minutes)
4. Group discussion: ethical considerations when using DST as a research method (30 minutes)

Learning Objectives

1. Describe how Digital Storytelling supports efforts to capture patient experiences including those from under-represented populations.
2. List and describe the 7 steps involved in the Digital Storytelling method.
3. Describe how Digital Storytelling can be applied in person-centered outcomes research.

Organizers:

• Sandra Zelinsky, University of Calgary, Calgary, Alberta, Canada
• Maria Santana, PhD, University of Calgary, Calgary, Alberta, Canada
• Danielle Lavallee, PhD, British Columbia Academic Health Science Network, Vancouver, British Columbia, Canada
• Sadia Ahmed, Masters, University of Calgary, Calgary, Alberta, Canada

Workshop level: Basic

Workshop Goals:

Regardless of whether patient-reported outcome measure (PROM) data is collected via app or integrated into the electronic health record, the numeric results must be transformed into a format that is easy to read and interpret for busy clinicians and their patients. The workshop goals were to present and discuss examples and best practices for visualizing PROM data in the context of clinical care.

Intended Audience

Clinicians, clinical informaticists, clinical researchers, implementation scientists

Overview/Outline

Session 1: Data visualization for clinical PROM interpretation: an overview – 30 min
Abi Williams, Adelphi Values, Boston, United States

• This session highlighted the data visualization research and applications that have been developed to help clinicians interpret results.

Session 2: Real-world application in the clinical setting – 30 min
Galina Velikova, MD PhD FRCP, University of Leeds, Leeds, United Kingdom

• The presenter provided examples of how clinicians use data visualization to interpret patient-reported outcomes in a setting where there is routine collection of PROMs allowing for visualization of symptom trends over time and aggregate data assessment.

Session 3: Recommendations for PRO data visualization – 30 min
Claire Snyder, PhD, Johns Hopkins University, Baltimore, United States

• Dr. Snyder reviewed research findings regarding differences in accuracy of interpretation and ratings of clarity across different PROM data display formats. She then described how this evidence base informed development of recommendations for PROM data display and how these recommendations meet stakeholders’ needs.

Sessions 1-3 Q&A – 15 min
Session 4: Interactive computer demonstration – 30 min
Ethan Arenson, MA MS, Adelphi Values, Boston, United States

• Using a demo computer application, participants entered responses for a hypothetical patient and viewed a dashboard with visuals to interpret results.

Session 5: What works for clinicians? A discussion panel – 45 min
Panel: Angela Wolff, PhD RN, Trinity Western University, Langley, BC, Canada; Saied Shahraz, ICON PLC, Dublin, Ireland
Moderator: Emily Henkle, PhD MPH, OHSU-PSU SPH, Portland, United States

• This panel facilitated by researchers focusing on implementation, end-user needs, and clinical informatics invited participants to discuss data visualization methods that have helped with the interpretation of PROM results and enabled their use to meaningfully guide patient care.

Learning Objectives

1. To learn about different approaches for visualizing patient-reported outcome measure (PROM) results for clinical interpretations.
2. To understand evidence-based recommendations for presenting PROM data.
3. To discuss ways data visualization can improve the interpretation of PROMs in the clinical setting.

Organizers:

• Emily Henkle, PhD MPH, OHSU-PSU School of Public Health, Portland, United States
• Ethan Arenson, MA MS, Adelphi Values, Boston, United States

Additional Workshop Speakers:

• Abi Williams, Adelphi Values, Boston, United States
• Claire Snyder, PhD, Johns Hopkins University, Baltimore, United States
• Angela Wolff, PhD RN, Trinity Western University, Langley, BC, Canada
• Saied Shahraz, ICON PLC, Dublin, Ireland

Description:

This discussion focused on how the use of PROMs and PREMs can function as enablers and triggers for the integration of person-centred care in healthcare practice. Possible questions for discussion: What are the features of a person-centred practice? What are the users’ interests as related to using PROMs and PREMs? What are the practice interests that can drive the development?

Host:

Joakim Öhlén, RN PhD
Institute of Health and Care Sciences and Centre for Person-Centred Care at the University of Gothenburg, and Palliative Centre Sahlgrenska University Hospital

Description:

At this roundtable, we invited participants to discuss the use of patient-reported outcome measures to inform person-centered care and to transform health care. Participants brought their own expertise and evidence to this interactive roundtable.

Hosts:

Richard Sawatzky, PhD RN
Trinity Western University & Centre for Health Evaluation and Outcome Sciences

Maria J. Santana, PhD
University of Calgary

Kara Schick-Makaroff, PhD RN
University of Alberta

Description:

ePROs-based online interventions are complex, with active components influencing patients, clinicians and clinical pathways. Patient benefit depends on the content and design of the intervention, patients’ and clinicians’ engagement and associated clinical actions or self-management. We discussed different approaches across oncology contexts to learn how interventions could be designed and delivered to maximise benefit. Participants brought their questions, examples and experiences.

Host:

Galina Velikova, MD PhD FRCP
University of Leeds

Description:

In this roundtable, we discussed the challenging process of Implementing patient-reported outcomes (PROs) into clinical practice with a focus on the spectrum of approaches from Volkswagen to Cadillac. We used the Canadian pediatric landscape as an example. 

Hosts:

Robert Klaassen, MD FRCP(c)
Children’s Hospital of Eastern Ontario

Samantha J. Anthony, PhD MSW RSW
SickKids

Description:

Who does the ‘work’ of implementing patient-reported measures in routine care settings, and how? In this roundtable we discussed common barriers to implementing PROMs and PREMs in clinical practice, and explored practical strategies for overcoming them informed by implementation science theory. 

Host:

Caroline Potter, DPhil
University of Oxford

Description:

In this roundtable, we discussed the process, barriers, and enablers of implementing patient-reported outcome measures (PROMs) in routine care settings. This was an interactive session covering broad principles such as what implementation science can do for PROMs, theories that may be helpful, and how to evaluate PROM implementation initiatives.

Host:

Angela Stover, PhD
University of North Carolina – Chapel Hill

Description:

This roundtable discussed the ability of PROMs, as a Clinical Outcome Assessment (COA), to provide real-world evidence (RWE) for making regulatory decision. This was a discussion for those who have implemented PROMs at a registry or health system level and were interested in how PROMs may be used to support post-marketing surveillance.

Host:

Richard Skolasky, ScD
Johns Hopkins Univeristy

Description:

This roundtable discussed the clinical implementation of PROMs from a health economic framework. This was a discussion for those who have implemented and those who were wanting to implement; addressing both how to design implementations with this framework in mind and once implemented how PROMs can be used to improve quality and decrease cost.

Host:

Amy Cizik, PhD MPH
University of Utah

Description:

Effective engagement with patients and members of the public as research partners can have a profound impact on the conceptualisation and conduct of research. Seven years since the ‘values’ for patient engagement in health-related quality of life research were proposed, how well are we doing? This roundtable was an opportunity to explore the barriers, opportunities and advances in patient engagement. 

Host:

Kirstie Haywood, DPHiL BSc (Hons)
University of Warwick
Warwick Medical School
Warwick Research in Nursing

Description:

Historically, validity has been seen as a static property of an instrument, established during its development, whereas modern validity theory holds that validity is a property of the inferences drawn from data for a particular context. This Roundtable discussed the five sources of validity evidence (generated mainly, but not only, by quantitative research methods) from the Standards for Educational and Psychological Testing in contrast to types of validity.

Host:

Richard Osborne, PhD
Swinburne University of Technology

Description:

Historically, validity has been seen as a static property of an instrument, established during its development, whereas modern validity theory holds that validity is a property of the inferences drawn from data for a particular context. This Roundtable discussed the five sources of validity evidence (generated mainly, but not only, by qualitative research methods) from the Standards for Educational and Psychological Testing in contrast to types of validity.

Host:

Melanie Hawkins, PhD
Swinburne University of Technology

Moderator:

John Eric Chaplin, PhD, Sahlgrenska Academy at University of Gothenburg, Gothenburg, Sweden

Individual Presentations:

The impact of anxiety in relation to COVID-19 on the life-situation of young people in Sweden

Malin Berghammer, Ass. Professor, Queen Silvias Childrens Hospital, Gothenburg, Sweden
Margaretha Jenholt Nolbris, Ass. Professor, Institute of Health and Care Sciences, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden

A longitudinal study on mental and social health of children and adolescents; a Dutch general population

Lotte Haverman, PhD, Emma Children’s Hospital, Amsterdam UMC, Amsterdam, Netherlands

Responding to Challenges Faced by Indigenous Children & Youth During COVID

Dr. Nancy Young, PhD, Laurentian University, Sudbury, Ontario, Canada

The Key to Creating a COVID Research Agenda in 2021 and Beyond: Safe, Intentional and Meaningful Engagement

Alisha Daya, The University of British Columbia, Burnaby, British Columbia, Canada
Skye Barbic, PhD, University of British Columbia, Vancouver, British Columbia, Canada

Moderator:

Jessica Roydhouse, PhD, University of Tasmania, Hobart, Australia

Individual Presentations:

Identifying appropriate measures for caregivers in a lung cancer longitudinal study

Bellinda King-Kallimanis, PhD, LUNGevity Foundation, Bethesda, United States

Among study participants with dementia, at what level of cognitive impairment should family caregiver proxy report be used instead of patient self-report for the assessment symptoms, function and quality of life?

Antonia Bennett, PhD, University of North Carolina, Chapel Hill, United States

Optimal Methods for Reducing Proxy-Introduced Bias on Patient-Reported Outcome Measurements for Group-Level Analyses

Brittany Lapin, PhD MPH, Cleveland Clinic, Cleveland, United States

A Matter of Perspective? Assessing the Role of Phrasing on Proxy-Reported Health Experiences

Jessica Roydhouse, PhD, University of Tasmania, Hobart, Australia

Moderator:

Elizabeth Unni, PhD, Touro College of Pharmacy, New York, United States

Individual Presentations:

Current evidence supporting the implementation of electronic patient-reported outcome measures (ePROMs) in the management of chronic diseases – a structured review

Olalekan Lee Aiyegbusi, MBChB PhD, University of Birmingham, Birmingham, United Kingdom

Remote monitoring of symptoms from both disease and treatment in men with advanced prostate cancer – a single arm clinical trial

Natasha Roberts, PhD, University of Queensland, Herston, Australia

Electronic patient-reported outcome measure use during the management of patients with advanced chronic kidney disease: a pilot/feasibility trial

Derek Kyte, PhD, University of Worcester, Worcester, United Kingdom

Remote follow-up using patient-reported outcome measures in patients with chronic kidney disease: The PROKID study – an ongoing non-inferiority pragmatic randomised controlled trial

Birgith Grove, PhD student, AmbuFlex/Center for Patient-Reported Outcome, Herning, Denmark

Remote monitoring of Cystic Fibrosis (CF) due to COVID-19: use of telephone consultations combined with patient-reported outcome measures (PROMs) – a feasibility study

Liv Marit Valen Schougaard, AmbuFlex, Regional Hospital West Jutland, Herning, Denmark

Moderator:

Maria J. Santana, PhD, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada

Individual Presentations:

A Patients/Family Perspective

D’Arcy Duquette, Health Quality Council of Alberta, Calgary, Alberta, Canada

Measuring experiences of hospitalized patients in Alberta, Canada using validated surveys

Kyle Kemp, PhD, University of Calgary, Calgary, Alberta, Canada  

Categorizing patient concerns using natural language processing techniques

Paul Fairie, PhD, University of Calgary, Calgary, Alberta, Canada  

Innovative System for Patient-Reported Experience in Surgery (INSPiRES): semi-structured emotional reporting system for surgical patients and trainees.

Christopher Gibbons, The University of Texas MD Anderson Cancer Center, Houston, United States

Moderator:

Danielle Lavallee, PharmD PhD, British Columbia Academic Health Sciences Network, Vancouver, British Columbia, Canada

Individual Presentations:

Priorities for advancing methods for measuring what matters: patient and researcher perspectives

Lena Cuthbertson, BHSc (OT) MEd OT(C) PMP, Office of Patient-Centred Measurement, British Columbia Ministry of Health, Vancouver, British Columbia, Canada

Considerations for the assessment and sharing of caregiver-reported outcomes: A qualitative interpretive description study

Fuchsia Howard, PhD RN, University of British Columbia, Vancouver, British Columbia, Canada 

Making Visible the Needs of Vulnerable and Marginalized Populations

Kelli Stajduhar, PhD, University of Victoria, Victoria, British Columbia, Canada

Centering Lived Expertise and Equity in Health Metrics

Saraswathi Vedam, RM PhD FACNM, Birth Place Lab, University of British Columbia, Vancouver, British Columbia, Canada

Moderator:

Michelle K. White, PhD, QualityMetric Incorporated, United States

Speakers:

Tom Willgoss, PhD MSc BA, Roche Products, United Kingdom
Louise Newton, MSc, Clinical Outcome Solutions, United Kingdom
Sarah Knight, MSc, Clarivate, United Kingdom
Selena Daniels, PharmD MS, FDA, United States

Discussants:

Paul Wicks, PhD, Wicks Digital Health, United Kingdom
Mindy Leffler, Casimir, United States

Social media sources allow for pragmatic, cost-effective approaches to incorporating patient perspectives in medical product development research. Risks related to the validity of the data may not outweigh the benefits of efficiency, lower cost, and access to unfiltered data. This symposium aimed to expand knowledge of the benefits and challenges of social media approaches for patient-centric research. Panelists reviewed the use of social media for recruitment, data collection, and data analysis. Strategies for avoiding the pitfalls of problematic research design and analysis plans when incorporating social media sources were examined, with a focus on practical application from lessons learned, including use of case studies. A speaker from the Division of Clinical Outcome Assessment (COA) of FDA addressed the regulatory perspective on the use of social media sources for COA development and patient experience studies. Discussants, including a patient advocate, crafted point-counterpoint summations and questions for consideration.

Moderator:

Angela Wolff, PhD RN, Associate Professor, School of Nursing, Trinity Western University, Canada

Speakers:

Angela Wolff, PhD RN, Associate Professor, School of Nursing, Trinity Western University, Canada
Cecilia Pompili, PhD MD, Leeds Institute of Medical Research at St James’s, University of Leeds, Leeds Cancer Centre, St James’s University Hospital, United Kingdom
Angela M. Stover, PhD, Department of Health Policy and Management and Lineberger Comprehensive Cancer Center, UNC-Chapel Hill, United States
Lorynn Teela, MSc, Emma Children’s Hospital, Amsterdam UMC, University of Amsterdam, Child and Adolescent Psychiatry & Psychosocial Care, Amsterdam Reproduction and Development, Amsterdam Public Health, The Netherlands
Gita Mody, MPH MD, Assistant Professor, Director of Thoracic Surgical Oncology, UNC-Chapel Hill, United States

To facilitate the use of patient-reported outcome measures (PROMs) in clinical practice, past research guided by implementation science theory (IST) primarily focuses on the characteristics of the intervention, organization/care setting, and implementation process; however, a noteworthy gap is the specific needs of clinicians. Understanding these needs and individual factors that influence clinicians’ behavior is paramount for both initial and ongoing PROMs implementation. Our aims were to include the voices of clinicians, as end-users of PROMs data, to facilitate patient-centred care by:
1. Describing IST to inform clinicians use of PROMs in daily practice.
2. Sharing real-world examples examining clinicians’ needs in Canada, USA, Netherlands, and UK.
3. Learning about a User’s Guide that illustrates clinicians’ needs/individual factors.
By emphasizing the importance of engaging clinicians and attending to their needs, the outcome of this symposium was to facilitate behavioural change of clinicians and improve the daily use of PROMs in various practice settings.

Moderators:

Carolyn E. Schwartz, ScD, DeltaQuest Foundation and Tufts University School of Medicine, United States
Richard L. Skolasky, Jr., ScD, Johns Hopkins University, United States

Speakers:

Richard L. Skolasky, Jr., ScD, Johns Hopkins University, United States
Gudrun Elin Rohde, RN PhD, University of Agder and Sorlandet Hospital, Norway
Carolyn E. Schwartz, ScD, DeltaQuest Foundation and Tufts University School of Medicine, United States
I-Chan Huang, PhD, St. Jude Children’s Research Hospital, United States

Discussant:

Joseph Lipscomb, PhD, Rollins School of Public Health, Emory University, United States

For over two decades, quality-of-life (QOL) researchers have studied response-shift effects in patient-reported outcomes. Building on the Sprangers and Schwartz (1999) and Rapkin and Schwartz (2004) theoretical models, researchers have noted paradoxical findings and have regularly attributed them to response shift. The growth in response-shift methods has enabled a stronger empirical foundation for response-shift research, but many of these methods utilize language in framing the research question(s) and interpreting results that treat response-shift effects as ‘bias’, ‘noise’, ‘error’, or otherwise something that should be removed from the results rather than information that matters. Since language influences thought (Sapir-Whorf hypothesis), it is important to become aware of how these ‘habits’ for talking about response-shift effects can be detrimental to the field. This symposium characterized temporal patterns in how response-shift effects have been described in the literature, proposed nomenclature for framing research questions and results, and described a new method for detecting response-shift effects that is feasible across sample sizes.

ISOQOL members are more than their publications and presentations; there’s plenty of hidden creativity out there! During the Annual Conference, we highlighted the talents and hobbies of members—beyond the bounds of research.

Videos and photos were compiled and broadcast live during the virtual conference. Conference attendees viewing live were encouraged to type comments in the chat and share reactions during the performances. The goal of this social event was to provide insights into the unique and creative lives of your fellow scientists—those you don’t know and those you thought you knew.

Attendees got to know fellow ISOQOL members during a virtual speed networking session hosted during the Annual Conference. During a 50-minute session, participants had the opportunity to meet one-on-one with five different people in Zoom breakouts. Each breakout had two people and only lasted 7 minutes before switching to the next pairing. Icebreaker questions were provided but this was an opportunity to informally meet other ISOQOL attendees, just as you would during a coffee break at an in-person conference.