Measuring What Matters 2021
Program Archive
Symposium Theme
Intensive longitudinal measurement:
Methodologies for collecting, analyzing and interpreting patient-generated data from daily diaries, wearables and sensors
As we collect data more frequently in both clinical and real-world studies, patients are often assessed using intensive longitudinal measurement techniques, such as daily diaries, wearable devices, and/or sensor technology. This allows the recording of data at regular intervals between site visits, the capture of acute events as they occur for clinical monitoring, and/or continuous monitoring in order to answer research questions that were previously unanswerable through traditional data collection.
Despite our ability to collect more data, our knowledge about evaluating new data collection techniques and using these rich data sources effectively is limited.
This symposium will cover
- patient-centered approaches to endpoint selection/development
- the subsequent identification of appropriate methods and technology for assessing these endpoints
- the different ways we currently summarize this data and how this leads to a loss of information
- ways to maximise our data using improved summary methods and more complex modelling techniques
Attendees will leave this symposium with new insights and better understanding on how to better use, analyze, and interpret intensive longitudinal data generated from daily diary and wearable devices in a more effective and complementary manner.
Measuring What Matters Symposium Committee Co-Chairs
Pip Griffiths, PhD
IQVIA
United Kingdom
Jiat Ling Poon, PhD
Eli Lilly and Company
Indiana, United States
R.J. Wirth, PhD
Vector Psychometric Group, LLC
North Carolina, United States
Sessions
Session I: Intensive longitudinal measurement and you: How do you measure what matters?
Monday, 19 July | 8:10 am – 9:10 am CDT
Description:
What does intensive longitudinal measurement mean to you and your research? Daily diary, wearables, and sensors all collect lots of data over a continuous period of time. This is known as intensive longitudinal measurement. When collecting this kind of data, there are additional considerations above and beyond those made when using typical clinical outcome assessment measures. How do you translate the patient experience into selecting the right endpoint and data collection modality, in order to handle the resulting gargantuan quantities of data?
Speakers:
Elizabeth (Nicki) Bush, MHS, Eli Lilly and Company, Indianapolis, Indiana, United States
Elizabeth (Nicki) Bush is Senior Advisor and Head of the Patient-Focused Outcomes Center of Expertise at Eli Lilly and Company, responsible for the design and implementation of patient-centered measurement strategies to support drug development efforts across Lilly’s portfolio. Nicki has held research and leadership positions in the patient-centered measurement field in both consultancy and industry, including the role of Industry Co-Director of the PRO Consortium from 2016-2018. She continues to serve as a measurement expert on various taskforces, collaborating frequently with a diverse group of colleagues including those from industry, patient advocacy groups, regulatory agencies, and academia.
Elektra Papadopoulos, MD MPH, U.S. Food and Drug Administration, Silver Spring, Maryland, United States
Elektra Papadopoulos, MD MPH, serves as the Acting Deputy Director of the Division of Clinical Outcome Assessment in the Office of New Drugs in the Center for Drug Evaluation and Research (CDER) at FDA. The Division provides consultation to CDER’s therapeutic area review divisions and other FDA Centers on clinical outcome assessments (COAs) including their development and validation for use in medical product development. Elektra has participated in policy development related to COAs, including multiple methodologic and disease-specific guidance documents. She also leads CDER’s COA Drug Development Tool Qualification Program, which includes innovative assessment methods for use as study endpoints.
Jennifer Goldsack, MChem MA MBA, Digital Medicine Society (DiMe), Boston, Massachusetts, United States
Jennifer C. Goldsack co-founded and serves as the Executive Director of the Digital Medicine Society (DiMe), a 501(c)(3) non-profit organization dedicated to advancing digital medicine to optimize human health. Jen’s research focuses on applied approaches to the safe, effective, and equitable use of digital technologies to improve health, healthcare, and health research.
Session II: Walking a mile in the patient’s shoes
Monday, 19 July | 9:25 am – 10:25 am CDT
Description:
Have you ever had to have a box constantly strapped to your chest that shows from under your clothes, sleep in a helmet, and answer the same question every day for 365 days? How would that make you feel? How likely would you be to do it? The only way we can understand the patients’ study experience is by engaging them at relevant steps in the research journey. Through understanding the patient experience, we can select more practical measurement tools that can lead to data of better quality, while minimizing patient burden.
Speakers:
Bray Patrick-Lake, Evidation Health, Erie, Colorado, United States
Bray Patrick-Lake, MFS is the Director of Strategic Partnerships at Evidation Health. She develops collaborations to support the design and implementation of participant-centered studies and digital measures. She serves on the All of Us National Advisory Panel and the Digital Medicine Society Scientific Leadership Board. Previously, Bray served on the National Academies of Sciences, Engineering, and Medicine (NASEM) Health Science Policy Board, led engagement for the Duke Clinical Research Institute Project Baseline Study Coordinating Center, and served as co-chair on the Advisory Committee to the NIH Director that authored the Precision Medicine Initiative’s Cohort Program.
W. Benjamin Nowell, PhD MSW, Global Healthy Living Foundation, New York, New York, United States
W. Benjamin Nowell, PhD MSW, is Director, Patient-Centered Research, at the Global Healthy Living Foundation (GHLF), CreakyJoints and Principal Investigator of ArthritisPower research registry. Ben’s research interests include examination of the factors that facilitate patient engagement and adherence to providing digital health measures, use of wearable devices in the measurement of disease activity, and patient-reported outcomes and shared decision making in rheumatologic conditions. Prior to joining GHLF, Ben worked as a medical social worker and Community and Long-Term Care Coordinator for the Ottawa Regional Stroke Centre, and Research Coordinator for evaluation of Arthritis Foundation chronic disease self-management programs.
Session III: Having your cake and eating it, too: Mixing modalities to enhance interpretation of data
Monday, 19 July | 10:40 am – 11:40 am CDT
Description:
So you want to have your cake and eat it, too? Continuous wearable and sensor data is great, but without context it can be difficult to interpret. Diary data provides rich information about how patients feel and perceive their functioning. Together, one provides the broader context to the finer detail of other. However, it is important to carefully consider when it is necessary to collect data from multiple modalities, and when it is sufficient and less burdensome to rely on fewer. Which begs the question, is too much cake bad for you?
Speakers:
Sonya Eremenco, MA, Critical Path Institute, Tucson, Arizona, United States
Sonya Eremenco is Director of the Patient-Reported Outcome (PRO) Consortium at the Critical Path Institute. Sonya has over 20 years of experience in PRO (and other clinical outcome assessment) measure development, with a focus on multicultural development, linguistic validation, and electronic implementation. She currently serves on the Steering Committee of ISOQOL’s Translation and Cultural Adaptation Special Interest Group. Sonya holds a Bachelor of Arts in Cultural Anthropology from Duke University and a Master of Arts in Multicultural Communication from DePaul University.
Jay Trudeau, PhD, Janssen Global Services, LLC, Raritan, New Jersey, United States
Dr. Jeremiah (Jay) Trudeau is currently a Director in the Patient-Reported Outcomes group of Janssen Pharmaceuticals and has been working in the field of novel clinical outcome assessments since 2007. His current areas of responsibility include oncology and digital health innovations. He has previously worked in immunology, pain, and cognitive disorders. Jay received his PhD in Experimental Psychology from the University of Connecticut with a specialization in Quantitative Research Methods. Prior to working in industry he held academic appointments at Mt. Holyoke College and Hampshire College.
Session IV: Why did the researcher throw away their data? - How to handle intensive longitudinal data as a research outcome
Tuesday, 20 July | 8:00 am – 9:15 am CDT
Description:
Have you ever felt like you were drowning in data and were tempted to summarise it as an average? The central tendency is not your friend. Reducing data leads to a loss of information and while this can help make data sets manageable, there are techniques available to avoid this. Intensive longitudinal data analysis can maximise the use of your collected data and allows you to answer a range of additional questions. If you’ve got a lot of data, why not try and use it all?
Speakers:
Carrie Houts, PhD, Vector Psychometric Group, Tempe, Arizona, United States
Carrie R. Houts, PhD, is director of psychometrics at Vector Psychometric Group, LLC, joining the company in 2011. She received her PhD in Quantitative Psychology from The Ohio State University. Her primary areas of interest involve methodological issues in measurement with a special focus on item response theory and factor analysis and the application of modern measurement techniques to substantive areas. Her methodological and substantive works have appeared in such publications as Psychological Methods, Applied Psychological Measurement, Journal of Consumer Affairs, and Quality of Life Research.
Don Hedeker, PhD, University of Chicago, Chicago, Illinois, United States
Donald Hedeker, PhD, is a Professor of Biostatistics in the Department of Public Health Sciences at The University of Chicago. Don’s main expertise is in the development of methods for clustered and longitudinal data, with emphasis on mixed-effects models. With Robert Gibbons, Don is the author of the text “Longitudinal Data Analysis,” published by Wiley in 2006. He is also the primary author of several freeware computer programs for mixed-effects analysis. Don was named a Fellow of the American Statistical Association in 2000, and he is an Associate Editor for Statistics in Medicine.
Willie Muehlhausen, Safira Clinical Research, Dublin, Ireland
Willie is a successful intrapreneur who turned entrepreneur when he co-founding Safira Clinical Research in 2021. For more than 20 years he worked with leading eCOA vendors and for the last 10 years he collaborated with other researchers to improve patients’ experience in a clinical trial. Willie has led initiatives in BYOD, wearable devices, instrument validation process and standardisation of implementation processes. His work helps to understand what matters to patients and supports eCOA products and DCT projects globally. Willie currently researches possible uses of AI/ML and Blockchain in eCOA to further improve the eCOA experience for all.
Session V: What makes a step a step? A step by any other algorithm…
Tuesday, 20 July | 9:30 am – 10:45 am CDT
Description:
What makes a step a step? Research endpoints need to have meaning and context. Wearables and sensors do not directly count meaningful units such as steps and heart rate. Instead, they measure raw data, such as x-, y-, z- coordinates, or blood volume (photoplethysmography). Before it is ready for our analysis and interpretation, algorithms run complex operations on the raw data to add meaning and context – turning coordinates and blood volume into steps and beats. But…is the definition of a step or a heart beat the same for all patient groups?
Speakers:
Brinnae Bent, PhD, Edge Analytics, Durham, North Carolina, United States
Brinnae Bent was a data science researcher and lead developer of the Digital Biomarker Discovery Pipeline. Her doctoral thesis in the Big Ideas Lab at Duke University focused on developing digital biomarkers of glycemic health from noninvasive wearables. She has led projects establishing best practices for wearable sensor validation, investigating wearable sensor device inaccuracy, optimizing sampling rate and data compression of wearable sensors, and using wearable sensors for human activity recognition. Previously, she worked with the NSF ASSIST Center to develop the next generation of wearable sensors for sleep disorders and asthma.
Ieuan Clay, PhD, Evidation Health, Munich, Germany
Recognised leader in the field of Digital Health: More than 10 years experience developing and implementing new sensor-based measures for clinical development and post-market applications; Extensively published, editor, lecturer, invited speaker. Bringing together broad experience in applied data science, clinical practice and large-scale data handling to advance the clinical utility of digital healthcare technologies to improve patient lives.
Ilya Lipkovich, Eli Lilly, Indianapolis, Indiana, United States
Ilya Lipkovich is a Sr. Research Advisor at Eli Lilly and Company. Ilya received his PhD in Statistics from Virginia Tech in 2002 and has more than 15 years of statistical consulting experience in pharmaceutical industry. He is an ASA Fellow and published on subgroup identification in clinical data, analysis with missing data, and causal inference. He is a frequent presenter at conferences, a co-developer of subgroup identification methods, and a co-author of the books “Analyzing Longitudinal Clinical Trial Data. A Practical Guide” and “Estimands, Estimators and Sensitivity Analysis in Clinical Trials.”
Session VI: When good science meets good standards - Regulatory considerations around intensive longitudinal data
Tuesday, 20 July | 11:00 am – 12:00 pm CDT
Description:
Are regulatory standards fluid enough to accommodate the evolving landscape of collection and analysis of intensive longitudinal data? Algorithmic interpretation of raw sensor data is a prerequisite for analysis. However, algorithms are typically black boxes, the validity of which is difficult to demonstrate to regulators. Similarly, intensive longitudinal analytical techniques are not commonplace in regulatory decision-making. Such techniques provide powerful ways of understanding patients’ health, but as with emerging technologies, it is as yet unclear what level of evidence is needed to demonstrate this. How do we strike a balance between accommodating evolving techniques and maintaining scientific and safety standards?
Speakers:
Andrew Potter, PhD, U.S. Food and Drug Administration, Silver Spring, Maryland, United States
Dr. Andrew Potter is a mathematical statistician in the Division of Biometrics I in CDER supporting the review work in the Division of Psychiatry. His research interests include the use of digital health technologies in clinical trials and the analysis of high frequency outcome data and in involved in working groups at FDA on this topic. He received his PhD in Biostatistics from the University of Pittsburgh.
Jennifer Goldsack, MChem MA MBA, Digital Medicine Society (DiMe), Boston, Massachusetts, United States
Jennifer C. Goldsack co-founded and serves as the Executive Director of the Digital Medicine Society (DiMe), a 501(c)(3) non-profit organization dedicated to advancing digital medicine to optimize human health. Jen’s research focuses on applied approaches to the safe, effective, and equitable use of digital technologies to improve health, healthcare, and health research.
Session VII: A World of Pure Imagination: The evolving landscape and undefined future
Tuesday, 20 July | 12:15 pm – 1:15 pm CDT
Description:
Throughout the preceding sessions, there have been a lot of questions answered, but many more have been raised. Our panelists will debate whether diaries and sensors are distinct COA measures, or whether they are individual components of a larger assessment of the patient experience that need to be considered together rather than in isolation. In the changing landscape of clinical development, these are important questions that need to be considered to guide where we go next.
Speakers:
Sonya Eremenco, MA, Critical Path Institute, Tucson, Arizona, United States
Sonya Eremenco is Director of the Patient-Reported Outcome (PRO) Consortium at the Critical Path Institute. Sonya has over 20 years of experience in PRO (and other clinical outcome assessment) measure development, with a focus on multicultural development, linguistic validation, and electronic implementation. She currently serves on the Steering Committee of ISOQOL’s Translation and Cultural Adaptation Special Interest Group. Sonya holds a Bachelor of Arts in Cultural Anthropology from Duke University and a Master of Arts in Multicultural Communication from DePaul University.
R.J. Wirth, PhD, Vector Psychometric Group, LLC, Chapel Hill, North Carolina, United States
R.J. Wirth is CEO, co-Founder, and a managing partner at Vector Psychometric Group, LLC as well as CEO and Chairman at Scientific Software International, Inc. Dr. Wirth received his PhD in quantitative psychology / psychometrics from the historic LL Thurstone Psychometric Laboratory at University of North Carolina – Chapel Hill. For more than a decade, Dr. Wirth has collaborated within the pharmaceutical and medical device industries to develop/evaluate the measurement properties of clinical outcome assessments. Dr. Wirth has authored numerous peer-reviewed publications on such topics as item factor analysis, structural equation modeling, validity, as well as various aspects of scale development.
Elektra Papadopoulos, MD MPH, U.S. Food and Drug Administration, Silver Spring, Maryland, United States
Elektra Papadopoulos, MD MPH, serves as the Acting Deputy Director of the Division of Clinical Outcome Assessment in the Office of New Drugs in the Center for Drug Evaluation and Research (CDER) at FDA. The Division provides consultation to CDER’s therapeutic area review divisions and other FDA Centers on clinical outcome assessments (COAs) including their development and validation for use in medical product development. Elektra has participated in policy development related to COAs, including multiple methodologic and disease-specific guidance documents. She also leads CDER’s COA Drug Development Tool Qualification Program, which includes innovative assessment methods for use as study endpoints.
Symposium Sponsor
The International Society for Quality of Life Research (ISOQOL) is a global community of researchers, clinicians, health care professionals, industry professionals, consultants, and patient research partners advancing health related quality of life research (HRQL).
Together, we are creating a future in which patient perspective is integral to health research, care and policy.