Letter from the President: Evaluation

By ISOQOL President Lori Frank, PhD

I’ve written in the last few months about member inreach, about Society outreach, and now I’d like to conclude my blogs as President with a reflection on evaluation. 

We have been following the ISOQOL Strategic Plan, which was finalized in 2017. The plan is aging now – it ends in 2022. Claire Snyder initiated the process and Chema Valderas steered the Society through the early years of the strategic plan. I’ve been fortunate to have my term as President coincide with the latter part of the implementation of the Strategic Plan. We’ve been reviewing progress and adapting course as needed all along, and flexible and quick adaptation is one practice I worked to incorporate into ISOQOL. But out with the old, in with the new… 

Task Forces, Committees, and the SIGs have completed substantial work toward the strategic goals, and we’ve had the benefit of additional ISOQOL staff to be able to devote the time needed.  One of the concrete outcomes of this effort was the first (!) and fabulous redesign of the ISOQOL website, which had been pretty much the same since I joined ISOQOL in the 1990s. That was a long time ago and the website was due for an update. The new website brings great new capabilities for connection. The advances made on the education-related offerings to support the Strategic Plan are also very impressive. We’ve substantially expanded offerings and are ready to provide an online offering.  

The work of the collaborations committee is also key to strategic success, and new partnerships have been established. For example, we partnered with AAAS for the annual conference seminar on communicating science. During this time we have reorganized the regional network and special interest group (SIG) structure, strengthened the cross-SIG sharing including through a reframed SIG Council meeting at the annual conference, and added new SIGs!  

The work of the SIGs, beyond the numbers of them, is where substantial Society work gets done. We’ve had extremely productive SIGs, with multiple manuscript submissions based on SIG initiated projects, and several workshops and symposia emerged from our SIGs during this time. 

So – this sounds like success, but how do we know?  

When I joined the 2019 Board meeting as President-Elect I asked the Board to consider self-evaluation of their work for the Society. How did the work they were doing compare to what they said they wanted to do on their Board ballots? Now that they were familiar with the Strategic Plan and the strategies designed to meet the strategic goals, how should they restructure the time they invest in ISOQOL? How are we, collectively, performing as a Board? Are we meeting our Board-level goals? 

We also asked ourselves: what are the outcomes of all of the activity around the Society? Is it what we want it to be? Where are changes warranted? How can we make quick changes as needed, in response to changing conditions in our field, and in response to other changes like, say, a global pandemic? I’ve learned a great deal about how ISOQOL can be responsive to changes without upending the orderly pursuit of strategic goals. The need to pivot to virtual conferences is a prime example.   

As a small Society we must constantly be monitoring our environment to identify opportunities for ISOQOL’s authoritative leadership to benefit the field, and to identify ways our members can benefit, even beyond our strategic plan.  

So – How are we doing?  

Over the last two years I found it instructive to reflect on the ways in which an evaluation lens has influenced my thinking about ISOQOL. In my own work, that evaluation view has deepened not just what I learn but how I learn it. 

Our field has changed a great deal over the last decade, especially in terms of taking the inclusion of the patient voice in patient-reported outcomes and expanding it in different directions. The expansion was toward multiple forms of patient-generated data, incorporating technology directly and in a new way, and the expansion extended to engagement of patient and other stakeholder in research itself. Rather than patients as research subjects only, now inclusion of the patient voice was changing to mean working with individual patients, the patient community in various forms, and with others who are not researchers, to hear what we hadn’t heard before and to learn to see things from different perspectives.  

This was hard change and not welcome in many quarters.  At the time, ISOQOL reflected on its own structure and established a Task Force to determine how best the Society should meet the new challenges to measurement that this reframed view of research created, and how members, as researchers, should meet the challenges. The Task Force is now an active SIG on Patient Engagement, and the principles of this engagement cut through an increasing number of members’ work. Change is messy and hard some times, but can yield worthwhile benefits like this. 

An aspect of patient engagement that I think gets lost in many conversations is the value of multiple perspectives. Historically researchers in health outcomes have thought deeply about capturing perspectives, more so than in many other disciplines, but this thinking is based on our training in objective empiricism. From my work with persons living with dementia in outcomes assessment, I have learned a great deal from and about those with the lived experience of disease. In reflecting on the learnings the following points have served as good guideposts:   

1. Time is of the essence. With neurodegenerative disorders, the opportunity to learn from the perspective of those affected has a notable time limitation. The resulting sense of urgency can be good for science. More generally, important work needs to move efficiently.  
2. Communication within research teams can be unidirectional and didactic, and researchers and clinicians have valuable information to teach research partners and other stakeholders interested in research. There’s also value, though in seeking out what others can teach about our work. In my own research I learned specific and useful information about how best to manage coauthorship through conversations with the persons living with dementia who were part of the work.   
3. Attend to process as well as content: A focus on work content is required but addressing the process, research conduct, is also crucial, particularly if partnering with nonresearchers. Are we including the right subjects? Are we including the right partners? Are we open to hearing views of others about our work? Which decisions support the empirical work, and which support ethical dimensions of the work? 
4. We are all people with views and with emotions. Working with persons living with dementia requires accommodation to interruptions caused by the disease.  Here we all are now, with pandemic interruptions to our work. What does our reaction to these changes mean to our own input to and reactions about our work? My father was dying, with dementia, as I completed a project with persons living with dementia. I brought something different to the project and took something different from it as a result. Reflecting on the objectivity I wanted to bring, I concluded that I need to bring dual minds to my own work. There’s the empirical, subject/object view that enables us, sometimes, to see things more objectively than we would otherwise, evaluating evidence carefully. Then there’s the experiential view that enables us, sometimes, to see things in a different way, which can be to the benefit of the work. The hard part is to be mindful of which view we are prioritizing. 

ISOQOL involves managing the tension between keeping things the same and striking out into new areas.  So much about the early days endures. So much innovation is happening as well! My goal is to help keep ISOQOL moving forward with a sense of urgency, an openness to learning, and attention to the ways in which our training and our biases influence the Society. I look forward to watching the future leaders in ISOQOL thoughtfully evaluate the work of the Society, to keep the best and to also creatively change in the future.