Project abstract (1000 Characters): Patient Reported Outcomes (PROs) directly reflect the “voice of the patient” and provide an avenue for patient engagement and shared decision making. PROs provide a quantifiable bridge between subjective patient symptoms and subjective physician interpretation. However, various challenges prevent PRO data from being routinely available electronically for clinical care or for research use. PRISM is a user-friendly mobile app that makes it easy to collect standardized PRO patient data in various clinical settings and seamlessly integrates that data into the electronic health record (EHR) making it available to clinicians at the point of care. Patients have found the app to be easy to use and facilitates quick survey completion. Clinicians have found the PRO data to help inform their clinical decision making and support changes to clinical care. PRISM uses the latest Fast Healthcare Interoperability Resources (FHIR) standard for accessing PROs and for storing the results into the EHR.

Project rationale, impact and innovation (3500 Characters): Patient Reported Outcomes (PROs) directly reflect the “voice of the patient” and provide an avenue for patient engagement and shared decision making. But while they have the potential to improve care delivery, quality and value, they have not been widely adopted into routine clinical practice. PRO use in individual treatment plans remains limited due to poor value propositions for both patients and providers. Typically, the PRO instruments are administered on paper or through an external electronic survey at the point of care. Results are then scanned into the EHR and since they are not entered into discrete fields, clinical decision support cannot be driven from the PRO data. In addition, it is challenging to share PRO data with other providers because these data are not standardized. Usually, providers find it difficult to view the information, which impacts utilization rates. Finally, this manual method usually requires patients to be in the provider’s office to complete the survey, but many PROs vary day to day and the variation between visits is not captured. To address these barriers, there needs to be an easy and timely way to collect standardized PRO data while patients are outside the clinical setting with the ability to integrate results into discrete fields within an EHR to inform individual clinical decisions.

PRISM is a user-friendly mobile app that addresses these problems. The app makes it easy to collect standardized PRO data in both clinical and non-clinical settings with seamless data integration into the EHR. The design of the PRISM app was guided by the success of Fairview Health Kidney Stone Institute (KSI) which has been successfully using Patient-Reported Outcomes Measurement Information System (PROMIS) (a validated set of PROs developed by the National Institutes of Health (NIH)). Their experience helped us create a user friendly and engaging app. The innovative features of the PRISM app include:
1. The app can automatically update the EHR with structured data using the latest HL7 FHIR standard.
2. Patients have options to answer the PROMIS questions before, during or after the visit including having
clinic staff help fill it out. Multiple devices are supported via a secure mobile or web application. Patients are reminded to complete surveys via convenient SMS messages.
3. The app displays a patient’s historical trends of their PROMIS measure scores, how they compare to other patient populations, and personalizes recommendations of educational materials. These comparisons and recommendations help promote patient engagement with the app.
4. The app makes it easy to answer PROMIS questions, including using Computer Adaptive Testing (CAT) to reduce the number of questions a patient has to answer while still maintaining the validity of the results.
5. Patients can answer PROMIS measure questions during the clinic intake process using their own mobile phone instead of having to rely on a clinic provided device.

In the short term, PRISM can engage patients to track their outcomes and make PRO data available for shared decision making to improve patients’ outcomes. PRISM can be used in either primary care or specialty care practices; thus, it has the potential to improve all ambulatory care patients’ quality of care. In the long term, standardized PRO data collected through PRISM can be used for population health management and other research purposes through efficient data pulling from different sources.

Project design and implementation (7000 characters): The Agency for Healthcare Research and Quality (AHRQ) sponsored a contest called the “Step Up App Challenge: Advancing Care Through Patient Assessments” to encourage development of mobile apps that simplify the process of collecting, aggregating, and sharing standardized PRO data in the ambulatory care setting. The challenge asked participants to use the latest HL7 FHIR technical specifications when developing the app so that standardized PRO data can be easily captured and move between vendor systems both within and across different providers, as well as through third-party applications for direct use by both clinicians and patients.

The PRISM team developed the app based on the success that the HealthEast Kidney Stone Institute (KSI) has had using paper based PROMIS measures. PRISM was developed by a strong multidisciplinary team with experience in clinical use of PROMIS measures, mobile software development, and clinical analytics using healthcare data. The mobile app uses Computer Adaptive Testing (CAT) to reduce the question burden and uses SMS notifications to streamline processes and give patient convenient reminders. Patient engagement is fostered by giving patients realtime feedback of their measure trends, where they fit in the overall population, and personalized recommendations. The platform supports any PRO instrument that uses the HL7 FHIR technical specifications and allows seamless integration with EHRs.

All KSI patients completed PROMIS scores at each clinic visit using a paper based form. KSI discovered that patients with pain intensity scores >60 were 7 times less likely to attempt stone passage and 2.7 times more likely to fail attempted passage. Implementation of a standardized transition of care protocol from Emergency Department to KSI clinic was associated with a 50% reduction in elevated PROMIS scores, a 25% increase in patients attempting stone passage, and a 24% reduction in surgical intervention. This specific example illustrates the benefits of PROMIS instruments at the individual and population levels, which could be generalized across specialties and to other PROMIS instruments.

The PRISM app incorporates lessons from KSI’s use of PROMIS measures. The lessons include:
1. Make it very easy and intuitive to fill out the questions for the surveys
2. Give the patient feedback for how they scored, what the score means and how it compares to an “average” patient
3. Discuss the scores and the PROMIS surveys during office visits so that the patients are more engaged and
know that their answers matter and are part of their care.

In addition to ease of use, our solution, PRISM, addresses the “4Ps” of mobile health (mHealth) applications. It is:
1. Personalized to the patient: the app is user friendly, and can be accessed through multiple devices and a web site and allows flexibility of being completed outside of the clinical environment
2. Predictive of patient measure scores based on computer adaptive testing (CAT) item response theory to ask as few questions as possible yet still produce a valid score
3. Participatory by patients, providers, and health systems: designed with a patient/staff/clinician collaboration model in mind. Staff can initiate and monitor PRO measures. Through a FHIR connection, PRISM sends the results into discrete fields within the EHR providing systems the opportunity to build point of care clinical decision support and quality metrics off the data from within the EHR.
4. Preventive of unnecessary emergency medical care: Optimized to help identify sub-optimal outcomes which may result in readmissions, and inappropriate emergency room visits.

The PRISM app is designed and built using modern open web standards. We did not reinvent the wheel so we used, as much as is possible, prebuilt industry standard resources for things such as hosting, security, and data standards. We relied heavily on the HL7 FHIR standards for interactions with external systems.

The PRISM app complies with HIPAA and is architected to allow protected health information (PHI) and the PROMIS Measures collected by the app to be stored and maintained both within the EHR and outside of the EHR on secure HIPAA-compliant servers from Amazon Web Services (AWS) and uses industry best practices for healthcare software development and delivery to ensure HIPAA compliance.

As with many software projects, many issues we encountered were not technical. The process for deploying a mobile app in a health system involves many critical processes for ensuring the software is of high quality, patient privacy is protected, security is addressed and clinical workflows are not impacted. These reviews and approvals took more time than we planned, but they were critical to ensuring the integrity of the app.

Project evaluation and sustainability (3500 characters): For the initial development of the PRISM mobile app, the team conducted extensive user experience design sessions. Patients from diverse populations (across different age ranges, clinical conditions, and technology comfort levels) gave feedback on various iterations of PRISM user interfaces. Based on this feedback, the team was able to incorporate user experience aspects that were critical from a patient’s point of view.

After patients used the app, we assessed their impressions of the usability through the System Usability Scale. Overwhelmingly, the app scored well (92% rated it Good or Excellent). One critical piece of feedback we received was that patients who lack access to cell phones with data plans or email should not be target users for PRISM. Through this usability testing, we learned that the following features were crucial to adoption 1) the app’s ease of registration and use, 2) SMS reminder notifications, 3) long-term results tracking, 4) links to educational materials, and 5) seamless integration into EHR systems.

The PRISM app is now being pilot tested with support from AHRQ in nine practice settings affiliated with MedStar Health in Washington, D.C., Maryland, and Virginia. MedStar rolled out the PRISM app to these sites. MedStar is leading the integration of the PRO data into different EHR systems and evaluating the implementation process and the usability of the app. As part of the implementation MedStar is also leading the training of practice site administrators on the use of the app. The planned evaluation will assess both the technical integration and the implementation in the nine primary and specialty care practices. Medstar has designed a rigorous evaluation to assess the feasibility and impact of the testing at each pilot site. Evaluation metrics have been chosen to identify facilitators and barriers to implementation and use of the app from the perspectives of each stakeholder group (patients, providers and health IT staff). Evaluation will include pre-implementation site assessments to identify existing workflows and site-specific requirements for integration of the app. Patient-specific evaluation will include administering the Systems Usability Survey (SUS), and semi-structured interviews to further assess satisfaction, preferences and barriers to use of the app. Provider and health IT staff evaluation will include semi-structured interviews to identify facilitators and barriers to implementation and use of the app and the integration with the EHR, as well as the perceived usability and utility of the PRO data visualized within the EHR. The results of this study are expected to be available in October 2019, in time to report at the AMIA Annual Symposium in November.

Our sustainability model is to release an open-source version of PRISM so that it can continue to be enhanced and become more widely adopted. There are numerous examples of successful open-source projects being widely adopted within healthcare. The hope is that PRISM can remove implementation barriers and make it easy for patients to provide PROs and have those PROs seamlessly integrated into the EHR. We will also disseminate lessons learned from the implementation at the nine sites to inform future implementations at other ambulatory sites.

Twitter project summary (140 characters): PRISM makes it easy for patients to submit Patient Reported Outcomes integrated with the EHR to be used by physicians during clinical care

How is FHIR used in the App being demonstrated (500 characters)? : HL7 FHIR for structured data capture (http://build.fhir.org/ig/HL7/sdc/) is used in multiple areas of the PRISM app. First, PRISM uses the FHIR Questionnaire bundle to retrieve PROMIS measure questions and deliver them to the patient via the mobile app. Secondly, PRISM uses the FHIR QuestionnaireResult bundle to send the answers (PROs) to the EHR to be stored with the patient’s health record.

1. What FHIR release does your application use? (500 characters)?: PRISM uses R4 (v4.0.0) of the FHIR spec release December 27, 2018

What is the data source for the FHIR resources and how are the FHIR resources accessed? (500 characters): The data source for the PROMIS surveys and instruments uses FHIR bundles (Questionnaire, QuestionnaireResponse) retrieved from a server that provides an API for Computer Adaptive Testing (CAT) versions of the PROMIS measures.

Any other information about the project we should know about (1500 characters)?: We want to clarify one point: PRISM does not have paying customers since the PRISM app will be released as open source and freely available for any organization to use.

Authors:

Steven Johnson (Presenter)
University of Minnesota

Chun-Ju Hsiao, Agency for Healthcare Research and Quality
Deliya Wesley, MedStar Health Research Institute
Joseph Blumenthal, MedStar Health Research Institute
michael borofsky, University of Minnesota
Andrew Portis, Fairview Health System
Zach McGill, PerkHealth
Doug DeBold, PerkHealth
Elisha Friesema, University of Minnesota
De Liu, University of Minnesota
Pinar Karaca Mandic, University of Minnesota