Comparing User Experiences of Health IT Products

A discussion paper just published by the Institute of Medicine (IOM) and developed at the request of the Office of the National Coordinator for Health IT (ONC), “Comparative User Experiences of Health IT Products: How User Experiences Would Be Reported and Used” describes a number of ways “to generate, house, disseminate, and thus maximally leverage comparative user experience to improve the health IT environment and improve patient care.” The discussion paper was developed following a recommendation from the November 2011 consensus report, “Health IT and Patient Safety: Building Safer Systems for Better Care.”

The authors state, “We hope public reporting of user experiences with different health IT products will promote a more robust environment for improvements in health IT design, as it does in other industries. Consumer Reports ranks products, influencing purchasers in their decisions and, arguably, manufacturers in their future product development. Websites and social media forums provide public vehicles for users to share experiences, learn from others, and move the marketplace, and can thus harness the collective experiences of customers to affect change.”

Because it is difficult to meet all needs for comparative user experience using a single measure,  the report suggests a multi-modal approach towards collecting and reporting  on user experiences including:

  • In vitro “flight simulator” laboratory evaluation of test scenarios
  • In vivo point-of-use reporting
  • Data mining of use patterns
  • Third party-administered user surveys
  • Direct user-to-public reporting, and
  • A formalized system of hazards reporting.

No technology is perfect. As users become more accustomed to their EHR system they develop mechanisms to work around some of the hazards that are specific to a product, resulting in that hazard being invisible or unrecognized as a necessary key improvement. In addition, the opportunity to compare EHR products against one another in terms of their abilities to support (or inhibit) new models of care is another advantage of a robust comparative system.

One of the key recommendations is to find ways to remove the barriers to reporting problems with usability of EHRs. Users are often fearful of retribution by their EHR vendor, vendors fear liability issues, and the time-consuming nature of reporting in conjunction with concerns regarding the impartiality of the entity collecting the data limit the ability to collect this information.

The report suggests a single website, hosted by a trusted government entity, to serve as a hub for comparative user experiences, linking to various sources of data including third-party evaluations and other resources such as AmericanEHR Partners and KLAS.

The authors also recognize that because of the diversity of clinical situations and user needs, a single rating score is not the goal, rather an ability to assess the importance of specific modalities that are important to that specific user.

From our experience collecting EHR comparative data over a number of years via AmericanEHR Partners, we have identified additional factors that influence the measurement of EHR usability. These include the need for specialty-specific comparative data, the dynamic nature of the market with multiple versions of the same EHR product that are in use but have different features and functionalities, and the importance of near real-time reporting of data to ensure that the information is relevant. We welcome the opportunity to participate in this important process as it further develops.

What do you think about the IOM’s proposals for assessing user experiences? Would information from such a resource have been helpful to you in your selection of an EHR? If you are in the process of identifying an EHR, how would you use such information? Add your comments below.

Leave a Reply

Your email address will not be published. Required fields are marked *