Survey Says… MU Isn’t Usability

On September the 11th, 2012, I had the pleasure of speaking at the Health IT Meaningful Usability Summit hosted by HealthIT Now and Sudler eHealth in Washington, D.C. Our mandate was to explore pressing issues and current challenges in eHealth, and to reflect on how they impact and drive usability of technology.

Other speakers included representatives from Congress, The Office of the National Coordinator, The Institute of Medicine, Caregivers and Healthcare Providers. I felt that this would be a great opportunity to dig deeper into the data we’ve been collecting from AmericanEHR ratings over the last couple of years, and share it with key stakeholders in the health IT industry. My goal for the summit was to showcase the power that an online community can have in terms of collecting and reporting user experiences.

Download the entire presentation “Survey Says… MU Isn’t Usability” (PDF format).

I started with a quick overview of AmericanEHR’s growth to date:

  • 21 Partnering Medical Societies representing over 720,000 licensed physicians
  • Over 15,000 registered members, and over 80,000 clinicians surveyed to date
  • Over 3,500 detailed EHR satisfaction ratings (139-question survey) completed since launch
  • Overview of key features: EHR Product Ratings, Top 10 Dashboards, Comparison Tools, and Advanced Search

I then did some deeper data analysis, and offered some previously unpublished EHR rating information. First, we looked at the benchmark ratings scores for the Average Satisfaction Ratings per Category across all EHR systems. What we discovered were very high satisfaction scores for ePrescribing (3.81), Purchase Experience (3.66), and Order Management (3.56), and much lower overall rating scores for Interfaces (3.36), Population Management (2.92), and Workflow Management (2.83).

We then looked at the Correlation Between MU Attestation and EHR Satisfaction. To do this we took the top 10 products by MU attestation (attestation data was derived from Data.gov), and compared them to their average AmericanEHR product rating. The results demonstrated that 50% of the top 10 most attested EHR products (Centricity EMR, NextGen, Allscripts, athenaClinicals, and Vitera EMR) were actually scoring below average in terms of overall EHR product satisfaction. We determined this by comparing each product’s average satisfaction rating to our industry benchmark scores — which we calculate by totaling the average cumulative score, for all approved EHR ratings, across all EHR products.

Another key finding was Top 10 Satisfaction Ratings vs. Attestation Share. We listed the top 10 products rated for overall satisfaction on AmericanEHR and correlated that with the corresponding MU attestation market share data for each EHR product. The results stimulated many spirited responses from the audience, and this conversation flowed well into the post presentation Q&A session. The reason for all the attention was simple — our EHR satisfaction rating data strongly indicates that the more likely a product is to achieve Meaningful Use, the less satisfied users are with it. If these indications are accurate, this is a trend that will need to change in order for the EHR stimulus program to be a long-term success.

At the conclusion of the summit the event’s host, Joel White, Executive Director of Health IT now singled out the ratings work that AmericanEHR has been doing, and suggested that this type of peer review may be an important part of solving the communication divide between practitioners and policy makers in order to promote better usability of health IT.

I couldn’t agree more. AmericanEHR is the culmination of over five years of work between Cientis Technologies and the American College of Physicians to develop tools to help medical practices adopt, implement, and effectively use health IT. All of the product recommendations are derived from verified medical practitioners, thus providing AmericanEHR members with a high level of confidence. Medicine has always been about developing evidence-based best practices, and user feedback is a critical component in terms of developing a superior EHR product.

One of the challenges with collecting user feedback is the presence of “gag clauses” in many EHR vendor’s contracts. Under these contracts EHR users are forbidden to publicly comment on, or provide feedback on, their product in a way that could be perceived as negative. In other words, if you warn your peers about challenges they may face with a product, you personally risk litigation. At AmericanEHR we believe in the power of community feedback, and social media, and we would strongly discourage any practice from signing an agreement that forbids you from sharing your user experiences.

In addition to new data, we also released two slides showcasing some de-identified qualitative feedback received during EHR ratings. These comments showcase the frustration, and disruption that was experienced by many practices through not only adoption of an EHR, but also from those who chose to migrate to another EHR system, often with little improvement.

Everyone knows that EHR adoption is a huge undertaking, but what many don’t know is that there is also a high likelihood of failure. Medscape put this eloquently in a recent article: “Use of electronic health records is snowballing, and so is the number of unhappy users. Half of EHR systems sold to physician practices are now replacements, up from 30% last year, according to a recent study by research firm KLAS.”

If this number seems high to you, then think again. Over the last two years we’ve asked all of our EHR raters if they would recommend their EHR to a colleague — of all respondents, only 47% said yes. That seems to substantiate KLAS’s findings.

After reviewing our data, I put forward some suggestions at the summit that I believe will promote greater health IT usability:

  1. Providers need to share experiences openly
  2. Incentive funding could be linked to mandatory product ratings
  3. Certification should incorporate benchmarks for satisfaction
  4. Transparency should be facilitated by disallowing “gag orders” for certified products
  5. Vendors should be able to leverage user feedback in order to make product and support improvements

In the spirit of collaboration, I’d like to throw this one back to the AmericanEHR community: What recommendations would you make to drive better usability in health IT?

Download the entire presentation “Survey Says… MU Isn’t Usability” (PDF format).

Leave a Reply

Your email address will not be published. Required fields are marked *