User Experience and Comprehensibility of German Public Reporting Portals That Present Hospital Outcome Data
|
If you are the presenter of this abstract (or if you cite this abstract in a talk or on a poster), please show the QR code in your slide or poster (QR code contains this URL). |
Abstract
Background: An increasing number of health portals publish information about the quality of health care providers. Outcomes data for hospitals, like mortality rates, are considered to be a valid source of information when choosing an appropriate provider. Limited information is available whether the information presentation approach which is used by different portals influences the interpretation of data and the user satisfaction.
Objective: To investigate whether the information presentation of four portals 1) influences the interpretation of data by users, 2) influences the user experience and 3) whether reasons can be identified to account for these influences.
Methods: Internet pages from four leading German public reporting portals (Krankenhausspiegel, Techniker Krankenkasse, Weisse Liste, Qualitätskliniken) containing outcome quality indicators about coronary artery bypass graft surgery or primary hip replacement were presented to 35 subjects. Subjects were then asked to determine whether one hospital had better results than another one or than the national average. Additionally, they were asked to substantiate their decision and were asked to assess the usability of the portals. 56 experiments were recorded with the use of eye tracking.
Results: The four portals named the outcome indicators in different ways, published different details of quality information and used different tables and graphical representations of data. In 13 of 56 experiments (23.2 %) the subjects did not correctly identify the better hospital. 57.1 % of subjects did not understand at least one technical term, still 37.5% more than one term and 55.4 % stated their comprehension to have been impaired by technicals term they hadn’t understood. 39.3% were not satisfied with the knowledge that they had gained when using the respective portal.
6 of 14 subjects using “Techniker Krankenkasse†did not correctly identify the hospital with better outcome data. 12 stated a negative impact on comprehension because of incomprehensible technical terms. 9 were dissatisfied with the portal. The very concise tabular presentation and the almost complete absence of explanation of technical terms might have led to discontent and misinterpretation of data.
3 of 9 subjects using “Qualitätskliniken†did not identify the better hospital. 4 reported incomprehensible technical terms and their negative impact on comprehension. The usability of tables and graphs was considered worst of all portals - not enough additional information was available, graphics were confusing, images and data small and disarranged. Eye tracking data showed that only few used explanatory information and that terms and numbers were not consistent when users changed details levels.
3 of 15 subjects using the “Weisse Liste†did not identify the better hospital. The portal provided more statistical information, including confidence intervals, than other portals. Eye tracking data showed that a part of this detailed information was perceived. However, 9 subjects stated that obscure technical terms had worsened their understanding. Tables were perceived to be incomprehensible, detailed explanatory information was missing, abbreviations were incomprehensible and the significance of the results remained unclear.
Only 1 of 18 subjects using the “Krankenhausspiegel†failed to identify the better hospital. Users of this portal were more satisfied with the presentation and the results of their research than other users. This portal presents important information in general terms and in simple graphs, including explanations of quality indicators and outcome results. Essential information is provided twice in different displays.
Conclusion: Comprehension by the public of hospital outcome indicators varies by the information presentation approach of public reporting portals. Poor information presentations have led to misinterpretation and dissatisfied users. However, only a limited number of portals that publish comparative health care information were evaluated. Further work is needed to obtain a more comprehensive overview.
Objective: To investigate whether the information presentation of four portals 1) influences the interpretation of data by users, 2) influences the user experience and 3) whether reasons can be identified to account for these influences.
Methods: Internet pages from four leading German public reporting portals (Krankenhausspiegel, Techniker Krankenkasse, Weisse Liste, Qualitätskliniken) containing outcome quality indicators about coronary artery bypass graft surgery or primary hip replacement were presented to 35 subjects. Subjects were then asked to determine whether one hospital had better results than another one or than the national average. Additionally, they were asked to substantiate their decision and were asked to assess the usability of the portals. 56 experiments were recorded with the use of eye tracking.
Results: The four portals named the outcome indicators in different ways, published different details of quality information and used different tables and graphical representations of data. In 13 of 56 experiments (23.2 %) the subjects did not correctly identify the better hospital. 57.1 % of subjects did not understand at least one technical term, still 37.5% more than one term and 55.4 % stated their comprehension to have been impaired by technicals term they hadn’t understood. 39.3% were not satisfied with the knowledge that they had gained when using the respective portal.
6 of 14 subjects using “Techniker Krankenkasse†did not correctly identify the hospital with better outcome data. 12 stated a negative impact on comprehension because of incomprehensible technical terms. 9 were dissatisfied with the portal. The very concise tabular presentation and the almost complete absence of explanation of technical terms might have led to discontent and misinterpretation of data.
3 of 9 subjects using “Qualitätskliniken†did not identify the better hospital. 4 reported incomprehensible technical terms and their negative impact on comprehension. The usability of tables and graphs was considered worst of all portals - not enough additional information was available, graphics were confusing, images and data small and disarranged. Eye tracking data showed that only few used explanatory information and that terms and numbers were not consistent when users changed details levels.
3 of 15 subjects using the “Weisse Liste†did not identify the better hospital. The portal provided more statistical information, including confidence intervals, than other portals. Eye tracking data showed that a part of this detailed information was perceived. However, 9 subjects stated that obscure technical terms had worsened their understanding. Tables were perceived to be incomprehensible, detailed explanatory information was missing, abbreviations were incomprehensible and the significance of the results remained unclear.
Only 1 of 18 subjects using the “Krankenhausspiegel†failed to identify the better hospital. Users of this portal were more satisfied with the presentation and the results of their research than other users. This portal presents important information in general terms and in simple graphs, including explanations of quality indicators and outcome results. Essential information is provided twice in different displays.
Conclusion: Comprehension by the public of hospital outcome indicators varies by the information presentation approach of public reporting portals. Poor information presentations have led to misinterpretation and dissatisfied users. However, only a limited number of portals that publish comparative health care information were evaluated. Further work is needed to obtain a more comprehensive overview.
Medicine 2.0® is happy to support and promote other conferences and workshops in this area. Contact us to produce, disseminate and promote your conference or workshop under this label and in this event series. In addition, we are always looking for hosts of future World Congresses. Medicine 2.0® is a registered trademark of JMIR Publications Inc., the leading academic ehealth publisher.
This work is licensed under a Creative Commons Attribution 3.0 License.