Design and development of a monitoring system
Ehlers F, Ammenwerth E, Hirsch B. Design and development of a monitoring system to assess the quality of hospital information systems: concept and structure. Stud Health Technol Inform. 2005;116:575-80.
The aim of the research project on which this paper is based is to develop and validate a comprehensive system to monitor HISs. Ehlers et al maintain that while HIS planning and institution are generally well supported, the systematic monitoring of HIS quality as well as the available tools and methods to do so are not. This monitoring system allows context-dependent assessment by the clinical and administrative staff. These are the stakeholders who the authors feel are the real experts of HIS quality. Their monitoring tool uses a matrix that applies accepted HIS quality criteria obtained from an extensive literature review, such as availability of information, completeness, readability, usability, and processing time, to each of the major process steps that they have identified as important in their institution, e.g., admission, decision making, order entry and execution and discharge. The quality criteria are on the X-axis and the process steps are on the Y-axis. Because it polls the end usability of the HIS and not the specific processes of the HIS itself (e.g., hardware, software, architecture) the monitoring system is meant to be a kind of screening or benchmarking instrument and is not meant to show in detail the reasons for good or bad information processing. The authors feel that what sets their study apart from others is their contextual descriptions of specific process steps. The authors admit that, “…it is sometimes argued that only objective, quantitative measurements create new knowledge,” however they feel that the measurement of such numbers may not be helpful because: 1) Their significance may be context dependent. 2) They may be too time-intensive to do. 3) There may not be an objective reality: “The ‘truth’ is constructed by people and does not exist in itself…a ‘good’ information system perceived by its users as a ‘poor’ system, is a poor system.”
In my opinion this is a useful tool to survey a system using the subjective impressions of multiple stakeholders which, in turn, is based on multiple objective, though not specifically measured, HIS interactions.