Review: evaluating information systems in nursing
Oroviogoicoechea C, Elliott B, Watson R. Review: evaluating information systems in nursing. J Clin Nurs. 2007 Dec 18;
QUESTION Are there better methods for evaluating the information systems used by nursing?
DATA SOURCES The electronic databases CINAHL and Medline were used to identify studies for review. In addition, a manual review was performed for articles published recently in Journal of American Medical Informatics Association, International Journal of Medical Information, and Computers in Nursing along with a manual search of papers in the reference lists of the systematic reviews.
METHOD Studies were selected from a search that included the terms ‘nursing information systems’, ‘clinical information systems’, ‘hospital information systems’, ‘documentation’, ‘nursing records’, and ‘charting’ combined with ‘electronic*’ and ‘computer*’. Journal articles, research papers, and systematic reviews from 1995 to 2005 were included. Excluded were anecdotes, responses, brief items and commentaries, management and legal publications, and those with fewer than 10 references. Only those articles that pertained to full electronic patient record systems or electronic nursing record systems were reviewed.
RESULTS Five areas of importance in the evaluation of Nursing Information Systems are identified. These include: 1. the requirements of the IT system, 2. the indicators of successful IT systems, 3. individualized care and structured data, 4. user satisfaction, and 5. current issues in evaluation of IT systems.
First, examined are the three requirements of IT systems cited by Oroviogoicoechea: 1) an integrated patient record that allows health professionals’ entry and access to data from different places simultaneously, 2) user involvement in all phases of implementation, and 3) recognizing the importance of organizational factors.
Second, the indicators of successful IT systems are defined as a multi-dimensional concept which encompasses system, individual, and organizational factors. Completeness and accuracy of the record, user satisfaction, and data entry have been studied. Within this context, the usefulness of the data is studied. Is the information sufficient but not overly detailed?
Next, the impact of the data captured by nurses is examined. Better data may be obtained by the use of structured formats and predefined care plans, in conjunction to the use of a formalized nursing language. However, the data may not fully convey the patient’s situation between nurses. Nurses indicated that there is more to understanding a patient than the data gathered by formal assessment.
Oroviogoicoechea then explores research on user satisfaction with the electronic record. While most agree that nurses are satisfied “with the timely and efficient retrieval of results with IT systems.” The criticism of these systems includes that the use is time consuming, not clinically relevant and there were system problems.
Finally, the following issues are raised in regard to the evaluation of IT systems. The first is a lack of quality research and measurement tools. The second is that many studies are not addressing the relationship between the system, the context, and both the users and the organization.
DISCUSSION Oroviogoicoechea points out that the current research into Nursing Information System implementations is incomplete. Two methods of studying this topic are presented. They are the socio-technical approach and the theory-driven approach. In the former more emphasis is placed on evaluation of the social and communication oriented nature of healthcare information systems. The latter uses a realist evaluation and includes a context-mechanisms-outcomes model to generate a theory that is then tested.
CONCLUSION Oroviogoicoechea summarizes that a “complete evaluation of the implementation of IT systems is not feasible.” She notes that study of NIS systems that have already been implemented need re-evaluation to determine “not just whether they work, but how and in what circumstances they work.” Realistic evaluation is purported to be a means to better understand “…why and how a programme or intervention works…” Realistic evaluation is a research tool that is increasing in application to the study of the implementation of Nursing Information Systems.
Michael Lees BMI 512 Winter 2008