Evaluating computerised health information systems: hard lessons still to be learnt

Enormous investment has gone into computerised hospital information systems worldwide. The estimated costs for each large hospital are about $50m (£33m), yet the overall benefits and costs of hospital information systems have rarely been assessed. When systems are evaluated, about three quarters are considered to have failed, and there is no evidence that they improve the productivity of health professionals. To generate information that is useful to decision makers, evaluations of hospital information systems need to be multidimensional, covering many aspects beyond technical functionality. A major new information and communication technology initiative in South Africa gave us the opportunity to evaluate the introduction of computerisation into a new environment. We describe how the project and its evaluation were set up and examine where the project went wrong. The lessons learnt are applicable to the installation of all hospital information systems.