Quality and Safety in Health Care Journal

Evaluation is an important component of service improvement but is not undertaken often enough. Many evaluation guides are available, and the language describing approaches is complex and varied. This variation is a potential barrier to evaluation, particularly in communication between practitioners. A structured literature review of development and evaluation frameworks for healthcare quality improvement was carried out. Four frameworks with diverse methodological perspectives were identified. Partial mapping was possible against an existing structure for health promotion research. When the questions in this framework were adapted for healthcare evaluation, it provided a common structure for the four frameworks. There is significant common ground underpinning these terminological differences between descriptions of evaluation techniques. The common language developed here has potential to ease communication in healthcare evaluation and improvements in quality and safety. It is an aid for practitioners in framing pragmatic evaluation of improvement projects.

While substantial public and scholarly attention has focused on the ethical review and oversight of quality improvement (QI) initiatives, there are no systematic data regarding the institutional mechanisms currently in place to review the conduct of QI and the ethical considerations guiding this work.

The authors recruited quality improvement practitioners (QIP) affiliated with the Institute for Healthcare Improvement (IHI) ‘100 000 Lives’ Campaign to participate in a web-based survey. Standard statistical methods were used to analyse the data.

Surveys were completed by 132 QIPs (response rate=26%). Most respondents indicated that QI initiatives conducted with their organisation are subject to review prior to implementation. Respondents strongly agreed that ensuring minimal risk to patients, and privacy and confidentiality are relevant ethical considerations for QI initiatives conducted at their institution. A majority of respondents also agreed that assessing established practices, scientifically sound design, transparency, and the identification and minimisation of potential conflicts are relevant ethical considerations for QI initiatives.

Many QI efforts seem to be getting some oversight, and those engaged in the QI initiatives endorse a range of ethical considerations that are suited to this work. Yet most of these reviews are not independent of those conducting the intervention and are not conducted by people trained in ethics. These findings should facilitate the development of a conceptual and policy framework that is informed by the realities of QI.

The use of national quality campaigns to foster evidence-based hospital practices is increasing. Because campaigns typically do not limit access to their resources, they may influence non-enrolled hospitals as well.

To examine the relative impact of a national campaign, the Door-to-Balloon (D2B) Alliance, on enrolled and non-enrolled hospitals.

In this prospective cohort study, we compared the use of D2B Alliance resources (eg, webinars, online community, mentor network), changes in the use of strategies recommended by the D2B Alliance, and perceived impact of the D2B Alliance between hospitals that enrolled in the D2B Alliance (n=264) and hospitals that declined enrolment (n=101).

More than half (53.2%) of non-enrolled hospitals reported using at least some of the resources made available by the D2B Alliance to improve door-to-balloon times. This compared with 83.5% of enrolled hospitals reporting that they used D2B Alliance resources (p<0.01). Both enrolled and non-enrolled hospitals significantly increased their use of recommended hospital strategies between 2005 and 2008, although the use of strategies remained incomplete (35.5–91.5% use). There was no significant difference between the use of these strategies between enrolled and non-enrolled hospitals at follow-up (p≥0.51), adjusted for baseline use. About half of all hospitals reported that door-to-balloon times would have been worse at their hospital without the existence of the D2B Alliance.

This research suggests that national quality campaigns with open access to campaign resources may have substantial spillover effects on non-enrolled hospitals.

The diversity of quality improvement interventions (QIIs) has impeded the use of evidence review to advance quality improvement activities. An agreed-upon framework for identifying QII articles would facilitate evidence review and consensus around best practices.

To adapt and test evidence review methods for identifying empirical QII evaluations that would be suitable for assessing QII effectiveness, impact or success.

Literature search with measurement of multilevel inter-rater agreement and review of disagreement.

Ten journals (2005-2007) were searched electronically and the output was screened based on title and abstract. Three pairs of reviewers then independently rated 22 articles, randomly selected from the screened list. Kappa statistics and percentage agreement were assessed. 12 stakeholders in quality improvement, including QII experts and journal editors, rated and discussed publications about which reviewers disagreed.

The level of agreement among reviewers for identifying empirical evaluations of QII development, implementation or results was 73% (with a paradoxically low kappa of 0.041). Discussion by raters and stakeholders regarding how to improve agreement focused on three controversial article selection issues: no data on patient health, provider behaviour or process of care outcomes; no evidence for adaptation of an intervention to a local context; and a design using only observational methods, as correlational analyses, with no comparison group.

The level of reviewer agreement was only moderate. Reliable identification of relevant articles is an initial step in assessing published evidence. Advancement in quality improvement will depend on the theory- and consensus-based development and testing of a generalizable framework for identifying QII evaluations.

Interruptions and multitasking are implicated as a major cause of clinical inefficiency and error.

The aim was to measure the association between emergency doctors' rates of interruption and task completion times and rates.

The authors conducted a prospective observational time and motion study in the emergency department of a 400-bed teaching hospital. Forty doctors (91% of medical staff) were observed for 210.45 h on weekdays. The authors calculated the time on task (TOT); the relationship between TOT and interruptions; and the proportion of time in work task categories. Length-biased sampling was controlled for.

Doctors were interrupted 6.6 times/h. 11% of all tasks were interrupted, 3.3% more than once. Doctors multitasked for 12.8% of time. The mean TOT was 1:26 min. Interruptions were associated with a significant increase in TOT. However, when length-biased sampling was accounted for, interrupted tasks were unexpectedly completed in a shorter time than uninterrupted tasks. Doctors failed to return to 18.5% (95% CI 15.9% to 21.1%) of interrupted tasks.

It appears that in busy interrupt-driven clinical environments, clinicians reduce the time they spend on clinical tasks if they experience interruptions, and may delay or fail to return to a significant portion of interrupted tasks. Task shortening may occur because interrupted tasks are truncated to ‘catch up’ for lost time, which may have significant implications for patient safety.

Incomplete medication lists increase the risk of medication errors and adverse drug effects. In Denmark, dispensing data and pharmacy records are available directly online to treating physicians. We aimed (1) to describe if use of pharmacy records improved the medication history among patients consulting their general practitioner and (2) to characterise inconsistencies between the medication history reported by the patient and the general practitioner's recordings.

Patients attending a general practitioner clinic were interviewed about their current medication use. Subsequently, the patients were contacted by phone and asked to verify the medication list previously obtained. Half of the patients were randomly selected for further questioning guided by their dispensing data: during the telephone interview, these patients were asked to clarify whether drugs registered in their pharmacy records were still in use. Pharmacy records show all drugs acquired on prescription from any national pharmacy in the preceding 2 years. The medication list was corrected accordingly. In all patients, the medication lists obtained on the in-clinic and telephone interviews were compared to the general practitioner's registrations.

The 150 patients included in the study had a median age of 56 years (range 18–93 years), and 90 (60%) were women. Patients reported use of 849 drugs (median 5, range 0–16) at the in-clinic interview. Another 41 drugs (median 0, range 0–4) were added during the telephone interview. In the subgroup of 75 patients interviewed guided by pharmacy records, additionally 53 drugs (10%) were added to the 474 drugs already mentioned. The 27 patients adding more drugs guided by pharmacy records were significantly older and used more drugs (both p<0.05) than the 48 patients not adding drugs. When the medication lists were compared with the general practitioner's lists, specifically use of over-the-counter products and prescription-only medications from Anatomical Therapeutic Chemical Classification System group J, A, D, N and R were not registered by the general practitioner.

Dispensing data provide further improvement to a medication history based on thorough in-clinic and telephone interviews. Use of pharmacy records as a supplement when recording a medication history seems beneficial, especially among older patients treated with polypharmacy.

Collaboration and communication as dimensions of patient safety climate have been measured in acute care hospital units, and discrepant viewpoints have been documented between different professional groups, particularly between physicians and nurses. In the ambulatory care setting, these groups often work more closely together throughout the day than in acute care settings, thereby enhancing effective collaboration and communication. This study sought to determine if the communication differences that are known to impact patient safety, which are found in acute care, also exist in ambulatory care.

The Safety Attitudes Questionnaire, a 77-item survey of collaboration, communication and safety attitudes, was administered to the primary care staff at four Midwestern military ambulatory care clinics.

There were 107 participants consisting of nurses (n=46), nurse practitioners (n=12), pharmacists (n=10) and physicians (n=39), yielding an overall response rate of 65%. All groups rated their peer group higher than other professional groups. The ratings of nurses and physicians were very similar: 85.0% of nurses rated physicians, and 85.7% of physicians rated nurses as high or very high in communication and collaboration. Pharmacists were rated the lowest by each of the other professional groups. Only 60% of pharmacists rated physicians as high or very high.

Collaboration and communication ratings among physicians and nurses appear to be higher in the ambulatory care setting than in the acute care. However, interactions with pharmacists are more problematic, perceived as adversarial. Teamwork training that focuses on specific interactions among professional groups should target these concerns.

To measure self-reported inpatient experience in Hong Kong.

Data were derived from the 2005 Thematic Household Survey.

24 364 non-institutional and 3390 institutionalised respondents aged at least 18 years systematically drawn to represent the Hong Kong adult population, 6.9% of whom were admitted at least once as an inpatient during the previous 12 months. Data from this group was analysed.

Picker Patient Experience Questionnaire-15.

Overall, respondents scored their last inpatient episode 39.6 (range=0–100, the lower the score, the better the patient experience). Patients who sought care from private hospitals reported a lower Picker Patient Experience Questionnaire-15 score than those cared for in public facilities (31.1 vs 41.8 respectively, p<0.001). We observed substantial differences between public hospital geographic clusters that were confirmed by multivariable regression. When benchmarked against the UK, Germany and the USA, Hong Kong patients tended to report a significantly higher number of problems.

We found systematic differences between the level of satisfaction and type of problems reported by Hong Kong Chinese compared to those in Euro-American settings. The observed heterogeneities among different public hospitals, between the private and public sectors, and among subgroups of inpatients should provide an evidence based on which quality improvement initiatives can be designed and evaluated.

To explore interpretations of "appropriate" and "inappropriate" elective referral from primary to secondary surgical care among senior clinical and non-clinical managers in five purposively sampled primary care trusts (PCTs) and their main associated acute hospitals in the English National Health Service (NHS).

Semi-structured face-to-face interviews were undertaken with senior managerial staff from clinical and non-clinical backgrounds. Interviews were tape-recorded, transcribed and analysed according to the Framework approach developed at the National Centre for Social Research using N6 (NUD*IST6) qualitative data analysis software.

Twenty-two people of 23 approached were interviewed (between three and five respondents per PCT and associated acute hospital). Three attributes relating to appropriateness of referral were identified: necessity: whether a patient with given characteristics was believed suitable for referral; destination or level: where or to whom a patient should be referred; and quality (or process): how a referral was carried out, including (eg, investigations undertaken before referral, information contained in the referral and extent of patient involvement in the referral decision. Attributes were hierarchical. "Necessity" was viewed as the most fundamental attribute, followed by "destination" and, finally, "quality". In general, but not always, all three attributes were perceived as necessary for a referral to be defined as appropriate.

For senior clinical and non-clinical managers at the local level in the English NHS,, three hierarchical attributes (necessity, appropriateness of destination and quality of referral process) contributed to the overall concept of appropriateness of referral from primary to secondary surgical care.

To assess effectiveness of guidelines for referral for elective surgical assessment.

Systematic review with descriptive synthesis.

Medline, EMBASE, CINAHL and Cochrane database up to 2008. Hand searches of journals and websites.

Studies evaluated guidelines for referral from primary to secondary care, for elective surgical assessment for adults.

Appropriateness of referral (usually measured as guideline compliance) including clinical appropriateness, appropriateness of destination and of pre-referral management (eg, diagnostic investigations), general practitioner knowledge of referral appropriateness, referral rates, health outcomes and costs.

24 eligible studies (5 randomised control trials, 6 cohort, 13 case series) included guidelines from UK, Europe, Canada and the USA for referral for musculoskeletal, urological, ENT, gynaecology, general surgical and ophthalmological conditions. Interventions varied from complex ("one-stop shops") to simple guidelines. Four randomized control trials reported increases in appropriateness of pre-referral care (diagnostic investigations and treatment). No evidence was found for effects on practitioner knowledge. Mixed evidence was reported on rates of referral and costs (rates and costs increased, decreased or stayed the same). Two studies reported on health outcomes finding no change.

Guidelines for elective surgical referral can improve appropriateness of care by improving pre-referral investigation and treatment, but there is no strong evidence in favour of other beneficial effects.

Interdisciplinary communication is critically important to provide safe and effective care, yet it has been inadequately studied for hospitalised medical patients. Our objective was to characterise nurse–physician communication and their agreement on patients' plan of care.

During a one-month period, randomly selected hospitalised patients, their nurses and their physicians were interviewed. Nurses and physicians were asked to identify one another, whether communication had occurred, and about six aspects of the plan of care. Two internists rated nurse–physician agreement on aspects of the plan of care as none, partial or complete agreement. Measures included the percentage of nurses and physicians able to identify one another and reporting communication and the percentage of nurse–physician pairs in agreement on aspects of the plan of care.

310 (91%) and 301 (88%) of 342 eligible nurses and physicians completed interviews. Nurses correctly identified patients' physicians 71% of the time and reported communicating with them 50% of the time. Physicians correctly identified the patients' nurses 36% of the time and reported communicating with them 62% of the time. Physicians and nurses showed no agreement on aspects of the plan of care ranging from 11% for planned procedures to 42% for medication changes.

Nurses and physicians did not reliably communicate with one another and were often not in agreement on the plan of care for hospitalised medical patients.

Prolonged emergency department boarding times (EDBT) are associated with adverse clinical outcomes and longer hospital stays. A rapid admission protocol was designed at our institution to reduce both EDBT and time to admission orders (EDTAO) for patients admitted to the internal medicine service.

The existing admission process was examined by a team of clinical and administrative leaders who focused on developing a change management architecture, narrowing clinical roles, mandating direct communication, establishing clear boundaries for patient responsibility and instituting carefully constructed holding orders. The number of steps in the admission process was reduced from 50 to 10. We collected EDBT and EDTAO for all patients admitted to the internal medicine service before and after intervention using a simple interrupted time-series design.

The study involved a total of 9604 admissions to one of three inpatient destinations (general medicine ward, telemetry or intensive care unit). The overall EDBT decreased from 360 min in the preintervention period to 270 min in phase 4 (p<0.001). The overall time to admission orders decreased from 210 min in the preintervention period to 75 min in phase 4 (p<0.001) overall. However, no improvements were noted in EDBT for telemetry or ICU patients.

Institution of a rapid admission protocol successfully reduced overall EDBT at our institution, although few gains were noted for patients with a telemetry or ICU destination. In total, the intervention saved 27 884 h, or 1161 emergency department patient-days, over the course of a single year.

It is well recognised in healthcare settings that clinical staff have a major influence over change in how services are provided. If a culture of systematic service improvement is to be established, it is essential that clinical staff have an understanding of what is required and their role in its application.

This paper describes the development of short educational interventions (a module of 6–8 contact hours or a longer module of 18–30 h) for inclusion in the initial training of future clinical staff (nursing, medicine, physiotherapy, occupational therapy, dietetics, social work, operating department practice, public health and clinical psychology) and presents the results of an evaluation of their introduction. Each module included teaching on process/systems thinking, initiating and sustaining change, personal and organisational development, and public and patient involvement.

Over 90% of students considered the modules relevant to their career. Nearly 90% of students felt that they could put their learning into practice, although the actual rate of implementation of changes during the pilot period was much lower. The barriers to implementation most commonly cited were blocks presented by existing staff, lack of time and lack of status of students within the workforce.

This pilot demonstrates that short educational interventions focused on service improvement are valued by students and that those completing them feel ready to contribute. Nevertheless, the rate of translation into practice is low. While this may reflect the status of students in the health service, further research is needed to understand how this might be enhanced.

Arterial catheterisation is used for continuous haemodynamic monitoring in patients undergoing surgery and in critical care units. Although it is considered a safe procedure, a major complication such as arterial occlusion and limb gangrene can occur.

To determine the incidence, outcome and potential to avoid complications associated with arterial catheterisation.

The number of arterial catheterisation was determined using an anaesthesiology and critical care medicine billing database over a period of 4 years (1 January 2003 to 31 December 2006). Possible major complications were identified from two hospital databases; all identified charts were screened and then reviewed by an expert panel that determined causation. A major complication was defined as requiring operative intervention and/or resulting in permanent harm.

15 (0.084%) major complications were identified among 17 840 instances of arterial catheterisation insertions. Of 15 arterial catheterisations, nine were performed in the operating room and six in the intensive care unit. Nine patients suffered ischaemic injury, which progressed to gangrene in three patients. Three patients developed haematoma that required surgical evacuation; two of these required vascular repair. One patient had compartment syndrome requiring fasciotomy and two patients had sheared catheter fragments that needed to be removed. All 15 patients had multiple comorbidities, and those in the operating room had an American Society of Anesthesiologists score of ≥3. Seven (46.6%) had arterial catheterisation done under emergent circumstances. Six (40%) died during hospitalisation because of complications unrelated to arterial catheterisation.

Arterial catheterisation had a very low rate of major complications. They seem associated with high severity of illness and emergency surgery.

Several studies have identified higher mortality for patients admitted as emergencies at the weekend compared with emergency admissions during the week, but most have focused on specific conditions or have had a limited sample size.

Using routinely collected hospital administrative data, we examined in-hospital deaths for all emergency inpatient admissions to all public acute hospitals in England for 2005/2006. Odds of death were calculated for admissions at the weekend compared to admissions during the week, adjusted for age, sex, socioeconomic deprivation, comorbidity and diagnosis.

Of a total of 4 317 866 emergency admissions, we found 215 054 in-hospital deaths with an overall crude mortality rate of 5.0% (5.2% for all weekend admissions and 4.9% for all weekday admissions). The overall adjusted odds of death for all emergency admissions was 10% higher (OR 1.10, 95% CI 1.08 to 1.11) in those patients admitted at the weekend compared with patients admitted during a weekday (p<0.001).

This is the largest study published on weekend mortality and highlights an area of concern in relation to the delivery of acute services.

Legislation enacted in the US State of North Carolina in 2003 requires all licenced nursing homes to report all medication errors. In 2007, nursing homes were encouraged to voluntarily convert from aggregate reporting to a new online system where they reported each individual error.

A new optional web-based reporting tool was made available to all 393 North Carolina nursing homes to submit error reports for each distinct medication error as they occurred during the year.

A total of 5823 medication error reports were submitted by 203 sites (52%) using the new system during the reporting year, a median of 18 error reports per site. Of the 5823 error reports, 612 (10.5%) were categorised as serious. Serious errors were more likely to be caused by drugs given to the wrong patient (RR 4.39, CI 3.7 to 5.2), lab-work error (RR 2.40, CI 1.4 to 4.0), wrong product given (RR 2.22, CI 1.8 to 2.8) and medication overdoses (RR 1.49, 1.2 to 1.8). Serious errors were more likely to occur on second shift (RR 1.32, 1.1 to 1.5). Common medications that are involved in the most serious errors include warfarin (RR 2.58, CI 2.09 to 3.18) and insulin (RR 2.35, CI 1.86 to 2.97), and oxycodone combinations (RR 1.48, CI 1.07 to 2.06).

Data collected from a nursing home medication error system can provide helpful information on serious errors that can be used to focus patient safety efforts to reduce harm. This improved information will be useful in nursing homes for continuous quality improvement efforts.

Little is known about adverse events (AEs) that occur between physician visits for ambulatory chronic disease patients. An automated telephone self-management support programme for a diverse population of diabetes patients was implemented to capture AEs, describe the self-management domains from which they emanate and explore contributing causes.

AEs and potential AEs (PotAEs) were identified among 111 ethnically diverse diabetes patients. An AE is an injury that results from either medical management or patient self-management; a PotAE is an unsafe state likely to lead to an event if it persists without intervention. Medical record reviews were conducted to ascertain which self-management domain was involved with the event and to explore contributing causes.

Among the 111 patients, 86% had at least one event detected over the 9-month observation period. 111 AEs and 153 PotAEs were identified. For all events, medication management was the most common domain (166 events, 63%). Only 20% of events reflected a single contributing cause; in the remaining 80%, a combination of system, clinician and patient factors contributed to their occurrence. Patient actions were implicated in 205 (77%) events, systems issues in 183 (69%) events and inadequate physician–patient communication in 155 (59%) events. Aside from communication, primary care clinician actions contributed to the occurrence of the event in only 16 cases (6%).

Our findings reveal a complex safety ecology, with multiple contributing causes for AEs and PotAEs among ambulatory diabetes patients. Moreover, patients themselves seem to be key drivers of safety and of AEs, suggesting that patient-level self-management support and patient-centred communication are critical to AE prevention.

Following the introduction of an electronic Incident Information Management System (IIMS) in New South Wales, Australia, the authors investigated enablers and barriers to the use of IIMS and factors associated with increased, static and decreased reporting rates.

An online and paper-based, anonymous survey of 2185 health practitioners collected information about their reporting behaviour and experiences of enablers/barriers: training, system accessibility, ease of use, system security, feedback, perceived value of IIMS and workplace safety culture.

The 79.3% of respondents who reported on IIMS were distinguished from non-reporters by having undertaken IIMS training and evaluating this highly. Users reporting more incidents post-IIMS were more likely than those with static or decreased reporting rates to evaluate their training highly and to have experienced all enablers. Users reporting fewer incidents were least likely to do so. The relative likelihood of the three reporting groups experiencing various enablers was similar. Those most frequently experienced by all groups were system security and accessibility. Barriers most frequently encountered were more culturally embedded—for example, poor workplace safety culture. The ‘more’ reporting group actually reported most, and the ‘static’ group least, incidents.

The sample was large but not randomly selected, which limits the generalisability of findings.

Interventions to increase reporting should target provision of training that endorses and fosters conditions shown to enhance reporting rates.

Enablers to incident reporting have been shown to be associated not only with reporting per se but also with changes to reporting patterns and rates.