To help stem ever increasing health care costs, clinicians, administrators, payors, and families must better communicate about the value of care options. Our institution sought to create a value analysis framework, based on local implementation of over 50 clinical pathways.
From January to March 2015, a multidisciplinary team of clinicians, nursing, pharmacy, informatics, and analytics developed a cost dashboard, and a value analysis and decision-making framework. We reviewed all clinical pathway recommendations and measures, and examined issues related to implementing pathways. We examined specific line item costs for each pathway patient population.
Our value analysis and decision-making framework is based upon 3 population-level comparisons: operational factors/implementability, benefits, and costs/complications. We used the framework to scope new projects, to better target clinical recommendations, and to evaluate pathways postimplementation. From April through December 2015, we used this framework to prospectively analyze 19 pathway improvement proposals. The tool helped us to identify 16 improvements to implement (15 predicted to save costs), as well as identifying challenges facing 3 additional improvements.
The value analysis and decision-making framework helps to ensure clinical recommendations are evaluated more broadly than just from a cost perspective. Systematic evaluation using this framework can improve population-based decision-making. More research is needed in how to ensure that value analysis is performed to guide sustainable implementation of recommendations locally, and how to identify other institutions that may benefit from replicating successful implementations, so that the highest value care can be provided to patients everywhere.
In 2014, US health care spending increased 5.3% to reach $3 trillion.1 To try and stem this trend, the Affordable Care Act set forth structural and philosophical changes that strive to refocus the system on improving population health, improving patient experience, while reducing health care costs.2,3 As a result, health care organizations are moving beyond traditional frameworks for cost reduction to better understand the relationship between utilization and outcomes. Yet, while clinicians have significant potential to drive changes in treatment patterns, little has been developed to help clinicians and their professional organizations to take a standardized value-based approach when making clinical recommendations for care.4
Value frameworks are challenged by the difficulty in appropriately measuring both costs and patient health impact.3 One study revealed that only 28% of treatment comparison decisions were undisputedly more effective and less expensive. In the remaining 72% of cases, the more clinically efficacious option was not the least expensive.5 This suggests that in the majority of treatment decisions, the relationship between effectiveness and cost is highly nuanced. Porter et al6 challenge health care organizations to delve further into an understanding of value that considers both short- and long-term measures of health and recovery, as well as the process of recovery (including disutility of care), centering cost measures around patients and the compendium of care.6 For clinicians to effectively incorporate value into clinical decision-making, they must be able to balance costs, outcomes, and individual patient attributes in a unified way.7
Development of Framework
The Clinical Effectiveness (CE) team at our institution creates evidence-based clinical pathways, hardwires recommendations into the clinical information system (Cerner Millennium Kansas City, MO), and monitors clinical outcomes to facilitate iterative clinical improvement. Over 50 clinical pathways have been created for specific pediatric diagnoses. Core measures are tracked for each clinical pathway (total discharges, length of stay, readmissions, charges, and order set usage), as well as pathway-specific formative, summative, and balancing measures.
In January 2015, our team was challenged to determine ways to improve value of care. We examined utilization and costs for the pathways. Faculty, trainees, and staff showed a knowledge deficit related to costs, and thus to value tradeoffs when applying clinical recommendations to patients. This was further impeded by a lack of common terminology to discuss these topics. A literature review was conducted analyzing the effects of clinical recommendations on medical care, including cost-effectiveness, cost-benefit, cost-minimization, and time-derived activity based cost models. These models seemed too complex to be used by clinicians to inform real-time clinical decision-making and comparison of clinical recommendations.
During weekly meetings over a 3-month period, all pathway clinical recommendations, core measures, and pathway measures were reviewed to classify them into general groupings related to differences between previous care and current/future care. We also examined the issues that we had encountered over the course of the program related to success or failure when implementing clinical recommendations.
In addition to clinical care recommendations and process, outcome, and balancing measures, we examined line item costs. We obtained costs from our internal cost accounting system, Allscripts EPSi (Chicago, IL) Cost allocations from EPSi are based on industry standard cost to charge ratio-based allocations with adjustments made at the charge line item level to account for local variation in costs such as supplies and labor. Organizational costs are allocated across all departments at the charge line item level and with allocations for direct versus indirect, fixed versus variable and supply, labor and other specific costs. These costs were normalized to 2014 dollars by using the medical care component of the Consumer Price Index. Although the cost allocations are still a proxy for actual costs, they provide a scalable way to compare costs over time and across all services provided for all patients. The cost dashboard contains all billable items from the hospital stay, but does not include inpatient physician fees.
An interface was created so that EPSi data populates our Netezza Enterprise Data Warehouse (IBM Corporation; Armonk, NY), where it is joined with administrative data from Epic Hyperspace (Verona, WI) and clinical data from Cerner Millennium. The Enterprise Data Warehouse integrates disparate data sets into a large data source, which is queried to answer specific analytic questions. We created a Cost Dashboard that allows us to examine specific line item costs for a panel of patients, and used population definitions already created for our pathways to allow us to examine actual charges by patient population. Costs have been grouped into major buckets to determine key drivers of cost over time. Searchable charge item level costs are displayed to enable analysis for custom cost groupings. Examples of these reports are illustrated in Supplemental Figs 1 to 3.
Value Analysis and Decision-making Framework
Building off the concept of an accounting ledger, a value analysis framework was conceptualized that consolidated benefits, harms, and associated costs for each clinical alternative into a unified visual format that would allow clinicians to easily analyze value prospectively (Tables 1 and 2).8 We discovered 3 fundamental themes for comparison of clinical care at a population level: operational factors/implementability, benefits, and costs/complications.
When practicing medicine, clinical recommendations are often applied in the face of diagnostic uncertainty. For example, the American Academy of Pediatrics Bronchiolitis guideline9 recommends, “When clinicians diagnose bronchiolitis on the basis of history and physical examination, radiographic or laboratory studies should not be obtained routinely.” Local adherence to this recommendation reduced chest radiograph ordering from 35% to 20%. However, further reductions proved challenging because radiographs were ordered prospectively in the face of undifferentiated respiratory illness in the emergency department. Measured postimplementation adherence to clinical recommendations is affected greatly by what providers see prospectively, and expected adherence should reflect these realities of practice.
Additional challenges are illustrated when recommending suprapubic aspiration (SPA) for diagnosing urinary tract infections (UTIs). Per the 1999 American Academy of Pediatrics UTI guideline,10 catheterization has a sensitivity of 95% and a specificity of 99% compared with SPA, with catheterization also more likely to lead to contamination.11–14 Although SPA would thus appear to be the gold standard, 2 additional operational factors were discovered when considering SPA to diagnose UTI. Urine catheterization is performed by nursing, whereas SPA is performed by physicians. Many physicians had never performed SPA, and optimal circumstances for performing SPA include ultrasound-guidance with an experienced provider. Also, families dislike SPA and prefer urinary catheterization. Other pathways corroborated that both care team capabilities and family acceptability influence the implementability of clinical recommendations.
Benefits: Differences Between Care Options
When analyzing care options, Porter’s model6 applied in the setting of a pediatric admission revealed that a clinical care recommendation would likely affect (1) functional status at time of discharge, or (2) clinical status of the patient during the hospitalization (as patients viewed to be clinically similar at time of discharge could have had vastly different hospital courses). We incorporated these into our framework.
The process of weighing benefits should also consider benefits to the population that may accrue beyond those benefits of treating the individual patient. Examples of this include immunizations and antibiotic stewardship, which may result in herd immunity and decreased antimicrobial resistance, respectively.
Costs of Routine Care and of Harms/Complications
The cost dashboard suggests that for management of patients on clinical pathways on the medical unit, ∼70% of charges relate to the cost of the room. Some average daily room/care setting costs at our institution can be found in Supplemental Table 6. Based on these room/care setting costs, if equivalent care/patient outcomes are reached, increased value will result from (1) decreasing time in a care setting (eg, discharging patients earlier when medically stable) and/or (2) transitioning care to a lower acuity care setting (eg, appropriate transfer of a patient with diabetic ketoacidosis when stable from the ICU to the medical unit, even if total length of hospitalization is the same).
In addition to the room cost, there are costs related to diagnosis and treatment of patients. These include the cost of laboratory tests, medications, and radiologic studies. Often, these costs are the ones examined when looking at clinical recommendations. Sometimes, waiting to obtain tests and laboratories may also increase the time in the hospital; we suggest that the extra time inpatient would be counted under room/care setting costs, and the costs for the specific procedure/test be captured in diagnosis/treatment costs.
When planning care, there is a third cost that is often not accounted for up front. These costs relate to potential harms. If the potential harm is felt to be very likely, a care option might not be analyzed at all. Otherwise, this term refers to costs related to patient lack of response to therapy or development of complications due to (1) lack of treatment efficacy, (2) patient-specific factors, and/or (3) complications due to diagnostic and treatment delays resulting from delayed or improper therapy. As part of this model, an attempt is made to estimate the most likely complications, to try to quantify additional bed days/costs, as well as additional diagnostic/therapeutic interventions that would result given predictable treatment failure. Additional rows may be added if there are multiple potential complications.
How to Complete the Value Analysis and Decision-making Framework
Estimate the number of patients/year where the clinical decision is to be made. Option A is the reference care (usually current state, or previous state for postimplementation analyses). Option B is care after the new clinical recommendation. When filling in the summary, fill in anticipated effects and costs for a single patient. Use the cost dashboard to identify line items for current and recommended care, and provide the expected effects of the treatment on 1 individual patient, and costs given that care alternative for a single patient. In the “Preferred Option” column, determine which option is most preferable if measured in isolation. An “Assumptions Made” column is part of the tool to help promote transparency in the decision-making process related to specific care options. It provides a historical record to understand how certain costs and literature were used to inform decisions and also serves as a place to revisit the particular value for a particular recommendation should any of these assumptions change in the future.
The decision-making tool is used to determine the ideal course of action given the implementability, clinical implications, and costs of the options. Recommendations that are likely to meet the Triple Aim (implementable/good patient experience, improved population clinical care, and lower cost) will be recommended. Where there are tradeoffs between implementability/patient experience, population clinical care, and cost, the clinical pathway team will need to make an evidence/consensus informed decision about how best to proceed, with a time frame for re-evaluation, and a Final Value Statement is written that can be used for educating our faculty, staff, and trainees.
Examples using this framework for clinical pathway development, implementation, and improvement follow.
Example 1: Deep Vein Thrombosis Prophylaxis: Project Scoping
The gastroenterology and rheumatology divisions were interested in determining whether the evidence is strong enough to implement deep vein thrombosis (DVT) prophylaxis in selected populations. A value analysis tool was completed (Table 3 and Supplemental Table 7) to inform the decision of whether to proceed with the project. Our conclusion (Final Value Statement):
Pending verification of acceptability from families, venous thromboembolism prophylaxis in this clinical scenario appears to be both clinically and cost-effective. Several moderate quality randomized controlled trials studying prophylaxis revealed no increase in major bleeding although it should be noted that these studies may have been underpowered. This value recommendation was based primarily on a cost-effectiveness model.
The value tool suggested that prophylaxis would reveal both clinical benefit and reduced cost, although implementation of this recommendation would involve education of families and the care delivery team to ensure successful implementation.
Example 2: Intravenous Versus Nasogastric Rehydration: Evidence-based Pathway Update
For acute gastroenteritis (AGE), the evidence suggests similar efficacy between intravenous (IV) and nasogastric (NG) rehydration with Pedialyte (Abbott Laboratories, Abbott Park, IL). The current organizational standard is IV rehydration; therefore, a value analysis was performed on IV versus NG rehydration (Table 4 and Supplemental Tables 8 and 9). Our conclusion:
The clinical effectiveness of IV and NG rehydration is comparable; however, IV rehydration is more costly than NG rehydration. Given that the current standard of care is IV rehydration, and this is currently the family preference in the Seattle area, families should be engaged in shared decision-making about the choice between IV and NG rehydration. At such time that NG rehydration is deemed acceptable by most parents, it would be the rehydration method of choice. This value recommendation was based primarily on a cost-effectiveness model.
From the analysis, it is clear that operational factors and current standards favor IV rehydration. Although it is predicted that family and patient palatability also would favor IV rehydration, there is some debate about whether this might indeed be the case if NG rehydration would eventually be considered standard care (instead of IV rehydration). There are no differences predicted in any of the other categories of benefits, and it is clear that NG rehydration is the more cost-effective method of rehydration.
Once this analysis was presented to the pathway team and key stakeholders, the opinions about whether this recommendation and rehydration method should be considered drastically changed. Given the anticipated barriers to NG rehydration, the team agreed to consider offering shared decision-making for this option, planning for 20% implementation in this first plan, do, check, act (PDCA) cycle of work.
Example 3: Management of Neonatal Jaundice: Postimplementation Analysis
When the clinical pathway for neonatal jaundice was implemented, the evidence strongly supported measuring and adjusting irradiance to ensure effective phototherapy. The team was able to convince hospital administration to invest $40 515 in new equipment allowing for the measurement and adjustment of irradiance for phototherapy. The clinical pathway team also made a consensus based recommendation to try and reduce use of IV fluids, and to encourage infants to feed so that they could stool, thereby eliminating bilirubin. Standard inpatient and NICU admission criteria was defined.
IV fluid use was reduced from 81% to 43%, and length of stay was reduced from 1.43 to 0.87 days. At the same time, admission rate decreased from 66% to 51%. Balancing measures such as readmission rate stayed constant. The value analysis framework was applied to estimate the cost savings resulting from these clinical changes (Table 5 and Supplemental Table 10). Our conclusion:
Adopting the evidence-based pathway guidelines appears to have reduced cost by approximately $121,680 per year due to avoided admissions, reduced length of stay, and reduced lab and fluid usage (see cost table). Achieving these gains required an up front investment of $40,515 for new equipment related to measurement and adjustment of phototherapy. This analysis is derived from internal cost allocation data. Continue pathway care.
Once the up front investment of $40 515 for new equipment was made, the clinical recommendations did result in improved outcomes, stable balancing measures, and reduced cost (46% reduction in annual costs of hospitalization).
After its development in March 2015, we have piloted the use of this framework with our clinical pathway teams over the following 12 months. Nineteen opportunities to improve value were evaluated (Supplemental Table 11). Of 16 currently implementable opportunities, 15 appeared to safely improve care with reduced cost. The 16th recommended increased laboratory monitoring with increased cost, with the presumed benefit of improved patient safety. Together, these projects have an estimated annual cost savings of $762 987.
Two additional opportunities estimated to save $257 930 annually were revealed upon analysis to be difficult to operationalize at this time. We also evaluated a proposal to hasten the transition from IV to oral antibiotics in neonates with UTI; however, the team chose not to deploy it at this time due to perceived safety concerns and a lack of evidence.
The US health care system spends over one-sixth of our gross national product.1 Hospital care constitutes 32.1% of this figure. The unsustainability of such spending is self evident. Furthermore, historically, the tradition of medicine is one where a “craft” culture is valued. In such a culture, a practitioner’s personal experiences and individualized wisdom equates with a favorable patient outcome. This craft mentality, while potentially individually satisfying, produces inconsistent results because the lack of standards makes it difficult to know which intervention is best and how it can be improved.
The stylistic representation of Value (= Outcomes/Cost) is succinct. However, the “equation” does not inherently come with measurable units. As a result, hospital systems often try to improve value solely through minimizing costs, because costs are the most measurable in the face of variations in clinical practice. The Value Analysis Framework described above has transformed this conceptual equation into a functional tool that could be used in a variety of settings by groups or individuals across institutions, and can be used to analyze standardizing variable practice, or changing a standard practice. This comparative tool is able to use multiple value analysis strategies simultaneously (waste reduction, cost-minimization, cost-effectiveness). A value analysis for a given recommendation is made complete by considering 3 factors: (1) its ability to be implemented, (2) its potential to impact quality (eg, efficacy, potential to cause harm, population-related benefits), and (3) its cost (direct treatment and diagnostic costs and those costs associated with anticipated complications). This tool provides an easily digestible way for clinicians to communicate clinical care tradeoffs effectively with hospital administrators and with payors, and can help to organize new clinical recommendations to determine those with the greatest potential value. As simple as the “value equation” appears to be, it is difficult to translate into an enduring culture of value within the health care system. We are in the early stages of this journey. The advantage of using a simple yet robust framework is that it helps health care decision-makers to stop and think not only about costs, but also about implementability and benefits before pursuing projects. An example of this at our institution is the shift in thinking from “eliminate all point of care testing,” to the new thinking of “use point of care testing only where rapidity is more crucial than absolute accuracy.” We are raising our overall “value literacy” by educating staff on the proper use of the value analysis tool and cost dashboard, then implementing improvement projects identified during our pilot within the ordering providers’ workflow. When we implement these changes, we will instrument them, so that we can track whether anticipated outcomes result. Piloting the value analysis tool has helped us to balance the concepts of implementability and safety/clinical utility, so that we can look at the entire value proposition and not just at cost. Our pilot projects demonstrate that focusing on cost alone is not sufficient to ensure the provision of quality care.
This framework does suggest that some of the variation in care noted by Wennberg and Cooper15 may be due to operational differences between organizations, and differences in organizational approach, as well as provider variation. Important questions for the future include the following:
How can hospital infrastructures be improved to replicate successes within and between institutions, and to maintain widespread gains?
How can payment structures help hospitals to invest in the infrastructure necessary for high value care?
How will value analyses affect hospitals relationships with payor networks and accountable care organizations?
Is there a role for hospitals to collaborate/share value analyses and effective interventions?
What are the legal implications of a hospital system whose costs are transparent to providers and patients?
What is the return on investment of a mature culture of value?
The value analysis and decision-making framework is based on an analysis of internal work products in the context of team experience, informed by the medical literature. Repeat analysis of this data without the context of the team experience might lead to different conclusions. The cost dashboard data are specific to our institution and is dependent on the quality of data entered into the system, whereas some of this data has been normalized, there are sometimes minor idiosyncrasies that we have found that need to be further investigated and corrected. As the cost dashboard is sourced from an active financial cost accounting systems, changes in allocation methodologies applied over time will cause changes in the underlying data. Cost data are accurate at the time the dashboard is accessed, but results may be difficult to replicate as allocations can cause the data to be updated at any time. Cost dashboard information does not include physician inpatient billing or depreciation. We hypothesize that given the way that physician billing is determined, it would tend to either balance out, or make differences in cost more pronounced than what is estimated by the value analysis framework. We did not explicitly consider depreciation in our model.
This framework helps to analyze costs in the setting of population health (an Accountable Care perspective), caring for patients with specific diagnoses over a 12-month period. Although the framework makes these decisions more transparent, not all hospitals may plan to reduce resource utilization (admissions, bed days) to reduce costs.
To meet the Triple Aim, a new cultural norm must be created, which highlights safely spending less, replacing unnecessary care with necessary care, and teaching our new generation of providers that value is a virtue. The value analysis and decision-making framework and cost dashboard represent useful tools to help clinicians, administrators, payors, and families to discuss population-level quality of care tradeoffs, as aggregated treatment decisions for individuals. These tools represent a standard method by which all relevant dimensions of value are considered. Failure to adhere to such a standard may result in an “incomplete” value analysis. Unless all dimensions of value are considered, we may become prone to decisions that are disproportionally influenced by costs, or other unconscious bias. More research is needed in how to develop high value care recommendations, ensure that these recommendations are successfully implemented and sustained, and are presented to appropriate peer institutions for implementation (possibly with appropriate supplemental financial investment), so that the best care can be provided to patients everywhere.
We thank Seattle Children’s Hospital Administration at our institution for funding this strategic initiative. We acknowledge and thank the CE team, pathway owners, pathway stakeholders and team members (including nursing, pharmacy, laboratory medicine, and radiology), our clinical information system analysis and analytics teams, clinicians, and patients/families without which we would not have been successfully able to impact improvements to local clinical practice. This framework has been in use at our institution through 2015. It has been presented locally to the University of Washington Department of Biomedical Informatics and Medical Education on April 2, 2015, and September 11, 2015, at the American Board of Pediatrics on September 22, 2015, and at a Children’s Hospital Association Analytics meeting in 2015.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: This project was supported as part of the employment of the authors at Seattle Children’s Hospital in the Clinical Effectiveness group.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- ↵CMS.gov. National Health Expenditures 2014 highlights. Available at: https://www.cms.gov/Research-Statistics-Data-and-Systems/Statistics-Trends-and-Reports/NationalHealthExpendData/downloads/highlights.pdf. Accessed December 23, 2015
- Berwick DM,
- Nolan TW,
- Whittington J
- Lakdawalla D,
- Shafrin J,
- Lucarelli C,
- Nicholson S,
- Khan ZM,
- Philipson TJ
- Glick HA,
- McElligott S,
- Pauly MV,
- et al
- Braddick M,
- Stuart M,
- Hrachovec J
- Ralston SL,
- Lieberthal AS,
- Meissner HC,
- et al
- ↵American Academy of Pediatrics. Committee on Quality Improvement. Subcommittee on Urinary Tract Infection. Practice parameter: the diagnosis, treatment, and evaluation of the initial urinary tract infection in febrile infants and young children. Pediatrics. 1999;103(4 pt 1):843–852
- Pryles CV,
- Atkin MD,
- Morse TS,
- Welch KJ
- Phillips B
- Wennberg JE,
- Cooper MM
- Copyright © 2016 by the American Academy of Pediatrics