Balancing metrics and meaning: Rethinking data and models in healthcare

Data and models never fully capture the realities they represent. Scientist and philosopher Alfred Korzybski famously aphorized, “The map is not the territory,” and, “The word is not the thing.”¹ Data reflect the past or near-present moments, and models provide theoretical frameworks. Neither encompass the labyrinthine scope of reality.

The 2008 global financial crisis starkly illustrated the dangers of overreliance on mathematical models, such as those used to assess mortgage-backed securities, which failed to capture the full complexity and risk of real-world economic behavior.²

In healthcare, the risks of overreliance on data and models are similarly profound. Predictive algorithms built on biased data have misprioritized patients,³ as seen in early COVID-19 triage tools that underestimated risks for minority populations. Scheduling systems that lean too heavily on data can miss clinician workload nuances, contributing to burnout and reduced care quality.⁴ These are not rare exceptions. Biases and faulty generalizations are widespread in research and operational practice.

Still, when used wisely, data and models are indispensable. They bring structure to decision-making, enable predictive analytics for population health, and support evidence-based protocols that improve patient outcomes. For example, predictive models can flag at-risk patients for early intervention, helping reduce hospital readmissions.⁵ Data-driven insights also streamline operations, from staffing to supply chain logistics. The key is to treat these tools as decision support, not decision substitutes.

Buckingham and Goodall (2019) observed that solutions to complex problems lie in the “tangible and changing realities of the world as it really is,” not in abstract representations of the past.⁶ Models built on outdated data often misalign with present-day systems. In contrast, real-time, human-centered insights often outperform abstract models not designed for dynamic conditions or nuanced contexts. Strategically, overemphasizing data can mislead healthcare organizations, such as when hospitals prioritize short-term metrics like bed turnover rates at the expense of long-term community investments, ultimately undermining population health.⁷

However, dismissing data and models entirely ignores their clear benefits. Clinical decision support systems enhance diagnostic accuracy when paired with physician expertise, reducing misdiagnosis rates in oncology.⁸ Machine learning models have also enabled earlier disease detection — for example, sepsis algorithms now identify at-risk ICU patients with up to 85% accuracy before symptoms escalate.⁹ Responsibly applied, these tools save lives and optimize care. But when misused or overtrusted, they carry real risks.

The McNamara fallacy and evidence-based medicine vs. warm data

The empirical movement in healthcare, branded “evidence-based,” prioritizes measurable data and often creates a feedback loop: research favors what is easily measured, and what is measured is deemed valuable. This approach, rooted in the foundations of evidence-based medicine (EBM), tends to elevate experimental evidence — such as randomized controlled trials — over clinical judgment, implicitly assuming objectivity is possible without theoretical or observer bias.¹⁰

Yet philosophers and practitioners alike acknowledge that observation is never neutral; it is shaped by the observer. Nora Bateson, building on her father Gregory Bateson’s systems thinking, introduced the idea of “warm data” — information that captures interrelationships within complex systems, often invisible in quantitative silos.¹¹ Warm data challenges linear thinking by uncovering the “pattern that connects” across different contexts, revealing dependencies that isolated data points often miss.

The “McNamara fallacy,” coined by sociologist Daniel Yankelovich,¹² illustrates the perils of privileging quantitative data over contextual understanding. During the Vietnam War, U.S. Secretary of Defense Robert McNamara fixated on measurable statistics — e.g., body counts and tons of rice seized — while ignoring cultural and contextual knowledge. This obsession with quantifiable metrics contributed to flawed decisions and strategic failure.¹³ Yankelovich’s framework outlines the slippery slope:

1. Measure what is easily measured.
2. Disregard or arbitrarily quantify what cannot be measured.
3. Assume the unmeasurable is unimportant.
4. Conclude the unmeasurable does not exist.¹⁴

This logic leads to strategic blindness. As sociologist William Bruce Cameron warned, “Not everything that counts can be counted, and not everything that can be counted counts.”¹⁵

Despite its limitations, EBM has propelled important clinical advances. Randomized controlled trials (RCTs) have shaped improved standards of care, such as statin use to reduce cardiovascular mortality by up to 30%.¹⁶ But these trials often fail to represent diverse populations or account for individual variations requiring clinician judgment in real-world settings.¹⁷ Even EBM pioneer Donald Berwick cautioned¹⁸ that EBM’s dominance has created an “intellectual hegemony” that must be balanced with empathy, context, and patient-centered care.¹⁹

The perils of metrics-driven culture

In healthcare, an overemphasis on metrics, such as productivity scores or quality ratings, can erode patient trust and clinician morale. A fitting parallel comes from baseball: The Houston Astros’ 2017 sign-stealing scandal, where the team used hidden cameras to decode opponents’ signs, gaining an unfair advantage and damaging trust and integrity.²⁰ Similarly, overreliance on healthcare performance metrics can elevate efficiency over ethics, contributing to clinician burnout and increasing risk of medical errors.²¹ Unlike baseball, the stakes are far higher in healthcare. Compromised patient safety or inequitable care are life-altering consequences. Yet EBM’s focus on “proven” interventions can reduce the autonomy of the doctor-patient relationship, potentially denying treatments not deemed effective by its standards, or conversely, increasing costs by validating expensive interventions.²² We need balanced, human-centered approaches.

Still, well-designed metrics can improve care. For example, CMS infection reduction programs, built on hospital quality metrics, incentivized adherence to best practices and led to meaningful improvements in patient safety.²³ When metrics are transparent, thoughtfully implemented, and aligned with outcomes that matter to patients, they can drive excellence while maintaining trust.

But poorly applied metrics often backfire. Consider employee engagement surveys: while intended to gauge workforce morale and guide culture improvements, they are often reduced to numeric targets. Superficial interventions, like generic team-building exercises, may ignore deeper systemic issues, such as excessive workload, lack of autonomy, or ineffective leadership.²⁴ When workers sense that numbers matter more than their lived experiences, trust erodes.

This creates a “tail wagging the dog” scenario. Instead of solving systemic problems, organizations fixate on isolated data points. Leaders may launch significant initiatives simply to boost dashboard scores, while neglecting the complex interplay of cultural dynamics, staff wellbeing, and patient expectations that underlie those metrics. The more productive path is to pair quantitative data with human insights to guide interventions that reflect lived realities.

Are models illuminating and useful?

Statistician George Box famously quipped, “All models are wrong, some are useful.” Even in hard sciences, models are simplifications; their value lies in their utility, not perfect accuracy.²⁵

In healthcare, models like evidence-based protocols are maps, not the territory. Metrics are not assessments; productivity is not treatment; and quality scores are not patients. Models can fail when applied too rigidly. Risk adjustment formulas, for example, often underestimate disease burden in underserved populations, leading to underfunded care.²⁶ Overreliance on predictive models for hospital expansion has led to misaligned investments, like building facilities in areas with declining need.²⁷

Still, good models serve important roles. Risk stratification helps healthcare systems allocate resources for high-risk patients.²⁸ But the complex nature of healthcare requires constant reassessment: What do patients need? What constraints do clinicians face? What matters most to the community? Local, context-driven answers often yield better results than rigid adherence to abstract, research-friendly standards.

End users empower contextual intelligence

General Stanley McChrystal’s leadership of the Joint Special Operations Command (JSOC) during the Iraq War offers lessons for healthcare. He implemented daily Operations and Intelligence (O&I) briefings with up to 2,000 participants across ranks and agencies. Each contributor gave a one-minute update, prioritizing relevance over hierarchy. This democratic, spontaneous, and frequent exchange was built on key principles:

• Information grows stale quickly and must be shared rapidly.
• Coordination depends on shared context, not just top-down directives.
• End users, those closest to the work, are best equipped to assess and interpret the value of that information.^29,30

Healthcare systems can benefit from this same humility and openness. Nora Bateson’s Warm Data Labs, inspired by her father’s systems thinking, invite diverse stakeholders — clinicians, patients, and caregivers — to explore interwoven challenges through transcontextual conversations.³¹ These sessions often surface insights that metrics alone cannot. For example, a Bateson-style Warm Data Lab could explore how social determinants, cultural factors, and clinical protocols interact, revealing patterns that inform more equitable care.

The Greek myth of Procrustes, who forced guests to fit his iron bed by stretching or cutting their limbs, warns of the harms of rigid conformity. In healthcare, inflexible systems, such as standardized EHR templates, similarly distort practice, forcing clinicians to prioritize data entry over patient interaction. Seventy percent of physicians report EHR-related stress.³² National benchmarks, when applied without adaptation to rural healthcare entities, fuel closures and widen access gaps.³³

By contrast, McChrystal’s approach empowered users to adapt and interpret information in real time. In healthcare, this translates to flexible systems, such as customizable EHR dashboards, flexible workflows, and clinician input into system design.³⁴

Looking along and at: Balancing metrics and meaning

The tension between structured data and lived experience is universal. Philosopher Immanuel Kant warned that reason without experience invites false understanding.³⁵ C.S. Lewis described the difference between looking at a sunbeam from the outside, and looking along it — seeing what it illuminates. Both perspectives, he argued, are essential.³⁶

In healthcare, data help us to look at care. But only experience lets us look along it — through the eyes of patients and providers. The Patient-Centered Outcomes Research Institute (PCORI) exemplifies this balance by integrating patient perspectives with data-driven research to inform care.³⁷

By combining looking at data with looking along patient experiences, healthcare models are more likely to reflect human-sized goals.

To do this well, data and models must be paired with relational insight, clinical wisdom, collaborative decision-making, and humility. Leaders must digest models with grains of salt. Critics of EBM have long noted its narrow definitions of evidence, lack of empirical validation in complex cases, and limited applicability to individuals. When measurable data are prioritized over meaningful care, the result can be technocratic, rather than therapeutic.³⁸

Still, when used wisely, models and data enable precision medicine, operational efficiency, and support equitable resource allocation.

To navigate complexity, healthcare leaders must pair robust data infrastructure with values such as empathy, clinical expertise, contextual insight, deliberation, and human-centered judgment. And models must be tested, transparent, and frequently reevaluated, informed by data but anchored in human experience.

Notes:

1. Korzybski A. (1933). Science and sanity: An introduction to non-Aristotelian systems and general semantics. International Non-Aristotelian Library.
2. Hodgson GM. (2009). “The great crash of 2008 and the reform of economics.” Cambridge Journal of Economics, 33(6), 1205–1221. Available from: https://bit.ly/4lHIbTt
3. Obermeyer Z, Powers B, Vogeli C, Mullainathan S. (2019, October 25). “Dissecting racial bias in an algorithm used to manage the health of populations.” Science, 366(6464), 447–453. Available from: https://bit.ly/4krDex4
4. Shanafelt TD, Dyrbye LN, West CP. (2017, March 7). “Addressing physician burnout: The way forward.” JAMA, 317(9), 901–902. Available from: https://bit.ly/40ui8HB
5. Bates DW, Saria S, Ohno-Machado L, Shah A, Escobar G. (2014, July). “Big data in health care: Using analytics to identify and manage high-risk and high-cost patients.” Health Affairs, 33(7), 1123–1131. Available from: https://doi.org/10.1377/hlthaff.2014.0041
6. Buckingham M, Goodall A. (2019). Nine lies about work: A freethinking leader’s guide to the real world. Harvard Business Review Press.
7. Gottlieb LM, Wing H, Adler NE. (2017, November). “A systematic review of interventions on patients’ social and economic needs.” American Journal of Preventive Medicine, 53(5), 719–729. Available from: https://doi.org/10.1016/j.amepre.2017.05.011
8. Raciti P, Sue J, Retamero JA, Ceballos R, Godrich R, Kunz JD, Casson A, Thiagarajan D, Ebrahimzadeh Z, Viret J, Lee D, Schüffler PJ, DeMuth G, Gulturk E, Kanan C, Rothrock B, Reis-Filho J, Klimstra DS, Reuter V, Fuchs TJ. (2023). “Clinical validation of artificial intelligence–augmented pathology diagnosis demonstrates significant gains in diagnostic accuracy in prostate cancer detection.” Archives of Pathology & Laboratory Medicine, 147(10), 1178–1185. Available from: https://doi.org/10.5858/arpa.2022-0066-OA
9. Nemati S, Holder A, Razmi F, Stanley MD, Clifford GD, Buchman TG. (2018, April). “An interpretable machine learning model for accurate prediction of sepsis in the ICU.” Critical Care Medicine, 46(4), 547–553. Available from: https://doi.org/10.1097/CCM.0000000000002936
10. Cohen AM, Stavri PZ, Hersh WR. (2004). “A categorization and analysis of the criticisms of evidence-based medicine.” International Journal of Medical Informatics, 73(1), 35-43. Available from: https://doi.org/10.1016/j.ijmedinf.2003.11.002
11. Bateson N. (2017, May 28). “Warm Data.” Available from: https://bit.ly/3IiCTQg
12. Yankelovich D. (1972). Corporate priorities: A continuing study of the new demands on business. D. Yankelovich Inc.
13. Daddis GA. (2011). No sure victory: Measuring U.S. Army effectiveness and progress in the Vietnam War. New York, NY: Oxford University Press.
14. Yankelovich.
15. Cameron WB. (1963). Informal sociology: A casual introduction to sociological thinking. p. 13. Random House.
16. Cholesterol Treatment Trialists’ Collaboration. (2010, November 13). “Efficacy and safety of more intensive lowering of LDL cholesterol: A meta-analysis of data from 170,000 participants in 26 randomised trials.” The Lancet, 376(9753), 1670–1681. Available from: https://doi.org/10.1016/S0140-6736(10)61350-5
17. Cohen, et al.
18. Berwick DM. (2005). “Broadening the view of evidence-based medicine.” Quality and Safety in Health Care, 14(5), 315–316. Available from: https://doi.org/10.1136/qshc.2005.015669
19. Berwick DM. (2009). “What ‘patient-centered’ should mean: Confessions of an extremist.” Health Affairs, 28(4), w555–w565. Available from: https://doi.org/10.1377/hlthaff.28.4.w555
20. Goldman T. “How the Houston Astros’ sign-stealing scandal affects trust in baseball.” NPR. Feb. 26, 2020. Available from: https://bit.ly/3GBGJTT
21. Carayon P, Gurses AP. (2009, February). “Nursing workload and patient safety in intensive care units: A human factors engineering evaluation of the literature.” Intensive and Critical Care Nursing, 21(5), 284–301. Available from: https://doi.org/10.1016/j.iccn.2004.12.003
22. Cohen, et al.
23. Pronovost PJ, Watson SR, Goeschel CA, Hyzy RC, Berenholtz SM. (2015, January 21). “Sustaining reductions in central line–associated bloodstream infections in Michigan intensive care units: A 10-year analysis.” American Journal of Medical Quality, 31(3), 197–202. Available from: https://doi.org/10.1177/1062860614568647
24. West MA, Dawson JF, Admasachew L, Topakas A. (2011). NHS staff management and health service quality: Results from the NHS Staff Survey and related data. Department of Health. Available from: https://bit.ly/4lDhW0t
25. Box, GEP. (1979). Robustness in the strategy of scientific model building. In R.L. Launer & G.N. Wilkinson (Eds.), Robustness in statistics (pp. 201–236). Academic Press.
26. Ash AS, Mick EO, Ellis RP, Kiefe CI, Allison JJ, Clark MA. (2017, October). “Social determinants of health in managed care payment formulas.” JAMA Internal Medicine, 177(10), 1424–1430. Available from: https://doi.org/10.1001/jamainternmed.2017.3317
27. Ginter PM. (2018). The strategic management of health care organizations (8th ed.). Wiley.
28. Kansagara D, Englander H, Salanitro A, Kagen D, Theobald C, Freeman M, Kripalani S. (2011, October 19). “Risk prediction models for hospital readmission: A systematic review.” JAMA, 306(15), 1688–1698. Available from: https://jamanetwork.com/journals/jama/fullarticle/1104511
29. McChrystal S, Rose G. (2013, March/April). “Generation kill: A conversation with Stanley McChrystal.” Foreign Affairs 92(2). Available from: https://www.foreignaffairs.com/interviews/generation-kill
30. Buckingham, Goodall.
31. Bateson N. (2019). Warm Data Labs. The International Bateson Institute. Available from: https://bit.ly/44n5JaM
32. Gardner RL, Cooper E, Haskell J, Harris DA, Poplau S, Kroth PJ, Linzer M. (2019). "Physician stress and burnout: The impact of health information technology." Journal of the American Medical Informatics Association, 26(2), 106–114. Available from: https://doi.org/10.1093/jamia/ocy145
33. Wishner J, Solleveld P, Rudowitz R, Paradise J, Antonisse L. "A look at rural hospital closures and implications for access to care: Three case studies." Kaiser Family Foundation. July 7, 2016. Available from: https://bit.ly/4lkVZDV
34. Zahabi M, Kaber DB, Swangnetr M. (2015). “Usability and safety in electronic medical records interface design: A review of recent literature and guideline formulation.” Human Factors, 57(5), 805–834. Available from: https://doi.org/10.1177/0018720815576827
35. Kant I. (1899). Critique of pure reason (J. M. D. Meiklejohn, Trans.). The Colonial Press. (Original work published 1781)
36. Lewis CS. (1970). Meditation in a toolshed. In W. Hooper (Ed.), God in the dock: Essays on theology and ethics (pp. 212–215). Eerdmans Publishing Company.
37. Frank L, Basch E, Selby JV. (2014, October 15). “The PCORI perspective on patient-centered outcomes research.” JAMA, 312(15), 1513–1514. Available from: https://doi.org/10.1001/jama.2014.11100
38. Cohen, et al.