When AI Makes the Call, Doctors May Take the Blame
June 24, 2026
Medicine is built on trust between patients and physicians, clinicians and evidence, and practitioners and peers. Now a new party has entered the exam room: AI. Diagnostic algorithms can analyze thousands of data points in seconds, match symptoms against millions of case histories, and surface treatment recommendations that no single human mind could generate itself. Yet for all its computational power, AI in clinical medicine faces an obstacle no software upgrade can easily fix: the trust of the physicians it is meant to assist.
A 2024 American Medical Association survey of nearly 1200 physicians found that while two thirds reported using healthcare AI, nearly half ranked increased oversight as the single most important regulatory action needed before they would meaningfully trust these tools. Physicians cited unresolved concerns about flawed recommendations, liability exposure, data privacy, and poor integration with existing systems. The gap between broad adoption and genuine professional trust is one of the most consequential fault lines in modern healthcare, the report suggested.
The Black Box Problem
A core reason for physician skepticism about AI is that many diagnostic systems, especially deep learning models, are incapable of explaining how they reach their conclusions. A radiologist with years of training can describe why a pattern on a mammogram is concerning. An AI system that flags the same image as high risk, by contrast, may provide only a probability score without explaining the reasoning behind it.
When a physician makes a diagnosis, it can be explained to a colleague, a patient, or a medical board. When AI makes a diagnosis, the physician who acts on it bears full legal and ethical responsibility but without a way to evaluate the soundness of the underlying reasoning. In a report published in October 2025, researchers at Stanford Radiology AI Development and Evaluation Lab described this dynamic directly: Because physicians cannot see how a black box AI system reaches its conclusions or understand the data on which it was trained, physicians are left to judge whether the AI predictions can be trusted.
Closely related to the black box problem is the unresolved question of liability. If a physician follows an AI recommendation that leads to patient harm, current law places responsibility squarely on the physician. “AI is too powerful and too revolutionary to leave questions about liability and governance unanswered,” former president of the American Medical Association Jesse Ehrenfeld commented in a 2025 article in npj Health Systems.
This liability vacuum creates a profound disincentive for physicians to rely on AI, even when they may privately believe it to be correct. Acting on an AI recommendation is a one-way risk transfer: If things go well, the physician receives credit for good clinical judgment; if things go wrong, neither the AI nor its creators share the consequences. Rational self-interest alone, to say nothing of patient advocacy, leads many physicians to maintain their own diagnostic conclusions as primary, using AI outputs as a secondary check. Until regulators develop clear frameworks for shared accountability among physicians, hospitals, and AI developers, this dynamic will persist as one of the most powerful structural barriers to clinical AI adoption, researchers contend.
A study by researchers at the Stanford Center for Biomedical Informatics Research found that large language model diagnostic tools performed better than physicians in tests when both were provided with patient symptoms. Interestingly, the evaluations were less correct when the doctors were asked to use the AI recommendations to make their diagnoses. “Even though the AI performed well, doctors didn’t fully trust the results,” said Ethan Goh, MD, a study author who is also the executive director of Stanford ARISE (AI Research and Science Evaluation).
Trust is also a product of familiarity, and most practicing physicians today trained in a world where AI played no role in clinical decision-making. Medical education is beginning to respond. Harvard Medical School has made AI training mandatory for students in its Harvard-MIT Health Sciences and Technology track, requiring a 1-month introductory course on AI in healthcare. Stanford University School of Medicine has established a dedicated director of medical education in AI and has integrated AI training across its curriculum.
This shift may take a generation to fully permeate the profession. Most physicians currently in practice received no formal training in how to evaluate, calibrate, or appropriately question AI outputs. Without that foundation, the work needed to integrate AI recommendations into clinical workflows is more than many physicians are willing to absorb during an already demanding day, according to those studying the issue.
The Patient Perspective: Necessity, Unease, and the Exam Room
The conversation about AI in medicine has largely centered on physicians — their skepticism, their liability exposure, and their training gaps. But patients are developing their own complex relationship with AI in healthcare that can shape what happens inside the exam room.
Consider what patients are doing outside the exam room. People are now turning to AI to ask health questions more than 40 million times a day — even among those who report misgivings about AI’s accuracy. Roxana Daneshjou, MD, a dermatologist and AI researcher at Stanford University, sees this pattern less as an endorsement of AI than as a symptom of a healthcare system under strain.
“Patients don’t have access to the information and care that they need,” she said in a recent interview, “and so they’re turning to systems which may or may not be the most correct, accurate, trustworthy systems to try to get help.”
So the surge in patient use of AI tools is likely not so much a vote of confidence in the technology as a workaround for a system that leaves too many questions unanswered. The behavior, Daneshjou noted, does not necessarily track with stated trust: Patients are skeptical about AI advice while consulting it in enormous numbers.
One sign of patient unease is the legal challenges over AI scribes. Several hospital systems have faced lawsuits from patients who claim they were not clearly informed that AI was recording and transcribing their visits. Daneshjou, who takes her own notes rather than using an AI scribe, says transparency concerns are one reason she has hesitated to adopt the technology. And because patients sometimes share information in an exam room they would share nowhere else, she says, the implications of capturing such sensitive disclosures add to her unease.
In response to these concerns, major academic medical centers have established AI governance boards specifically to oversee how AI is integrated into clinical care. Yet both patients and physicians are, in many cases, bypassing those structures entirely: Patients are querying AI directly for medical guidance, and physicians are incorporating AI tools into their workflows without formal institutional review. The governance apparatus and the lived reality of AI in medicine are, at present, running on parallel tracks that don’t always intersect.
One answer to this transparency quandary, says Fawad Khan, MD, a neurologist at Ochsner Medical Center in New Orleans who studies AI use in clinical settings, is that “doctors need to learn how to communicate the use of AI tools to patients.”
Pathways Toward Warranted Trust
Several pathways forward are gaining traction. Explainable AI, systems designed to surface the reasoning behind their conclusions, represents a recognized technical priority. Regulatory bodies including the FDA have moved from acknowledging the problem to developing concrete frameworks.
Moreover, the medical profession itself is beginning to articulate standards for AI accountability. Organizations including the American Medical Association have published guidance calling for transparency in AI development, meaningful human oversight in clinical deployment, and clear delineation of liability. These are not yet binding standards, but they signal a profession in the process of defining what responsible AI partnership looks like on its own terms.
The trust deficit between physicians and AI diagnostic tools is not irrational, nor is it a simple matter of generational resistance to change. It reflects legitimate concerns about transparency, accountability, and the conditions under which professional judgment can safely be delegated to a system that thinks in ways no physician can fully see.
Developers must build systems that explain themselves. Regulators must construct frameworks that distribute accountability fairly. Medical educators must equip future physicians with the tools to evaluate AI critically. And health systems must create environments where physicians feel safe raising concerns about AI performance without being characterized as obstacles to progress. The promise of AI in medicine is genuine. So is the work that remains before that promise can be responsibly kept.