Christopher J. Peterson MD, MS, Sterling Rosqvist MD
Corresponding author: Christopher J. Peterson
Contact Information: Christopher.Peterson@unchealth.unc.edu
DOI: 10.12746/swjm.v13i57.1515
The development of generative artificial intelligence has caused many to wonder if AI technology will replace human jobs, including physicians. While AI can produce impressive output, extrapolations about its future capabilities are limited by the inherent unpredictability of the future. What’s more, AI technology is grounded in a physical reality with physical limitations, including the vast amount of resources required to develop and maintain AI algorithms. These algorithms cannot think, intuit, or possess any understanding of the information they produce nor the physical world people inhabit. Furthermore, the practice of medicine cannot be reduced to algorithms or formulae, and requires the creativity, intuition, experience, and humanity only found in a person. The proposition that AI can replace human physicians both undercuts the complexity of medicine and the richness of the human experience, reducing it to mere information output.
Keywords: Artificial intelligence, AI, ChatGPT, generative AI, general artificial intelligence, algorithm, machine learning, large language model, technology, medicine, healthcare, doctor, physician, future of medicine, humanity, clinical reasoning
Artificial intelligence has captured the imagination since its conception. Since the Dartmouth Conference in 1956 (which coined the term artificial intelligence or “AI”),1 there has been no shortage of theories about AI, its capabilities, its dangers, and just when the “singularity” (the development of machine intelligence equivalent to human intelligence) would occur.2,3 It has been the subject of apocalyptic science fiction,4 warnings from scientists and tech giants,5 and predicted to revolutionize industries from healthcare to business to manufacturing to entertainment- just about everything.6 Some claim it’s the key to a human utopia.7 Others, the cause of humanity’s demise.8
Typically, when pop culture has conceptualized AI, they’re referring to artificial general intelligence (AGI), or an AI system that can function with human-level intelligence at a wide variety of tasks.9 This is, without a doubt, the so-called “Holy Grail” of AI. The quest for AGI has captivated some of the best minds in computer science. Yet until recently, the history of AI research had been one of unrealized expectations.2 For decades, AI research ebbed and flowed, with few breakthroughs coming close to the promise of AGI.10 Breakthroughs like AlphaGo and DeepBlue, programs that could compete with and even beat humans at games like Go and chess, showed AI could be impressively effective at specific tasks.10 Still, technology that appeared truly comparable to human intelligence (think the Turing test for human-like intelligence11) had not yet been developed. That all changed, or at least appeared to, with the development of Generative Pre-Trained Transformer 3 (GPT-3), otherwise known as “Chat GPT”. Chat GPT is a large language model capable of generating text in convincing human-like diction in a matter of seconds.12 Like an oracle, users could ask ChatGPT any variety of questions or queries and receive an immediate, thorough, eloquently written response that seemed nearly indistinguishable from a response written by a human.13,14 It was considered a monumental breakthrough in AI research and signaled a new era in AI capabilities and research. Almost overnight, industries were rushing to find ways to develop and incorporate this new AI technology; a veritable “AI arms race”.15 Forecasters predicted no end of disruptions to work, daily life, and even human existence as we know it.16,17 The medical world was no exception. For example, in 2023, Chat GPT was able to pass the USMLE exam (in reality, it approached the minimum passing grade).18 Another study claims that generative AI was more empathic than physicians.19 Other studies showed ChatGPT could excel physicians in responses to patient questions20 as well as diagnostic reasoning and accuracy.21,22 It didn’t take long for claims about AI being smarter than doctors, better than doctors, and replacing doctors23 to surface. After all, how could the human mind compete against a machine with the world’s library at its disposal? Suddenly, a world without doctors started to seem plausible, perhaps even desirable, with human minds and emotions against razor-sharp computer precision.
We think these predictions about AI’s takeover of medicine are premature, overblown, and, at times, sensationalist. Even worse, these attitudes undermine the true cognitive and emotional challenges that comprise medicine, reducing it to numbers, algorithms, and probabilities. At some level, these predictions even diminish what it means to be human. For these reasons, we’ve written a counter to the claims of AI’s inevitable takeover of medicine with the hope that, in doing so, physicians will feel both more secure in their future and perhaps have greater hope for their profession and humanity.
There is no shortage of predictions about AI- what it will do, what it will become, how it will take over the world.3,24 The predictions themselves have arguably drawn as much or more attention than the actual AI algorithms themselves. To be fair, it’s natural to want to predict the future. Forecasting money markets, predicting elections, and football games-being able to accurately predict the future gives someone a distinct advantage. Unfortunately, we often forget that our predictions about the future are just that- predictions. We forecast the weather, we don’t foretell it. We predict the market trends, but we don’t promise them. True, we have sophisticated methods for making these educated guesses. And some phenomena, like the next fly-by of Haley’s comet, we can predict with remarkable accuracy. Other situations, not so much. For example, weather forecasting has dramatically improved in the past several decades, with the average temperature error for a 7-day forecast today comparable to that of a 3-day forecast in the 1990s.25 Current models may predict up to 14 days in the future.26 And yet, models will likely continue to be limited by randomness and unpredictability of our chaotic universe.27 Many believe that because of this, a near-perfect prediction of the weather for any substantial length of time is likely to be impossible.28 Politics is another example. In the 2016 US presidential campaign, Democratic candidates Hillary Clinton and Kamala Harris were slated to win each of their races by polls and pundits.29 Of course, neither candidate won their election as predicted. Quite ironically, Hillary Clinton’s campaign’s reliance on a machine learning algorithm to make campaign decisions is considered to be a major factor in dooming her presidential bid, with the algorithm effectively ignoring key stats and demographics.30 Of course, this is nothing new for political polling.31 The aptly titled study “Election Polls are 95% Confident but only 60% accurate”31 exemplifies the divide that often exists between our perception and true ability to predict political outcomes, a gap evidenced in numerous other studies.32 True, polling accuracy and forecasting may vary by methodology.33 Nonetheless, advances in technology have not removed the inherent randomness or unpredictability of the universe. This remains true in other areas like infectious disease epidemiology (think COVID-19 pandemic34 or influenza forecasts35,36) and economics (like difficulty predicting the 1990 or 2008 recessions37,38).39 The inherent propensity of overconfidence in imperfect models prompted one economist to suggest an “oath of humility” for any Nobel laureates in the same field.39 It would seem that our ability to prognosticate is largely not as good as we think.
This brings us to the current grab-bag of AI predictions. One concern is that the goalpost seems to keep moving. Computer scientist and author Ray Kurzweil predicted in 2014 that AGI would arrive in 2029, though more recently he has moved the date to 2045.40 Baked into many of these (over)confident predictions is that AI will continue to grow exponentially, undeterred, on and on into the future without slowing down or stopping. “If AI is this good today,” the reasoning goes, “imagine how better it will be tomorrow or next week or next year”. This seems like a logical approach, like a simple line plot taught in secondary schools. Unfortunately, the world we live in is non-linear and chaotic. It’s this uncertainty of living systems that makes them inherently unpredictable and the physical constraints of the universe that make undeterred, unending growth unlikely. Or, as put by engineer Christian Keil- “My 3 month-old son is now TWICE as big as when he was born. He’s on track to weigh 7.5 trillion pounds by age 10”.41 These limits apply not only to biological systems, but machines which are created, used, and surrounded by these unpredictable living systems.42 Therefore, it’s highly unlikely that AI technology will continue to grow at its current rate.43 Largely because it’s unlikely any technology can maintain indefinite exponential growth and development.44 Even Moore’s Law, both an observation and a prediction about the doubling of transistors in a circuit every 2 years, is unlikely to continue indefinitely.45–47
Rather, AI technology will almost certainly reach a plateau in the coming years (some argue it’s already arrived48). Other technologies have fared the same. During the space race, NASA had plans for a much more ambitious space program, which included a moon base and a Mars landing (complete with nuclear-powered spacecraft)- all within a few decades of Armstrong’s famous small step.49,50 Instead, NASA’s space program is a shadow of what it once was, only just returning to sending its own astronauts into space (and not without major setbacks51). Other types of technologies have also seemingly slowed, peaked, or plateaued.52 This somewhat predictable pattern is described as the Gartner Hype Cycle, named after the technology and consulting firm of the same name.52 It contends that most technology travels through a predictable pattern before finally settling on its “steady state” use in society. There is the initial breakthrough (“Innovation Trigger”), followed public optimism and unrealistic expectations (“Peak of Inflated Expectations”), which then transitions to failure to meet these expectations and waning enthusiasm (“Trough of Disillusionment”), followed by a resurgence where the technology is improved and placed into a more practical and realistic framework (“Slope on Enlightenment”), finally leveling off with more widespread integration and stable usage (“Plateau of Productivity”). What is observed in the Gartner Hype Cycle is both a reflection of physical technological limits as well as society’s predilection to be somewhat naive and overly optimistic about new ideas (or at least to play into the marketing hype). Of course, the Gartner Hype Cycle is only a model. It doesn’t dictate that things will follow this trajectory, only presents a template for what is likely to happen. And it’s not without criticism; well-known technologies fit the cycle, though ultimately many may not.53 While the Gartner Hype Cycle is admittedly also a prediction of the future, it is one with precedence. A specific technology that grows infinitely and exponentially, to our knowledge, has not and does not exist. The likelihood that AI is the exception to this is just as unlikely.
Yet one thing is clear- we do not have a great track record of understanding the true impact a technology will have. Failed technological predictions include dismissals of now staple technologies such as automobiles and the internet.54 While laughable in retrospect, the truth is that predicting the future is hard. This applies to laypersons and experts alike. While predictions from non-experts like billionaires or politicians may be dismissed, ominous warnings from computer scientists carry a different weight. Geoffrey Hinton, computer scientist and Nobel Laureate, often called the “Godfather of AI” for his work on neural networks, estimates there is a 10–20% chance that AI will “wipe us out” (admittedly a gut prediction).55 Naturally, the headlines and thumbnails for these interviews reflect the ominous nature of these statements, such as “I tried to Warn Them, But We’ve Already Lost Control!” and “‘Godfather of AI’ Predicts It Will Take Over the World”55,56 (Hinton also advocates for continued AI development with regulations, though these statements don’t seem to make the headlines as often).55,57 While predictions from those actually involved in the field have more credibility, they are not without fallibility. For example, Dr. Hinton confidently predicted in 2016 that within 5–10 years AI would be better than radiologists and that the field should stop training new ones (nearly 10 years later, AI hasn’t).58 Other “godfathers” of AI also made dramatic predictions- predictions which turned out to be entirely wrong. In 1970, Marvin Minsky claimed that in “three to eight years will have a machine with the general intelligence of an average human being”40 Economist Herbert Simon predicted that, “… machines will be capable, within twenty years, of doing any work that a man can do” (this was in 1960).59 As scientist and futurist Roy Amara puts it in what is known as “Amar’s Law,” “We tend to overestimate the effect of technology in the short run and underestimate the effect in the long run”.60 This seems particularly prescient here- too many are focused on the imminent “impending doom” of AI and less on what a steady state (“Plateau of Productivity”) with AI will look like.
So how to approach thoughts about AI and the future? There is no easy formula, and any reader should be just as cautious with this work as with any other claim or critique about AI. However, there are general principles that can be applied. For example, computer scientist Rodney Brooks lists what he calls “The Seven Deadly Sins of AI Predictions,”60 which include conflating technology with magic, output with understanding, and growth with indefinite exponential growth. We tend to agree. One unifying element of AI predictions is that they require massive leaps in innovation. To this end, the same author expresses skepticism of predictions of AI supplanting entire industries where, to date, not a single worker has been replaced by a machine.60 And to this we add not a single physician either.
We talk of AI in mythical terms as an all-powerful entity floating out there in the ether, like the Greek Gods of old living in the clouds (pun intended). Despite such mythical depictions, AI is still grounded in a physical reality. The hardware that makes AI possible is expensive, occupies space, and requires, well, money. It’s also limited by physical laws, from wire conductivity to memory capacity. Building, powering, housing, and cooling this technology requires tremendous resources, so much so that some are concerned about the effects of AI-driven data centers on climate change.61 So does the time and energy it takes to train and update the AI model.62 The first iteration of GPT required 1,287 MWh to train (comparable to the annual consumption of 1000 US households63),62 with ChatGPT queries requiring up to 10 times the energy of a Google search.64,65
Regardless of what companies are willing to invest in AI, some studies and commentators suggest that AI abilities have already begun to peak.48 Current concerns are driven by model collapse and Model Autophagy Disorder (MAD) resulting from training on AI-generated data.48,66–68 Other studies suggest that there is a limit to the current AI algorithms- it doesn’t necessarily get better with more information, more nodes, and more iterations.69,70 This is because data quality, the risk of concept drift and covariate shift, and the need for human oversight and interpretation of output remain both risks and necessities for a machine that neither understands the world nor is free from the human biases and limited data that feed it.71–74
Text-to-image generation serves as an example of both AI’s capabilities as well as its limitations. AI’s text-to-image generation is impressive, with AI tools like DALL-E producing artwork on command. While much art generated by the lay user has a distinct “AI look”,75 more sophisticated users can create impressive outputs (one such piece one an art competition).76 What’s more, image generation has created concerns about “deepfakes”, or media altered or wholly generated using AI with the intent to deceive and often with political motivations.77 This “AI art” has caused some to wonder if artists, too, won’t also be steamrolled and swallowed by the AI machine.78,79 We think that the claim of AI replacing artists, just like physicians, is flawed. This AI-generated art is based on human prompts (and subsequent human fine-tuning of the results) in order to produce realistic images.80 Thus, AI is not truly acting autonomously nor functioning as the creative force. Second, AI “art” does not reflect any lived experience, emotion, or internal workings of the soul. The opposite of the tortured artist, the eccentric creative, or the wandering poet, AI is simply the “made to fit”, assembly-line form of media. It becomes a reflection of the consumer, the masses, the datasphere, and not the mind and life of an individual. It takes no risks, makes no demands of its audience, and dares no one to look inward or outward or at anything at all.81 And it can claim no self-expression. True, people may be moved by an image regardless of whether it was produced by a human or a machine, though people tend to prefer art by artists.82 Furthermore, AI’s ability to produce art is, well, based on it being fed a large amount of other art as a precedent. Up until now, this was all human-produced art, meaning that AI really only imitates human art.83 Of course, all artists are influenced by someone else’s works. Arguably, AI’s consumption of other artists’ work may not seem any different. However, take what happens when a large portion of this art becomes AI-generated. In other words, what happens when AI trains on its own output? Rather than produce a positive feedback loop or more sophisticated forms of art, researchers have found that when algorithms are fed AI is generated by AI, the images become deranged, distorted, and down-right bizarre.84 The devolution of AI art after being trained on its own generated content is a phenomena known as “model collapse.”85,86 This is not a trivial concern. Model collapse will likely become a major challenge to AI model development as more and more AI-generated content becomes prevelent.87,88 The threat of model collapse could place limits on AI’s output, productivity, and so-called creativity.
If this output deterioration can occur with art, what would happen if other fields were to become dominated by AI? Would an AI “doctor” function more effectively if a majority of medical reports, textbooks, and guidelines were also generated by AI? Or would it acquiesce to a type of digital groupthink? We’ve seen the consequences of groupthink before- from the Challenger space shuttle disaster89 to Pearl Harbor and the Vietnam War.90 Science, research, and progress are negatively impacted by insular, derivative thinking on the arts and sciences.91,92 When we become derivative, we lose the complexity and nuance that make life beautiful and science innovative. Perhaps part of the reason for this is that medicine, art, and life can’t be reduced to algorithms.
In the opening scene of The Matrix, the famous cascade of electric 1’s and 0’s creates both the iconic futuristic opening and sets a foundation for the film- reality can be reduced to a series of 1’s and 0’s. Of course, not quite. Some characters notice dissimilarities, so-called “glitches” in The Matrix.93 To those who leave The Matrix and experience “the real”, returning to this world is an illusion. So too with medicine. To the outside observer, medicine might seem like something easily reducible to an algorithm. A lot of information, to be sure, but programable, codifiable, and reducible. Yet just like in The Matrix, those who have visited “the real” (i.e., actually practice medicine) understand the nuances, complexities, and differences.
A microcosm of this idea has taken shape in the medical community regarding AI and medicine. They claim that the work of a physician can be perfectly transferred over to a machine. The reality of a physician is replicated by 1’s and 0’s, if you will. After all, machines have more memory, faster recall, and quicker “processing” than a human being. Surely this means that it can practice medicine better? While impressive on the outside, the insidious nature of this argument is this: medicine can be reduced to an algorithm. That, regardless of the role of first-hand experience, critical thinking, clinical gestalt, and emotional intelligence- medicine is ultimately mathematical, predictable, and formulaic. A machine. To be fair, in the age of evidence-based medicine, it’s easy to think of medicine as the mere application of algorithms and guidelines. The transition to a more evidence-based approach was expected to usher in a new era using the best medical research to standardize and improve care on a large scale.94 And truly, there were good reasons for this. Large amounts of data synthesized by a committee of experts are expected to produce consistently better practice decisions than a single mind, a single physician, or one who may be working with far more limited data. We recognize that a person’s experience and decision-making can be biased, whereas clinical trials and committee rulings are seen as more objective. Unfortunately, this faith in committee recommendations risks an overreliance on them,95 almost as if the words “guideline” removed the human fallibility of those who made it. It’s true that clinical research has revealed holes in our reasoning. For example, anticoagulation (i.e., warfarin) was prescribed for post-myocardial infarction patients, recognizing that this could prevent further thrombosis.96 Trials later revealed that this rationale was limited by adverse events, as anticoagulation results in a higher rate of bleeding97 and no benefit over therapies using more newly developed P2Y12 inhibitors (clopidogrel).98
Yet guidelines don’t apply to many situations. For example, the Surviving Sepsis Campaign recommends aggressive fluid resuscitation for all patients with septic shock, though in a later iteration the evidence level for this was changed from “strong” to “weak”, likely because some patients (such as those who are volume overloaded with heart failure) would be harmed by such resuscitation.99 Furthermore, the guidelines are often changing, with as many as 1 in 5 recommendations being outdated after 3 years.100 Clinical trials often apply to a far narrower spectrum of patients and clinical scenarios than we realize. Guideline-directed medical therapy, or GMDT, is a buzzword for anyone with heart failure. And it has undoubtedly simplified and streamlined heart failure management. Nevertheless, the evidence behind GDMT is far more nuanced than the succinct summaries depicted in infographics. For example, the foundational trials for GDMT medicines often don’t reflect how they are dosed in clinical practice. The initial trial for sacubitril-valsartan tested this medicine at a dose of 97–103 mg.101 The major trial for carvedilol in heart failure test was designed to uptitrate this medicine to 50 mg twice daily in all patients who would tolerate it.102 In our experience, patients with heart failure are rarely on these doses. And yet, this evidence-based medicine is taken as gospel by many physicians who do not understand the evidence behind it. Indeed, the cognitive shift to the era of evidence-based medicine has not come without concern, including an overemphasis on algorithmic thinking, minimizing individualized care, downplaying of clinical reasoning and critical thinking, and acquiescing to clinical studies rather than critically evaluating them in turn.103,104
This false sense of certainty in our medical society pronouncements gives the illusion that medicine could indeed be reduced to an algorithm. Yet those who practice medicine should understand the limitations and failings of guidelines. An overreliance on and outsourcing to AI would be expected to worsen, not improve, these limitations. Those who think AI will replace doctors have likely never encountered the nuance of these guidelines nor the burden of this uncertainty. Though we may be trained to think algorithmically, the universe has not compartmentalized itself into our neat frameworks. Physicians who rely solely on algorithms most likely practice bad medicine and will be the kind of physicians who are the most likely to be replaced by AI. And perhaps the main reason for this is that the true practice of medicine requires intuition, abstract thinking, and creativity, things that are still uniquely human.
Creativity and abstract thinking are essential to scientific progress.81,105 AI currently does not possess these attributes and it’s unclear if it ever will.106 What appears to be creativity is better described as randomness, imitation, and derivation.107 AI lacks intentionality, emotion, purpose, meaning, and an understanding of the world or the art it produces.108,109 As mentioned earlier, AI-generated art becomes more derivative as the corpus of art AI is trained on becomes saturated with, well, more AI-generated art. Some argue AI “hallucinations”, or fabrications created by AI, are a form of creativity. Others think that these hallucinations might be better called “AI confabulations”110,111 though even that term is problematic as it continues to anthropomorphize AI (more on this later).112,113 While some argue this is a form of creativity, we find it hard to equate misinformation with mental effort. In a similar vein, one might call those behind the Enron scandal or the thieves from The Italian Job creatives. The quantum empirical model of physics introduces randomness into the equation of the universe, though one would certainly not consider this a creative act.114 Even if nothing is truly “random” in the universe, we recognize the difference between actions done with intentionality and those as simply results of natural occurrences. So too AI’s “hallucinations” shouldn’t be mistaken for true, productive, creative effort, in part because this so-called “creativity” is really just misinformation.
Intuition is also another crucial element in both human experience and medical practice. It is the ether of human experience, the manifestations of our subconscious, and is considered a means of discerning knowledge about the world by classical philosophers.115 Take what Jonathan Haidt says about our moral frameworks- while we think we are in control of all our moral values, making judgments and insights, and then developing the feeling to match them, he suggests things work quite differently.116 He uses the analogy of an elephant and a rider to explain how our subconscious influences and is inevitably intertwined with our rational mind. While the rider, representing the rational mind, may believe they are in complete control of the elephant, representing the unconscious mind and impulses, in reality, it’s the several-ton elephant that is truly in control. Similarly, we may believe that our beliefs and behaviors are wholly dictated by our rational minds when, in reality, our subconscious exerts tremendous influence. Even our moral frameworks, which we often consider rationally driven, are in many cases post hoc rationalizations of our inner emotions and intuitions. We are not as wholly rational, mechanical, or predictable as we would like to think. While this comes with limitations, the intuitive self also plays a major role in helping us develop productive moral values. Certainly, this inner intuition is not flawless. People have acted poorly based on their internal emotions. Neither is rational thought powerless (Haidt mentions that with great effort and focus, the rider can exert more control over their beast of burden). We simply state that, even at our best and most enlightened, there is more than strict rationalism at play. Motivations and instincts that transcend a rational argument. Beatings and burnings within the human heart compel us towards a better way of living. The point here is that we can’t separate the two, the intuitive and conscious rational mind, and expect to be successful. Likewise, neither can an AI with no such intuition hope to replace the cognitive and emotional framework of a person.
Intuition is an important element of medicine. Studies quantifying it as a real phenomenon and physicians have attested to its role in their clinical practice.117–124 Some may argue that physician intuition, while real, is still inferior to a machine with probabilities. In the end, a machine may be just as effective when all the variables are known perfectly, when the next steps are clear, well-defined, well-demonstrated, and with few, if any, caveats or pitfalls. Unfortunately, medicine rarely falls into this category. As described above, there are many scenarios where guidelines, algorithms, and protocols will not fit perfectly. There is the “fog of war”, where information is unknown, unavailable, or in short supply, as can be common in patient handoffs and transfers. Furthermore, physicians may encounter incomplete or frankly inaccurate information. An incapacitated patient. A hospital transfer without records. A CT scan delayed. A pathology result pending. All these variables are a daily reality in medicine. Navigating them is where a physician’s experience, insight, and perhaps even intuition come into play. That gut feeling that something is wrong. That idea to investigate an unusual diagnosis. Skepticism over a test result that doesn’t fit. Many physicians have reported having such experiences that have caused them to make a diagnosis or avoid an error.117,119,125 Of course, intuition is neither complete nor infallible. Physicians use intuition as part of the diagnostic process- it doesn’t replace analytic thinking, though it can be crucial in atypical cases or when information is severely limited.119 How will AI, a tool detached from the physical world and without awareness, insight, or intuition, navigate this world of uncertainty? What will it do when the next steps aren’t clear? When a patient does not fit neatly into an algorithm? When an unexpected event, like a medication shortage, natural disaster, or global pandemic, demand improvisation? Physician training isn’t just about knowledge; it’s about acquiring the experience and insight to handle situations beyond the textbook scenarios. AI can’t create the “golden mean” needed to solve ethical questions or the next philosophical dilemma in medicine.126 It’s about navigating the uncertainty and complexity of the world.
“The more I learn, the more I realize I don’t know”. It’s a trite phrase, commonly attributed to Albert Einstein. Yet ironically, modern society is too easily lulled into the sense that they have learned all there is to know. It is easy to think that our telescopes and telecoms have shrunk the universe down to size- in reality, it’s opened up possibilities and complexities that we didn’t know existed. Indeed, our models of the universe become more intricate the more we learn; we have not discovered a “theory of everything”, for physics or otherwise,127,128 and the further we go, the less likely it seems that we will.
Nonetheless, the information age has given us the impression that we know most, if not all, things. A professor of mine, then a medical student preparing to enter the next phase of his career, was asked by a mentor, “What do you want to do after you graduate?” He replied, “I’m interested in research”. “Research? Research! Why would you want to do research? All the big discoveries have already been made”. This was in the early 1970s. It’s laughable in retrospect. And yet, it is also tempting to think that in the age of Google searches and gene sequencing, we have a good grasp of the universe ourselves. Of course, if this is true (if all the big discoveries have been made), then it’s arguable that a machine that can master all of society’s knowledge will have effectively mastered the universe. This seems hardly to be the case. But because mystery, wonder, and uncertainty are lost in our modern ear, we likely underestimate just how expansive this universe is and how little we understand of it. To this point, here are some examples of just how much our knowledge continues to expand, and how much farther we may have to go:
In and of themselves, these examples don’t say much about whether or not AI will replace physicians. But it does strike at the insidious presumption that simply having access to information is sufficient and that our current compendium of knowledge is somehow complete. Even what we think we currently know is often subject to debate. The illusion of scientific certainty was criticized by astrophysicist Neil de Grasse Tyson, in response to conspiracies of scientist collusion, who stated, “Anyone who thinks scientists like agreeing with one another has never attended a scientific conference”.51 As more medical therapies and diagnostic tests become available, it’s more likely that the practice of medicine will become more compelling rather than be unified under a single algorithm test, or treatment. In this same vein, AI cannot replace physicians, as physicians do not agree among themselves on almost every topic, meaning that there is no truly unifying paradigm to guide AI. As we gain more information (look at the rate of our scientific knowledge), it’s going to become more difficult for any person or a machine like AI to unify all of this under one mode of thinking.
Rather, those who believe AI will replace physicians likely place a high premium on information and much less on rational thought. Simply having information in a rapidly expanding world is becoming less and less impressive the more readily information becomes available. The ability to think, to reason, to respond to change is the skill that will be needed to navigate an ever-changing world, where there is a flood of information but no set currents to follow. Or, as T.S. Eliot said, “Where is the wisdom we have lost in knowledge, where is the knowledge we have lost in information? The cycles of Heaven in twenty centuries bring us farther from God and nearer to the Dust.”158 So too for a world that forgets how to think.
Shared humanity is what makes a physician-patient relationship unique. Here, a person who is suffering is cared for by a person who also suffers, who also bleeds, who is also mortal, which creates a sense of empathy and duty on the physician’s part. This is particularly important when weighty decisions have to be made. During a difficult end-of-life discussion, a family asked the palliative care physician, “Are we doing the right thing?” The experienced physician agreed and, to emphasize that he meant it, he said, “If this were my father, I would do the same thing that you’re doing”. Not much more needed to be discussed after that. His words had a profound impact precisely because he was a person with parents who could easily find himself in a similar scenario. He was respected for his clinical acumen, but it was his shared humanity that ultimately gave it authenticity. We doubt that a machine reciting the same words could be so convincing. Another vignette comes from the show New York Medical, where a cardiothoracic surgeon is recommending a lobectomy/pneumonectomy to a patient. The clinching line is, “If you were my sister, I would tell you to have this operation”.159 That personal reassurance was palpable. This shared mortality matters because physicians are left to agonize over difficult decisions, knowing that a mistake could irrevocably harm one of our own.160 We mentally place ourselves or our family members in the hospital bed. We ask what we would want done. We sense the weight of the decision as not a game of numbers but one that bisects the human experience. It’s not simply a matter of conflicting options. It’s that there is no easy decision here. Rather than a coin flip or a cold calculator, physicians have to agonize over the choices they make because they recognize the humanity of their patients, their own fallibility, and the cost that comes with getting it wrong. And when mistakes do happen, many physicians experience guilt and shame over their mistakes, and agonize over the potential for future mistakes.161 We imagine most people would not want a physician with a short memory andan indifferent” attitude. While mistakes will happen and it is possible to dwell on them to a self-destructive degree, most patients would likely be concerned by their physicians feeling no anxiety over complex, challenging, and life-altering medical decisions, and feeling no remorse should a bad outcome occur. Some might consider them inhuman, robotic, or machine-like. Physicians who become truly numb to suffering are, unfortunately, victims of the grueling nature of medicine and the horrors of physiology. And yet, while we pity them or consider them martyrs, we don’t consider this an ideal path to walk. What will happen when AI has to have an end-of-life conversation? Or recommend a risky surgery? It may be able to come to a conclusion, it may be able to give reasons for it, but how convinced will its patients be? Furthermore, can AI truly attempt to determine the preference of the individual, or will its output simply reflect those of its programmers, owners, and the average of the data it was trained on? How comforted will we feel when the director of our medical care has neither heart nor humanity? How comforted will we feel when the director of our medical care has neither heart nor humanity? Yes, AI algorithms are undoubtedly efficient. They may be able to make decisions faster than doctors. Their efficiency may lead to lower costs and shorter wait times. But remember, it’s easy to make these kinds of decisions when you’re a robot.
A robot does not have to struggle with moral accountability, guilt, shame, worry, regret, or remorse. In short, it does not have a conscience. While AI may be able to make decisions quickly, it is in part because it does not have to struggle with the consequences of those decisions. It is not held accountable if something goes wrong, and it does not have to suffer the moral repercussionsof injuring another person. The burden of decision-making is nonexistent for AI, thus resulting in faster but not necessarily better decision-making. This lack of moral accountability is so unclear that a major ethical speedbump for AI is just who will take responsibility for the decisions made by these algorithms.162,163 Corporations will not want to accept reliability. Thus, their machines may hedge or “play it safe” to avoid culpability. Hospital systems will not want to accept responsibility, leading to even more bureaucratic red tape to shift culpability back to the manufacturer. Physicians will not want to accept responsibility for a machine whose inner workings will likely remain a mystery. However, in dire and challenging circumstances, a person who is willing to take risks is often needed, not a machine that plays it safe.
Consider the ineffable essence of consciousness, self-awareness, and life itself—the fundamental spark that makes something truly alive. This essence was captured in J.R.R. Tolkien’s legendarium when he wrote in The Silmarillion about the Flame Imperishable, also known as the Secret Fire.164 Described as the animating power to give life, free will, and intrinsic existence to beings in Tolkien’s cosmology, the Flame Imperishable was uniquely wielded by the One, Eru Iluvatar, to breathe consciousness and agency into creation. Melkor, the first and strongest of Iluvatar’s angels, attempted to venture into the Void, or the unorganized universe, to discover the Flame Imperishable so he too could have the same creative abilities. He was unsuccessful, but this proved to be the first of many actions that resulted in Melkor’s corrupting descent, ultimately becoming Morgoth Bauglir, the chief antagonist in Tolkien’s cosmology. With Morgoth denied the ability to create true life with free will, his power centered around the corruption, mockery, and perversion of the creations of Iluvatar and the other angels, resulting in orcs, trolls, dragons, and balrogs seen in The Lord of the Rings movies. Tolkien’s Silmarillion captures the idea that there are irreproducible elements to existence, which may prove true of AI.
James Masterson, MD, the psychiatrist and psychoanalyst, laid the foundation used today in the treatment of personality disorders, primarily narcissistic personality disorder (NPD) and borderline personality disorder (BPD). In The Search for the Real Self: Unmasking Personality Disorders of Our Age, Masterson’s model articulates a Real Self and a False Self.165 The Real Self is the element of the individual that is authentic, genuine, and holds thoughts, words, actions, and behaviors congruent with feelings of meaning, being real, and feeling alive. The False Self is a persona or mask created to shield the Real Self and gain external validation. The False Self is associated with emotional detachment, lack of fulfillment, and relationship challenges. When an individual experiences abandonment, neglect, trauma, or other environmental factors that promote conformity over authenticity, especially during childhood, a False Self develops in that individual. The False Self directs individuals to defer their creativity, desires, wants, and needs to conform to perceived external expectations, resulting in either the grandiosity seen in NPD, or the worthlessness felt by those with BPD. It could be postulated that AI, even AGI, does the same as it acts like a human being in its communication, explanations, and development of answers to prompts, but is only conforming to external expectations programmed into it or asked of it through a prompt. The algorithm that one is interacting with is hollow, much like how individuals with NPD or BPD feel as their personas perform for those around them, as there is no center that holds values and acts on those values, but acts on external directives and expectations. Simply put, AI is only an efficient and high-grade imitation of a human being. All values must be directed and placed into the machine learning model. While this is the case, human beings will have something unique over AI.
Even if AI develops the capacity to develop its own values, it is not clear that the mechanism by which individual human beings develop their values is understood and therefore can be properly replicated. As mentioned above, it is theorized in moral psychology, by Jonathan Haidt in his work The Righteous Mind: Why Good People are Divided by Politics and Religion, that our judgments and justifications operate via two different processes.116 He presents several experiments in the book that demonstrate that individuals have an intuitive response to a violation of boundaries or values and then attempt to rationalize their intuition. As mentioned previously with the example of the elephant and the rider, this demonstrates that moral intuitions, or more simply put, one’s values, arise first and are what primarily drive one’s thoughts, words, actions, and behaviors. If this is true, it is reasonable to propose that the Real Self is where these values are made and held. This is likely true as both Haidt and Masterson indicate that genuine interaction with the world in a vulnerable way is required to enact and update one’s values. This vulnerability is the only way that one can express the Real Self and is likely genuinely irreproducible and uniquely human, much like the Flame Imperishable described by Tolkien.
Human vulnerability is the heart of what is irreproducible by AI. In the 1880s and 1890s, the French painter James Tissot, after rediscovering his Catholicism, depicted several Biblical stories from the Old and New Testaments. One of which was L’hemoroïsse, known in English as The Woman with an Issue of Blood.166 This work depicts the account of the woman who likely was a hemophiliac, who over the course of 12 years “suffered many things of many physicians” and “spent all that she had and was nothing bettered but rather grew worse.” The painting depicts approximately 50 faces, in a crowded street, with greys, blues, yellows, greens, with some red, in dull and matte tones. The works surround the moment, guided by an intuitive sense that she may be healed if she is to touch the fringe of the robes of the Christ, the woman is seen reaching with her right hand in an act of vulnerability and genuine expression towards what she believes to be truth, healing, and direction. While this is a religious depiction, it illustrates a uniquely human experience of seeking truth through vulnerable action, guided by intuition and the Real Self.
There are countless examples of vulnerable movement towards a perceived good or truth despite struggle. Nellie Bly, the journalist who feigned insanity to be admitted to the insane asylum on Blackwell Island in 1887, endured unsanitary and deplorable conditions, trusting that those from the paper she was working for, the New York World, would be able to secure her release. Her book, 10 Days in a Mad-House, spurred investigations, government intervention, and set a foundation for rethinking the treatment of psychiatric illness.167 Viktor Frankl, the Jewish psychiatrist who was imprisoned, primarily, in the Nazi concentration camp of Türkheim, a satellite camp of Dachau, spent his time investigating the capacity for human beings to endure extreme hardship, cultivating his theory on the will to meaning, or one’s ability to cultivate purpose in life, as being central to psychological wellbeing, superseding Freud’s will to pleasure and Adler’s and Nietzsche’s will to power.168 Marie Curie, despite the death of her husband after they both won the Nobel Prize in Physics in 1903, sexism, and xenophobia, she went on to win the Nobel Prize in Chemistry, becoming the first and only person in history to win two Nobel Prizes in separate fields. Her work set the foundation for the field of radiation oncology with the discovery that neoplastic cells were more sensitive to radiation compared to healthy cells. Other examples include Nelson Mandela, Malala Yousafzai, Frederick Douglass, Socrates, Mahatma Gandhi, and many others. Imagine the contributions made by these individuals being derived from a protocolized algorithm or a hypothetical self-aware machine.
Author and public speaker Simon Sinek noted the hollowness that comes from replacing human thought and intention with AI. He creates a scenario where, after an argument with his girlfriend, a man goes to ChatGPT in search of how to best apologize and ameliorate the situation. ChatGPT tells him exactly what to say- “I want to take full accountability” and “I care about this relationship”. These responses are all seemingly the right things to say. The young man goes back to his girlfriend and recites his AI-generated apology. “Did you get this from ChatGPT?” his girlfriend asks. What is her reaction going to be when he replies, “I did”?169 Despite saying all the right things, we would not expect such a synthetic apology to be well received. It lacks the intention, sincerity, and moral depth of an actual apology created by a person. The same will likely be true for AI-driven therapy, where AI takes the role of a therapist. Despite its accessibility and touted empathy, it’s unlikely that this can truly substitute for a patient-clinician interaction. Therapist-patient interactions are often highly dependent on a “goodness of fit” between the personalities involved;170 it is not simply reducible to psychological theory or technique. And while AI may be able to mimic the guidance from therapists, we suspect that these responses too will fall flat as more and more users realize this advice comes from a machine that has no understanding of what it means to be human, to be alive, or, for that matter, doesn’t understand anything at all. How could an AI counsel a dying patient? How could an AI claim it would have an operation if it stood in the patient’s shoes? Quite simply, it cannot. A depiction of what this failed attempt would look like is captured in the short film “Instant Doctor,” which portrays a futuristic healthcare system where an AI has replaced doctors and electronic photo-booth-esque “waiting rooms” have replaced doctors’ offices. A middle-aged man with a nagging cough, after missing his train, stops by one of these convenient “instant doctors” at the train station. Rather than a cold, he’s diagnosed with metastatic glioblastoma multiforme, followed by an equally abrupt prognosis (8 months), family members updated via TikTok, and referral to a palliative care center.171 At the end, the man is left alone on the station platform, diagnosed but without dignity.
Perhaps this is because “Patients don’t [want] to simply be treated. They want to be cared for”.172 Perhaps that’s one reason why people prefer human physicians to artificial intelligence.173–177 Perhaps that’s why, as recounted by physician Abraham Verghese, a patient with breast cancer would leave a highly respected medical center, where the medicine was cutting edge but not a single physician bothered to perform a physical exam.178 Perhaps that’s why the humanities, in addition to histopathology and health systems, are more and more frequently taught in medical schools.179 To replace a doctor with a machine denies the humanity within us that must also be cared for.
Though it’s unlikely that AI will replace doctors, it is likely that AI will become an integral part of medical practice. There will likely be tasks that AI can augment or replace, such as note writing, adding billing codes, chart review, and literature search.180,181 Like other tools or instruments, there are functions that AI can clearly perform better than humans (such as millisecond calculations). Even still, it is unlikely that even the tasks we outsource to AI will be without human oversight.182,183 As with any other tool, those who learn to use AI will have an advantage over those who don’t. Furthermore, while AI may be unable to replace doctors, it does not mean that people won’t try. AI may provide an enticing cost-saving alternative to human labor, regardless of the poorer care and outcomes that may result. The public may be particularly susceptible to the idea that AI could be a suitable alternative to a physician. It seems as though many of the people who believe AI can practice medicine have never practiced medicine themselves. Thus, doctors may need to better articulate the complexity and nuances of their work. To these points, we’ve included some practical thoughts on how to proceed into this new future.
One doesn’t need to be a computer scientist or learn how to code to understand the fundamentals of artificial intelligence. Resources like courses from the American Board of Artificial Intelligence in Medicine (ABAIM) or books like Intelligence-Based Medicine184 and Deep Medicine,185 can make the technology behind artificial intelligence accessible to physicians.
Artificial intelligence can aid in clinical decision-making and help process the wealth of ever-evolving medical information. While this is not a substitute for clinical reasoning, resources like Open Evidence and Glass Health can provide rapid ways for physicians to obtain answers to clinical questions, identify clinical studies, and help develop differentials. These algorithms are not without flaws and are not a substitute for clinical judgment. Nevertheless, it is the physician who fails to adapt to this new technology that will likely be at risk of falling behind.
Physicians spend years becoming experts in a particular discipline. As discussed, medical reasoning is complex and nuanced, and a result of both theory and experience. Unfortunately, the complex reasoning that goes into diagnosis and management is too infrequently articulated to other physicians or in the medical record. We often see this in subspecialist consults, where laconic recommendations are given without explanation or insight. We recognize that busy patient loads and electronic medical records that have produced more clicks than it has saved time contribute to this. Nevertheless, the mind of the expert too often remains below the surface, with only the bare bones of this hard-earned wisdom in view.
As discussed above, medicine can’t be neatly reduced to algorithms. That being said, some clinicians rely far too much on these guidelines and algorithms with a consummate lack of critical thinking. Guidelines are meant to provide a framework, not to be rigid, sacrosanct formulae for how medicine should be practiced. A skillful physician will understand the guidelines well enough to deviate from them when necessary. While institutional, bureaucratic pressure, and heavy patient loads may make it difficult not to follow these guidelines in all instances, with the term “standard of care” used as an excuse for standard reasoning, physicians need to nevertheless use that which separates them from AI: their cognition and intuition. If medicine outsources these truly human resources of algorithms, then AI may indeed replace physicians to the benefit of those who own, program, and train the AI rather than the benefit of the patient. AI will serve its owners rather than the patient.
Over the past few decades, the medical humanities have become continually integrated into medical care, likely out of a recognition that caring for people requires more than cognition. We would encourage physicians to continue to engage with the medical humanities throughout their careers, as we feel the humanities provide the strongest defense against a medical field overrun with AI. Failing to engage with and develop that which separates us from machines may make it easier to believe the lie that physicians are merely reducible to output. Furthermore, as explored in this article, AI creates challenging questions around identity, consciousness, and privacy, questions that benefit from training in ethics and philosophy. Even more than this, physicians should engage in practices that develop and strengthen the humanistic qualities that fundamentally separate us from AI in the first place.
We write this article at a unique time in medicine, society, and history. No one knows what a future with generative AI will look like, though one thing is clear- things will never be the same. And yet, this has always been a truism every time something new, something daunting, something poorly understood has burst onto the horizon of the human landscape. Generative AI is impressive, useful, and even daunting, though it is neither infallible nor incorruptible. And most of all, it is not human. Indeed, the truly insidious nature of arguments about AI replacing humans is the implicit reduction of humanity to information output. We don’t argue that tools like AI can produce more information more quickly than any one human. And yet mere output is not synonymous with reason, insight, or intuition. While proponents of the AI surpassing humans argument do note that humans maintain some unique faculties, like emotional intelligence, these faculties are sometimes relegated to a lesser form of intelligence, one useful for interacting with other humans but not understanding the fabric of the universe. And yet, as we hope we’ve shown, reality is understood and experienced by far more than binary output and mechanical thinking. To say that AI is interchangeable with, or even superior to, the human mind is to cheapen the nature of knowledge, undercut the nature of reality, and vastly degrade the human experience. Medicine, perhaps more than any other discipline, highlights the need for an intersection of multiple forms of thinking, perception, and reasoning, as well as the irreducible element of the human psyche. AI will create new challenges and opportunities. It will likely become integrated into everyday life. It has and will almost certainly continue to be used in the practice of medicine. Yet despite the hype and hubris surrounding its potential, we believe that such tools cannot replace humanity nor those things driven by it.
We would like to offer our sincere thanks to Alexander Cook, Benjamin Lee, and David Peterson for their thoughtful review and valuable insight.
Article citation: Peterson CJ, Rosqvist S. Artificial intelligence and medicine: Hype, hubris, and the humanity we can’t replace. The Southwest Journal of Medicine 2025;13(57):1–21
From: Department of Internal Medicine, Virginia Tech Carilion School of Medicine, Roanoke, VA (CJP) Department of Internal Medicine, University of Utah School of Medicine, Salt Lake City, UT (SR)