WITH THE PROLIFERATION of artificial intelligence (AI)-powered tools in today’s engineering workforce, a case study relating to the downfall of IBM’s $4 billion health care AI system offers a message of caution for engineering professionals when engaging with new technologies. SITUATION Longtime followers of the television quiz show Jeopardy are likely to recall watching the program in February 2011, when IBM’s machine-learning system Watson defeated two of the show’s greatest human champions over a series of three episodes. While the event was designed to offer a public demonstration of the industry giant’s growing capabilities in natural language processing, it also gave a major boost to IBM’s efforts to bring AI-powered tools into the commercial marketplace. In 2015, IBM announced the launch of Watson for Oncology, an AI platform its creators boldly claimed would revolutionize the care of cancer patients. Trained on massive amounts of patient records, case reports, medical papers, and other data, Watson for Oncology was designed to provide new insights to clinicians in diagnosing and treating even the rarest and most complex cases. To build this new platform, IBM sank a reported $4 billion into the acquisition of firms offering health care data and analytic capabilities. It also reached a partnership agreement with one of the US’s most prestigious cancer treatment hospitals, whose team of clinicians would train the AI tool to “think like a doctor.” Yet the doctors working to train the Watson for Oncology system reportedly found that using real case reports was “messier” than expected. Doctors’ notes were often difficult to read or transcribe; case reports, gene sequencing, and other patient data were incomplete or scattered across multiple, sometimes incompatible, digital platforms. Even complete patient profiles proved to be of questionable value due to the pace of changes in treatment guidelines. To resolve these challenges, the team of trainers increasingly turned to “synthetic” case studies: hypothetical patient data used as the basis to input diagnoses and treatment protocols. Unfortunately, this change meant that, instead of representing the collective wisdom of doctors worldwide, the Watson for Oncology system was largely trained based on cases involving the recommendations of a handful of doctors at a single (albeit prestigious) US hospital. Users of the new tool complained that the system’s recommendations did not account for differences in physician preferences or local clinical practices, nor could they be generalized to all cases. These issues were only magnified for the system’s global users. IBM had marketed the tool as a means of reducing inequities in access to health care, yet all too often the system proposed recommendations that could not Ethics Mishandling AI Tools Puts Engineers at Risk for Ethical Violations By Tara Hoke, Aff.M.ASCE Editor’s Note: This periodic column on business ethics is an adaptation of a series originally written for the American Society of Civil Engineers (ASCE), but it also may be applicable to our members. Questions about specific ethical issues related to IIBEC members should be addressed to IIBEC Ethics Committee Chair Matt Durrett, RRO, or IIBEC staff liaison, COO Melany Rizzo, CAE. be implemented in parts of the world with differences in drug availability or required treatment protocols. Worse yet, in 2017, media sources reported that internal testing by IBM found “multiple examples of unsafe and incorrect treatment recommendations” from the system. For example, in a test involving a hypothetical 65-year-old man with lung cancer and a history of severe bleeding, the system prescribed bevacizumab, which is a drug associated with an increased risk of severe or deadly hemorrhage. While IBM dismissed these errors as part of the training process and not reflective of the platform’s true performance in practice, the negative publicity proved harmful to the already beleaguered tool. Perhaps the greatest source of criticism for the Watson for Oncology tool, though, was its failure to live up to its own hype. While many customers found it useful as a research aid, there was little evidence that Watson for Oncology offered any measurable improvement in clinical decision-making or patient outcomes. Far from being a source of new insights or advancements, it seemed the platform’s value was primarily limited to confirming treatment plans or diagnoses a clinician had already made. With sales flagging and several high-profile customers signaling their intent to look elsewhere for AI solutions, by 2019 IBM had begun to scale back on its support of the Watson for Oncology platform. Finally, in 2023, IBM ended years of speculation about the platform’s prospects by announcing that it was exiting the health care business. Watson’s health care data and analytics assets were sold to a private equity company for an undisclosed amount, leading more than one major technology reporter to quip that the AI tool had been “sold for parts.” QUESTION What ethical lessons does this case study offer to the engineering profession? DISCUSSION While civil engineering practice is constantly being reshaped by new techniques and innovations, perhaps no advancement in recent memory has elicited as much excitement—or fear—as AI. Whether it is viewed as a technological marvel that will raise human achievement to unprecedented heights or a data-crunching juggernaut that will replace vast segments of the workforce, few seem to have any doubt of AI’s transformational effect on the field of civil engineering. Whatever the future may hold, it is clear that AI is already a powerful new tool for the engineering profession, but like all such tools, it carries with it an ethical obligation for engineers to investigate and understand its range of uses. This case study demonstrates many of the risks surrounding AI, including concerns about the integrity of data used in training AI 44 • IIBEC Interface May/June 2025 systems, the ease of introducing unintended biases, and an overall lack of transparency about the quality or verifiability of AI outputs. Other concerns not directly referenced in this case include data security, privacy, and legal or ethical questions about the ownership of AI-created content. In view of these concerns, misuse of AI tools may pose a multitude of risks to an engineering professional. Overreliance on AI may entice an engineer to exceed his or her technical competence, may result in subpar service to a client or employer, or may lead to careless or ill-informed decisions that threaten the health or safety of persons who depend on the engineer’s services. The choice to carefully consider AI’s capabilities, limitations, and implications is thus not merely a wise business practice but also an integral piece of the engineer’s ethical duty to others. Despite the faults of the Watson for Oncology platform, it is heartening to note that there are no reports of the system causing harm to patients. It seems clear that the practitioners who used the tool were careful to always balance Watson’s guidance against their own professional judgment, not relying on the technology to the detriment of their ethical obligations or the individuals they served. This example is also noteworthy, then, as a model of ethical behavior for engineers engaging with AI or with any other innovation. ABOUT THE AUTHOR This column, by Tara Hoke, Aff.M.ASCE, general counsel to the American Society of Civil Engineers (ASCE), and a contributing editor to Civil Engineering magazine, is reprinted with permission from the March/April 2025 issue of Civil Engineering and edited for IIBEC applicability. Hoke is a member of the Virginia bar and received her juris doctorate degree from Georgetown University Law Center. This column examines ethical issues considered by the ASCE Committee on Professional Conduct that may be applicable to IIBEC members in the course of their business practices. This article first appeared in the March/April 2025 issue of Civil Engineering as “Mishandling AI Tools Puts Civil Engineers at Risk for Ethical Violations.” TARA HOKE, AFF.M.ASCE HAVE A NICE DAY PHOTO/SHUTTERSTOCK.COM May/June 2025 IIBEC Interface • 45
