As Colorado’s new artificial intelligence law moves toward implementation, healthcare organizations and their lawyers face a question that goes beyond whether an AI system complies with the law: Who is responsible for governing the technology once it is deployed?
Chris Hutchins, a healthcare technology and data strategist with more than 30 years of experience, said the answer involves more than a technology team: Lawyers, clinicians, data experts, risk professionals and other stakeholders need to work together throughout an AI system’s life cycle, from implementation through ongoing monitoring and updates.
Colorado’s new law, SB26-189, taking effect Jan. 1, 2027, establishes requirements for developers and deployers of covered automated decision-making technology used in consequential decisions. Healthcare services are among the areas covered by the law. Among other requirements, the law calls for technical documentation, information about training data and known limitations, instructions for appropriate use and human review, records retention and consumer notice. Developers also must notify deployers about material modifications to covered systems.
For healthcare organizations, Hutchins said, meeting those requirements is only part of the governance challenge.
Since AI systems change quickly, particularly as vendors update models, governance cannot end once a system has been approved or implemented. Organizations need to determine who has authority to approve a system, who monitors it after implementation and what happens when the technology changes, he said.
Those organizational questions become more complicated within the technology environments of hospitals and health systems, which often rely on a variety of systems developed at different times and not necessarily designed to communicate with one another.
“Everyone’s got this really ugly spider web or matrix of systems that don’t talk to each other,” Hutchins said.
Connecting those systems can create additional risks, particularly when information from different parts of an organization begins flowing into an AI application.
“There’s some area that might be between here and here, where those things, if they connect or intersect, they create more risk. That’s one of the bigger challenges in terms of governance,” he said. “The context is really the most important part of it.”
Clinical documentation provides an example. AI tools can help physicians by gathering information and producing a draft record, potentially reducing the administrative work that follows a patient encounter. Yet the technology does not eliminate the clinician’s responsibility to review what it produces.
“It does provide that guided and well-framed data set for them to review ‘What did I say? What did the patient say?’” Hutchins said.
At the same time, anytime AI is used, another potential source of error is introduced. “Caution is always warranted. I love it when I hear it from an attorney or a risk management person or an internal audit person,” he said, via ongoing evaluations.
That said, “It’s not an option to be scared or not get involved and go forward,” he added.
For Hutchins, the primary question is whether a technology improves the delivery of care rather than simply layering on systems and processes.
“No one actually comes to a hospital or a doctor’s office, because they got great analytics,” he said.
Technology, instead, is intended to support the people providing care. As Colorado’s new requirements approach implementation, he said, preserving that human judgment will remain central to responsible AI adoption.