Capability&Consequence

Executive brief

The AI Dependency Your Validation Framework Doesn't Cover

A validated model can still create continuity risk if its replacement takes longer than the supplier allows.

Document
CAC-004-G
Issue
1.0
Published
July 16, 2026
Reading time
7 minutes
Review due
July 15, 2027

Companion Artifact

TL;DR

Validation can establish that a model is fit for purpose while leaving its supported life and replacement exposure unresolved. Add supplier, lifecycle, notice and replacement accountabilities to the existing inventory. Test whether replacement can be completed in time, preserve decision-level provenance and keep protective limits outside the model.

The exposure in one paragraph

Your institution may have mature processes for model oversight, including documented lineage, independent validation and periodic review. Those processes can establish that a model is suitable for use while leaving a different question unresolved: can the institution keep the validated component available, or replace it within the period allowed by its supplier? The answer matters wherever an externally controlled model supports a consequential workflow.

A growing share of models inside consequential workflows are commercial services or ecosystem-dependent components with lives shorter than the surrounding platform or device. When the supplier withdraws one, or the serving stack becomes unsupported, the institution does not get to decline.

Why existing controls may not catch this

Management should examine three potential gaps in the coverage of existing controls.

Validation tests the model, not the dependency lifecycle. Independent validation asks whether a model is fit for purpose, conceptually sound and performing within tolerance. It may not ask how long the exact component will be supported, who controls that timetable or what happens to the validated state when it is replaced. A model can pass validation and still be scheduled for withdrawal.

Change control assumes the institution initiates the change. Banking model-risk management and clinical software change management are designed around a change the institution proposes, assesses and approves. A vendor deprecation is a change nobody inside proposed, on a timetable nobody inside selected.

Ownership can be reported as control. Institutions running fine-tuned open-weight or internally trained models may report the dependency as resolved. Owning weights removes direct withdrawal of the file. It moves obsolescence down to the inference runtime, libraries, toolchain and hardware. The assertion can be true as stated and misleading as understood.

The three exposures worth naming

Continuity. A component central to a decisioning or clinical workflow is withdrawn, and validating a replacement takes longer than the notice period. The fallback becomes a degraded manual process or an insufficiently validated substitute.

Evidence. A customer is declined, or a clinical recommendation is contested, and the question is why. “The model assessed it” is not enough. The record should identify the model, version, applicable policy and date. Where workflows route among several models, many institutions cannot produce that record without investigation.

Reproducibility. Assurance depends on being able to reconstruct how a decision or build was produced. If the tool or serving stack has been withdrawn, exact reconstruction may be impossible. The problem emerges years later during an examination or incident review rather than when the dependency was introduced.

Seven questions for management

  1. Which models in consequential workflows are supplied rather than fully controlled, and what is the contractual and practical notice period for each?
  2. For each, what is the measured elapsed time from a deprecation notice to a validated replacement in production? If that exceeds the notice period, the exposure exists now.
  3. Where the institution runs its own models, what is the supported life of the inference stack—runtime, libraries, toolchain and hardware—and who tracks it? Ownership of weights is not ownership of the executable stack.
  4. Can the institution produce, for any decision in the last twelve months, the specific model and version that produced it? If the answer requires a special investigation, the operational answer is no.
  5. What is the defined behavior when a model component is unavailable or materially changed? Is degraded mode specified, tested and validated as an operating state, or left undefined?
  6. Which controls protecting customers or patients are enforced outside the model? Limits, thresholds, contraindication checks and segregation of duties belong in deterministic mechanisms that survive model replacement.
  7. Who owns the replacement path for each dependency, and has the path been exercised? An untested recovery path is an assumption.

What good looks like

Institutions handling this well can adapt disciplines they already possess. They extend the existing model inventory to record supported life, supplier, notice period, serving-stack dependencies and replacement owner alongside the validation fields. The first step is to add information and accountabilities to a register the institution already maintains.

They establish replacement as a tested procedure, assign an owner and record the elapsed time required to exercise it. Comparing that measured time with the notice period reveals whether the institution can meet its continuity obligation when a supplier forces a change.

They move protective controls outside the model. Where a limit, threshold or exclusion protects a customer or patient, deterministic code or policy enforcement makes the protection stable across model changes and easier to evidence. They instrument decision provenance prospectively. Recording which model and version handled a decision is inexpensive as a design choice and expensive as a retrofit.

The reframe for the committee

The committee should ask management to assess the dependency against the institution's obligation to its customers or patients. The model-selection team supplies important technical information, but the accountability also reaches continuity, model risk and product support.

The exposure is created by the gap between a component's supported life and the institution's obligation. That gap is visible to the owner of continuity, customer or patient duty, model risk, product support or regulatory evidence—not only to procurement or data science.

The institution can apply existing supplier-risk, continuity, end-of-life and change-control disciplines to this component layer. Management should add lifecycle and replacement information to the register and report consequential dependencies whose tested replacement time exceeds the available notice. That produces a specific exposure for the committee to oversee, with an accountable owner and a defined corrective action.

Notes

Banking. In the United States, the Federal Reserve issued revised model-risk-management guidance in SR 26-2 in April 2026. In Canada, OSFI published the final Guideline E-23 in September 2025; it takes effect for federally regulated financial institutions on May 1, 2027. The details differ, but the direction reinforces inventory, governance, validation and ongoing oversight.

Healthcare and medical devices. IEC 62304 establishes medical-device software lifecycle processes. The FDA's 2025 final guidance on Predetermined Change Control Plans permits specified AI-enabled device modifications to be reviewed in advance when the modifications and methods are defined and verifiable. That is valuable for planned change and less complete for a change forced by an external component reaching end of support.

Full argument. The technical foundation is set out in The Model Will Be Gone Before the Product Is.

The decision this should change

Extend the existing inventory with lifecycle and replacement information. Require management to report consequential dependencies whose measured replacement time exceeds the available notice period, with an owner and corrective action.

What this adds

Prevailing consensus

Existing model validation, supplier-risk and business-continuity controls are sufficient if they establish that the model is fit for purpose and governed.

What this challenges

A model can pass validation while the dependency beneath it is nearing withdrawal, unsupported or impossible to replace inside the notice period.

New contribution

The brief turns AI dependency into a board-level lifecycle and replacement question, with seven management questions that expose whether the institution can actually evidence and recover the dependency.

What would weaken the argument

This brief weakens if, through 2028 examinations or assurance reviews, institutions with externally supplied consequential AI dependencies consistently demonstrate replacement paths shorter than notice periods and decision-level provenance without adding lifecycle fields to existing inventories.

Sources and references

  1. Federal Reserve — SR 26-2, Revised Guidance on Model Risk Management
  2. OSFI — Guideline E-23, Model Risk Management
  3. FDA — Predetermined Change Control Plans for AI-enabled device software
  4. IEC 62304 — Medical device software lifecycle processes