Retrain
Draft
Retrain a model on detected drift, within the delegated band
Retrain, restrict or escalate on drift, at or beyond the delegated band
Allocation
| L3-10 | L3-11 | |
|---|---|---|
| Decides | Model Owner | Business Accountable Executive |
| Consulted | Business Owner | Model Owner, Head of Risk and CAIO |
| Executes | ML Engineering | ML Engineering |
| Evidence | Retraining record referencing the drift trigger | Escalation record and the resulting Layer 4 decision |
| Delegated band | Delegated | Delegated |
In plain terms
Decide what to do when a model’s behavior moves. Inside the thresholds recorded at authorization it is maintenance. At or beyond them it stops being maintenance and becomes a decision about whether the system should keep operating.
What is being judged
Which side of the band the observation falls on, and then what the response is.
The band is the threshold record from L3-02, recorded before validation and bound to the version. Drift inside it is operational and stays with the Model Owner. Drift breaching it, or any drift at all on a system at or above materiality, leaves the layer.
Three kinds of movement get called drift and they do not have the same response. Input drift, where the data distribution has moved but performance holds; often the earliest signal and frequently ignored. Performance drift, where measured accuracy or fairness has degraded against the thresholds. Concept drift, where the relationship the model learned no longer holds, which retraining on recent data may fix or may quietly bake in.
Beyond the band, the options are wider than retraining. Restrict the system to a narrower population or a lower autonomy level. Withdraw it under L4-AUT-05. Retraining is the reflex and it is not always the right answer, particularly under concept drift where the historical relationship has genuinely changed.
What this decision does not cover
Inside the band it does not accept new risk. Beyond the band it does not perform the risk acceptance, which returns to L4-RSK-02 and L4-RSK-03 where the escalation results in continued operation.
When it fires
On event. On a monitoring observation crossing a recorded threshold. On any drift observation for a system at or above materiality. On a data domain fitness change under L3-03. On a source system change that alters input distribution.
On cycle. Monitoring is continuous; the decision is event-driven.
What you need before deciding
The threshold record from L3-02, which defines the band. Drift monitoring records showing which kind of movement occurred. Whether the underlying relationship changed or only the inputs. What the system has decided since drift began, which bounds the exposure.
How this goes wrong
The unmonitored production model: validated at launch, deployed, never instrumented. Performance is assumed stable because nothing is reported, and nothing is reported because nothing is watching. The Layer 3 anti-pattern. Retraining as reflex: every drift observation answered with retraining, so concept drift is repeatedly papered over. Band undefined: no drift bounds in the threshold record, so every observation is an ad hoc judgment and escalation depends on the Model Owner’s discretion, which is precisely what the band exists to remove.
Related decisions
Band defined by L3-02 threshold record.
Threshold Layer 4 §5A model drift response band.
Downstream L4-RSK-02 and L4-RSK-03 where operation continues, L4-AUT-05 withdrawal.
Upstream L4-CLS-04 classification, which decides whether any drift escalates.
Instrument references
NIST AI RMF MEASURE 4 addresses ongoing monitoring. ISO/IEC 42001 Annex A covers performance monitoring. EU AI Act Article 72 requires post-market monitoring for high-risk systems. None specifies the threshold at which a monitoring observation becomes a governance decision.
Correction
The maintainer answers corrections. There is no service level. Responses are best-effort and opportunistic within a reasonable time: a correction raised on a Monday is answered that week or sooner.