Rules get old

Imagine an online shop's customer-service assistant. On launch day it drafts replies from the company's returns policy and delivery rules so accurately that the service team is pleasantly surprised.

Three months later, the shop cuts its returns window from thirty days to fourteen and raises the threshold for free delivery. Nobody thinks to update the rules inside the system. The assistant carries on working — fluent, confident, and quoting the old policy.

After the first complaint from a customer, the team stops trusting it and quietly goes back to the answers they keep in their own notebooks. The system hasn't broken. The rules it learned have grown old.

AI is wrong with complete conviction

When traditional software goes wrong, it usually throws an error, freezes, or at least looks off. When AI goes wrong, the prose is still smooth, the formatting still tidy, the tone still certain. The more fluent the mistake, the harder it is to catch.

Which means you can't run an AI system on a fix-it-when-it-breaks basis. It won't break in front of you. It will simply drift.

Drift comes from four places

• The business changes. Prices, policies, products, processes — every adjustment can leave the rules in the system out of date.
• The data changes. A new kind of customer, a new document format: if the system hasn't seen it, it can only guess.
• The users change. New colleagues ask questions differently from old ones, and the system's performance shifts with them.
• The model changes. When the provider updates the underlying model, the same question can produce a different answer.

The first two are the most common and the most easily missed — because the person changing the policy usually has no idea the policy also lives inside a system.

Three roles, none optional

Keeping the rules current takes three roles:

Someone who spots the drift. Usually the front line, who see the system's output every day and are first to notice when something's off. Give them an easy place to record it, rather than letting it end as a muttered complaint.

Someone who approves the change. It has to be someone who understands the business, not a technician. Changing a returns rule is a business decision.

Someone who makes the change. They put the approved change into the system and keep the old version — if the change turns out to be wrong, you need to be able to roll back.

Many systems go live with only the third role in place.

One simple habit

Hold a fifteen-minute "wrong answers" meeting once a month: bring out the examples where the AI got it wrong, or where a person had to correct it, and decide on each one — change the rule, add data, or accept that this is something it will get wrong.

And watch one number: the share of AI output that people have to edit. If it rises month after month, the rules are ageing, and you don't need to wait for a customer to tell you.

Launch day is the first day it starts to age

An AI system isn't a project you finish and walk away from; it's more like a new colleague who needs regular coaching. Who spots problems, who approves changes, who carries them out — write it down before launch.

Rules will get old. The only question is whether you're the first to know, or the last.