A marketing chatbot that invents a product feature is embarrassing. A model that misclassifies a transaction in a regulated workflow is something else entirely. The first costs you a laugh and a quick correction. The second can cost you a fine, a licence, or a relationship with a regulator that took fifteen years to build. When compliance is in the picture, the question is not whether AI can be helpful. It usually can. The question is what happens on the day it is wrong, because it will be wrong sometimes, and the cost of that day is what you are really buying.
In maritime, in financial services, in anything touching safety or money laundering or data protection, the asymmetry is brutal. A model that is right ninety eight times out of a hundred sounds excellent until you realise the two it missed were the two that mattered, and that nobody downstream questioned them because the output looked authoritative. Confidence without correctness is the real danger. People trust clean formatting and a definite tone far more than they should.
So the cost of getting it wrong is not a software cost. It is the penalty for a filing that went out late, the cleanup when an exception slipped through unflagged, the time your best people spend reconstructing what the model decided and why. And in regulated work there is a further bill. You may be asked to explain a decision to someone with statutory power, and "the AI produced it" is not an answer that ends the conversation well.
This is why we are cautious about putting a model in the deciding seat for regulated steps. We would rather it sits beside the process, doing the patient work humans do badly, then surfacing anything unusual for a person to rule on. In the regulated workflows we have worked in, the exception checks were exactly this kind of arrangement. The model never had the final word on the things that carried real consequence. It made the humans faster at finding what deserved their attention.
Three things keep the cost contained. The first is a clear line around what the model is allowed to decide alone and what it must escalate, written down before you go live, not invented after an incident. The second is a record. Every decision the model touches should leave a trail you can hand to an auditor without flinching. The third is a human who is genuinely accountable and has the time and authority to overrule the machine, not a rubber stamp under pressure to keep the queue moving.
None of this means avoiding AI in regulated work. The gains are real and worth having. It means treating the model as a capable assistant rather than an oracle, and designing the workflow so that being wrong is caught early, recorded honestly, and cheap to fix. Get that right and AI earns its place. Skip it and the first bad day pays for everything you saved, with interest.
Facing something similar in your business?
Talk it through with our AI guide, or send the team a note. We will tell you straight whether and how we can help.