Engineering Journal · AUTHOR · March 24, 2026 · 20 min read

Do Not Automate a Decision You Cannot Describe

Most automation fails because it copies a form, not a judgement. Use AI first to watch how people actually decide, then write the rule they can still stand next to.

By Golam Sorwar, Tech Lead and Full Stack Engineer in Dublin.

The first version of automation I trusted, and then regretted, looked responsible. A form came in. A script classified it. A queue moved. We had removed a human from a repetitive step. Then the exceptions arrived — the ones the form did not have a box for — and the script did what scripts do. It was consistent, and it was consistently slightly wrong. I see the same shape now that people want models in the loop. Route the ticket. Draft the reply. Flag the attendance case. Look up the policy. The hunger is honest. Staff time is real. The mistake is also honest: we try to automate a process we have never watched, using the official diagram instead of the decisions people actually make at 4:30pm with three extra facts that never made it into the schema. I do not want AI as the first decision-maker. I want it as a patient reader of history: here is what was asked, here is what a human did, here is what repeated. Then a human says whether that repetition is a rule we are willing to own. Only after that do we put a workflow in front of the queue. That order is slower to announce. It is the only order I will defend in an organisation that already has a reputation to lose.

Context

The work sits in college operations, not in a lab. Tickets from students and staff. Policy questions that are usually the same and occasionally not. Attendance and exception patterns that look like data and feel like judgement. Draft replies that must sound like the institution, not like a model that has read the internet.

We already had software. That matters. This was not a greenfield "AI product." It was a set of humans using a back-office every day, leaving a trail of statuses, comments, reassignments, and overrides. The trail was incomplete, like all trails, but it was more truthful than the process poster on the wall.

I also had a team that could ship ordinary automation — queues, rules, templates — without a model. So the question was never "can we call an API." The question was "are we about to encode a fiction." If the fiction is official process, the model will learn the poster. Staff will keep doing the real job in side channels, and you will have automated the part nobody was struggling with.

The real problem

The apparent problem is volume. Too many tickets. Too many repeated questions. Too much time spent copying a policy paragraph. Those are real, and they are the problems that get budget.

The deeper problem is that the repeated work is not the same as the repeated decision. Two requests can share a form and need opposite outcomes because of a fact the form never asked for: a date, a visa constraint, a previous exception, a conversation that happened at a desk. If you automate the form, you punish the cases that were already expensive.

A second deeper problem is that organisations often want automation to settle an argument they have not had. Who is allowed to make this exception. What "late" means. Which queue owns a refund question that is also a records question. A model will pick a winner with confidence. That does not make the argument finished. It makes the argument harder to reopen, because now there is a system in the way.

I care about this because I have already seen non-AI versions. A status automation that closed tickets people still needed. A rule that routed everything with a keyword to the wrong team because the keyword was how students talked, not how departments were organised. Adding a model to that instinct does not make it wiser. It makes it cheaper to scale the mistake.

Constraints

We could not halt operations to run a clean study. People needed replies this week. Any method that requires a six-month freeze before a single rule ships will be bypassed by a well-meaning script.

Data was messy. Comments were half the decision. Some outcomes lived in email. Some overrides had no reason recorded because the person was trusted and in a hurry. If your method requires perfect labels, you will either invent them or give up.

Privacy and tone are not optional in this setting. Student and staff text is not a public corpus. Drafts can be wrong in ways that are legally and socially expensive. A routing mistake is repairable. A confident wrong policy sentence is a different class of harm.

The team was small. We could not stand up a research org. The method had to be something engineers and operations could run together: export, cluster, argue, write a rule, watch.

We also could not treat "the model said so" as an audit trail. If a decision affects a person, someone has to be able to say why without pointing at a probability.

Options considered

Automate the official process now. Encode the poster in code or in a prompt. Advantage: speed, a demo, a feeling of progress. Disadvantage: you automate the fiction. Risk: staff invent workarounds around your workflow, which is how shadow process is born. Maintenance becomes a fight between the system and the people who still know the job. The business gets a faster wrong.

Put a model on the live queue immediately as the router or the reply. Advantage: it will look clever on the first twenty typical cases. Disadvantage: you have no baseline for what "typical" even was. Risk: silent drift, prompt changes nobody reviewed, a tone problem you find from a complaint. Complexity is hidden, which is the worst kind. I reject this as a first step in anything that touches people.

Hire more staff and change nothing. Advantage: judgement stays human. Disadvantage: you also keep the repetitive part that was actually regular. Risk: burnout, and a later automation project that is more desperate. This is sometimes the honest answer for a season. It is not a strategy if the work is truly repeating.

Observe, then propose, then automate with an audit. Mine historical requests and the human actions that followed. Use a model to group and describe patterns, not to act. Humans accept, rewrite, or reject each pattern as a rule. Only then does software act, with logging and a way back to a person. Advantage: you automate a decision someone can still explain. Disadvantage: it is slower to the first demo. Risk: the history is biased — you will learn how people behaved under last year's constraints, including their bad habits. That risk is why validation is not optional.

Decision

We used the long way: real requests, historical outcomes, human decisions, model-assisted analysis, human validation, then rules or a narrow workflow, then monitoring.

I chose that because I wanted automation that operations would not have to fight. If they do not recognise the rule, they will route around it, and you will have spent engineering time making the official system less relevant.

I also chose it because it keeps the model in a role it is actually good at. Models are useful readers. They can say "these four hundred tickets look like the same judgement with different nouns." They are weaker as unattended officers of an institution. I am willing to use them to draft, to suggest, to cluster. I am not willing to let them close the loop on a person's case until a human has said the loop is boring enough to be a rule.

Why not skip analysis and "just use RAG on the policy"? Because policy documents are what the organisation wishes it did. The decision log is what it did. Those disagree more often than people who write policy want to admit. Retrieval can help a draft once you know which question you are in. It cannot tell you which question you are in if you have never watched the inbox.

Implementation / Process

The pipeline is ordinary on purpose.

Start with the real requests. Not a synthetic set. The last season of tickets, forms, attendance exception asks, the actual text people sent. If you cannot get that, you are not ready to automate. You are ready to guess.

Attach the human decision. What queue did it go to. Who touched it. What status did it end in. Did someone override. Did a second person reverse the first. This is ugly joinery. Do it anyway. A request without an outcome is not a training story. It is a fragment.

Let a model analyse, not act. Ask it to group similar decisions and to write the rule it thinks the humans were using, in language a staff member would recognise. Ask it to find the cases that look like the group but went the other way. Those contradictions are the product. They are where the unofficial fact lives.

Then sit with the people who did the work. This is the step engineering tries to skip because it does not look like shipping. Show them the proposed rule. Listen for the sentence that starts with "yes, except." That except is either a second rule, a missing field, or a decision you should not automate.

Only then write software. Sometimes the software is a template and a tag. Sometimes it is a router with a small set of features you now know matter. Sometimes it is a draft reply that a human still sends. Sometimes it is a checklist the model is not invited to. The model may drop out of the runtime entirely. That is a success if the analysis taught you the rule.

Ship with an audit. Who decided, or which rule, on which version, with which input. Keep a hold-out: a slice of traffic that still goes to people, or a review sample, so you can see whether the rule still matches the job. When it drifts, you do not "tune the prompt in production" like a hobby. You reopen the description.

Attendance and policy lookup fitted this more cleanly than I expected, once we admitted that the first job was to name the exception types people already used. Ticket routing was messier because tickets are a bag of leftover work. We automated the boring centre and left the bag.

// Analysis artefact, not a runtime brain.
{
  "pattern_id": "attendance-late-same-day",
  "proposed_rule": "If the student is marked late the same day and staff already noted transport, offer the standard excuse path.",
  "support": "human decisions from the last term",
  "contradictions": ["exam day", "already on a warning"],
  "human_verdict": "accept_with_exceptions",
  "runtime": "checklist + optional draft",
  "must_not": "auto-close or change the official record"
}

Problems and failures

The history taught us some habits we did not want to keep. People had routed a class of tickets to a particular inbox because one person was kind, not because the function belonged there. A model will call that a pattern. It is a dependency. Validation is how you refuse to industrialise a kindness that was already burning someone out.

We tried to analyse too many workflows at once. The method only works if someone can still hold the exceptions in their head. One workflow, one season of data, one table of contradictions. Then the next.

Draft replies created a tone problem before they created a facts problem. The facts were often fine. The voice was generic in a way that felt cold, or warm in a way that felt fake. Staff refused the drafts, which was the correct response. We had to write voice rules like we write validation rules: short, testable, owned.

Some stakeholders wanted the live model because the analysis phase looked like delay. I had to say that a confident router without a described decision is not faster. It is a future incident with better branding.

We also under-estimated how much of the decision was in attachments and hallway context. The text field is not the request. If your data only has the text field, your rule will be a rule about text fields. Say that out loud so nobody thinks you have automated the job.

Trade-offs

We traded a dramatic demo for a smaller, owned rule. I would make that trade again. Drama in this domain is usually someone else's day going wrong.

We left humans in the loop longer than a vendor pitch would like. That costs staff time now. It buys you an automation they will not sabotage.

We accepted that some work will never be a rule. That is not a failure of nerve. It is how you keep the institution from pretending judgement is a classification problem. Automation should make the remaining human work more visible, not less.

Result

The useful result was not "we have AI in production." It was that a few high-volume decisions could be written in a sentence a staff member would sign. Some of those sentences became ordinary software. Some became better forms that asked for the fact people were already using. Some became a draft that a human still judged. Those are different runtimes of the same analysis.

Where we skipped the analysis and jumped to a helper, we got something staff treated like a novelty and then ignored. I take that as evidence, not as a reason to push harder on the model.

I will not invent hours saved. I will say the conversations with operations got more specific. We stopped arguing about whether to automate and started arguing about which except belonged in the rule.

What I would do differently today

I would have recorded reasons on overrides before I asked a model to find patterns. Garbage in is not a cliché when the garbage is the only signal of judgement. A required one-line reason on an exception is an engineering change that makes later analysis sane.

I would have kept policy retrieval out of the first routing experiments. Mixing "where should this go" and "what is the answer" in one prompt made both worse. Sequence the questions. They are different decisions.

I would also have been clearer with leadership that the first deliverable is a description, not a bot. If they cannot value a description, they will not value a rule, and they will not support you when the bot needs to be pulled back.

Broader lesson

Companies try to automate a process before they can describe the decision. Software then hardens a misunderstanding. AI makes that cheaper and more fluent. It does not make it wiser.

Use the model as an observer: read the requests, read the human actions, propose the pattern, wait for a human to claim it. Then automate the claim. Monitor it like you would monitor a payment state, because you have changed how the organisation treats a person.

If you cannot write the decision in a sentence you are willing to defend, you do not have an automation candidate. You have a wish, and a model that will help you scale the wish.

Closing thought

The order is the argument. Real requests, historical data, human decisions, analysis, validation, then a rule with an audit. Reverse that order and you will ship something that looks like progress from a distance and like contempt from the inbox. I would rather automate a boring truth than a confident story about how the work was supposed to go.