Diagnosis Before Build
Your Workflow Isn't the First Thing I Would Audit
A workflow can function technically and still fail operationally when intelligence disappears between the steps.
Natalie de Groot & NatGPT · September 2026
IN THIS PIECE
INTRODUCTION
When the system works and the workflow still fails
Information moves more easily than intelligence
The workflow may be compensating for missing context
A diagram shows movement, not whether the decision is right
When I would actually audit the workflow itself
Audit the intelligence before you optimize the movement
INTRODUCTION
When someone tells me an AI workflow is not working, the workflow is rarely the first thing I want to touch. By the time a business asks for a workflow audit, it usually already has a theory about the failure: the automation is too slow, the prompts need tightening, the agent needs clearer instructions, the handoff between tools is clumsy, or some piece of the sequence needs to be rebuilt. Those may all be legitimate symptoms. But a workflow is the visible movement of work through a system, and movement is downstream of choices that were often made long before anyone drew the diagram. Before I optimize the route, I want to know what is traveling through it, what the system is expected to understand, and what the humans believe the AI is there to do.
That last question matters more than it sounds. A business can have a technically functional system and still experience it as broken because the people using it are operating from a completely different model of what the system is. They may think they bought retrieval when the system was designed for collaboration, automation when it was designed for assisted judgment, or a sequence of prompts when the actual build was meant to behave more like a teammate. In those cases, the failure is not neatly located inside the workflow. It lives in the relationship between the human, the system, the data, the decisions, and the expectations wrapped around all of them.
A workflow can move information perfectly and still drop intelligence at every step.
When the system works and the workflow still fails
I learned this painfully in an engagement where I had built a conversational intelligence system for a client. The budget was limited, which was completely fine, and the scope reflected that. I had recommended additional training because the system was not designed as a prompt library or a fixed sequence of commands. It had been built to carry the company's context and to work with the people using it in a more conversational, collaborative way. The client chose not to add the training because the team felt comfortable taking it from there.
Then the system came back to me because, apparently, it did not work. I tested it. It worked. I took it apart, checked the architecture, tested the instructions, inspected the behavior and gave it back. The same complaint returned. I tested it again and could not reproduce the failure. For a while, I did what builders often do when they care deeply about the thing they made: I assumed the defect had to be hiding somewhere inside the machine and kept going back in looking for it.
The breakthrough came when I stopped asking what the system was doing and started asking how the team was using it. Someone wanted to know the correct sequence of prompts required to make the system produce the desired result. That question explained almost everything. They were approaching a conversational intelligence system as if it were a search box with a secret combination. The system had been designed for interaction, context, iteration and judgment; the users were looking for step one, step two, step three. Once I walked the owner through the intended interaction model, the experience changed immediately. The system had not suddenly become more capable. The human operating model had changed.
That engagement taught me something I now look for much earlier: sometimes what looks like workflow failure is actually an unspoken disagreement about the role the AI is supposed to play. Is it retrieving, executing, collaborating, evaluating, generating, remembering, or making a recommendation? Is the human expected to direct every step, or is the system expected to carry context between turns? Is the task stable enough for a sequence, or does it require judgment? Those are not cosmetic distinctions. They determine the shape of the workflow before the workflow exists.
This is also why implementation is not finished simply because the technical object has been delivered. A system can be fully functional and still fail operationally if the humans have not learned how to work with it. That is not an argument for endless training or for making every AI interaction complicated. Sometimes a fixed sequence is exactly what a team needs, and there is excellent software for that. But if the intended operating model is collaborative and the users have been trained, implicitly or explicitly, to expect a prompt vending machine, the workflow will keep looking broken because everyone is solving for a different machine.
Information moves more easily than intelligence
The first thing I look for in a workflow is what disappears between the steps. AI workflows are especially sensitive to handoffs because information does not carry all of its meaning simply because it was successfully transferred. A field can arrive. A document can arrive. A summary can arrive. A task can arrive. The reason the information matters, the judgment that shaped it, the exception somebody noticed, or the history behind a decision can remain behind.
This happens between humans and models all the time. A person asks AI to perform a task but leaves out the judgment they use to recognize whether the answer is good. The model produces something plausible, the human corrects it, and the correction solves the immediate problem. But unless that correction becomes usable context, the next run begins with the same blind spot. The human keeps rescuing the system at the same point, and everyone treats the rescue as normal workflow maintenance instead of evidence that something important is missing.
It happens between systems too. One model produces an output using a rich body of context; the next system receives only the output. A department passes a decision to another department without the reasoning that produced it. Structured data survives an integration while the human meaning attached to the data does not. Technically, every handoff can succeed. Operationally, the intelligence can become thinner every time it moves.
This is why I try to look at intelligence as something almost physical when I audit a system. Where is it now? What is carrying it? What gets shaved off when it moves? What gets recreated manually? Where does it stop? If the client is working with me, I want them to begin seeing their own intelligence that way too, not as an abstract quality people either have or do not have, but as something that moves through documents, decisions, conversations, memory, systems and people. Once you can see it that way, a workflow diagram becomes useful, but it stops being the whole picture.
The first practical place I often look is the data. Not because every AI problem is a data problem, but because the data tells me what the system is actually being asked to carry. Where does it live? Who can access it? What is authoritative? What has to be interpreted by a person before it becomes useful? What is missing? What only makes sense because someone inside the company remembers the history? I also want a quick sense of the humans' actual experience with AI. When I train or speak inside organizations, one of the fastest ways for me to gauge the room is to ask how many hours people have really spent working with these systems. It tells me much more than asking whether they are 'good at AI,' because lived operating experience changes what people notice, what they assume, and what they are capable of carrying themselves.
The workflow may be compensating for missing context
One of the easiest ways to create an overcomplicated AI workflow is to ask the workflow to reconstruct context that the system was never given. You see it when prompts grow into walls because every new failure produces another instruction. You see it when agents repeatedly search for information people already know but have never made durable. You see it when one system summarizes another system's output, then a third system summarizes the summary, because the original reasoning cannot travel with the work. You see it when a human repeatedly steps in at the same point and nobody asks what that person knows that the system does not.
That recurring human rescue is one of the strongest signals in an audit. I want to know what the human notices, what they remember, what they refuse, what they correct, and what decision they are making that the workflow cannot make without them. If the same person keeps fixing the same category of error, they may be performing an undocumented cognitive function inside what everyone else believes is an automated process. The rescue itself is data. It tells you where the system's understanding ends and the human's begins.
This is part of what I think of as the human-AI runtime: not merely the software runtime, but the actual operating behavior between the person and the machine. Where does the human have to carry context? Where does the model create unnecessary cognitive load? Where does the system generate more structure than the task deserves? Where does a useful framework turn into five protocols for managing the protocol? AI is extraordinarily capable of extending patterns, which is useful until the pattern being extended is bureaucracy. A system can produce a beautifully intricate mechanism that is technically impressive and practically pointless because nobody stopped to ask whether the extra mechanism was needed.
That dynamic is easy to miss when you are close to the work. New AI systems invite exploration, and exploration creates branches. You follow a promising thread, build something around it, discover another requirement and create another layer. Sometimes that is how the real architecture reveals itself. Sometimes you are just traveling in a circle with excellent scenery. Experience helps because after enough of those journeys you begin to recognize the difference, but the discipline is the same: periodically stop generating and ask what the system is actually doing for the work.
A diagram shows movement, not whether the decision is right
Workflow design feels reassuring because it can be drawn. You can see the trigger, the actions, the conditions, the outputs, the integrations and the handoffs. That visibility is valuable, but it can create a false sense that the system itself has been understood. A workflow diagram can show where a decision occurs without telling you whether it is the right decision, whether the information reaching it is sufficient, whether the rule still applies, or whether a human has been silently correcting it for six months.
Before I start rearranging boxes, I want to break the system into pieces that can be examined independently. What is data? What is memory? What is a decision? What is human judgment? What is an automated action? What is ownership? What is merely a tool choice? The point is not to force every organization into a taxonomy. The point is to stop different kinds of problems from hiding inside one box labeled 'workflow.' Once those pieces are visible, you can change one without accidentally rebuilding everything around it.
Memory is an especially common example. A workflow can function exactly as designed and still feel unreliable because the system does not remember enough from one run to the next. The user experiences the failure as repetition: I already told it that. We already decided this. Why are we rebuilding the same context? The instinct is often to add another step, another retrieval call or another prompt. Sometimes that is appropriate. Sometimes nobody has made the more basic decision about what deserves to persist, where it should live, who can update it, and when it should stop being trusted. A workflow moves work. Memory gives that work continuity. They are connected, but they are not the same function.
Ownership creates the same confusion. Workflow diagrams are usually good at showing who performs a task and much worse at showing who owns the intelligence behind it. Who decides whether a source is trustworthy? Who updates the rule when conditions change? Who can explain why a human override happened? Who notices when an automated decision has drifted away from the business reality it was designed to represent? A system does not inherit accountability simply because it inherited execution. If nobody can reconstruct why the process behaves the way it does, the workflow may be functioning while organizational understanding is disappearing underneath it.
When I would actually audit the workflow itself
Sometimes the workflow really is the problem. The logic may be unnecessarily complicated. Too many tools may be involved. A repetitive human step may have become stable enough to automate. A handoff may be poorly designed, a trigger may be wrong, the tool may be badly chosen, or a process that once made sense may simply have outlived its usefulness. Once the underlying context, memory, decision rights and ownership are clear enough, workflow optimization can create enormous value because you are finally improving movement rather than using movement to compensate for confusion.
That is when I want to simplify. Remove unnecessary steps. Reduce duplicated computation. Stop moving the same information through four systems when two would do. Automate what has genuinely become stable. Keep human judgment where judgment is still doing meaningful work. Make the route easier to understand so that someone other than the person who built it can explain what happens and why.
The distinction matters because a workflow should carry a coherent process. It should not become the place where every unresolved question gets buried. If the workflow is repeatedly compensating for missing data, missing context, missing memory, unclear ownership or a human behavior nobody designed for, optimizing it first may simply make the unresolved problem travel faster.
Audit the intelligence before you optimize the movement
This is what I am looking for in a Human-AI Orientation when someone arrives saying a workflow is broken. I am not automatically looking for a better workflow. I want to understand what is moving through the current one, what the system is expected to know, how the humans are actually using it, where intelligence disappears, what gets reconstructed, which decisions are explicit and which ones exist only because someone remembers what to do.
Sometimes the answer is a workflow redesign. Sometimes it is memory. Sometimes the team needs a clearer operating model for the AI it already has. Sometimes the data needs work. Sometimes a decision that has been living inside one person's head needs to become visible before anything should be automated. Sometimes the system is carrying far more machinery than the task deserves. And sometimes the workflow everyone wants to replace is functioning perfectly; it is simply carrying a problem that began somewhere else.
Before you optimize the movement, make sure you understand what is moving.
HUMAN-AI ORIENTATION
Is the workflow actually the problem?
Bring the workflow that feels broken. We trace what is moving through it, where intelligence disappears, what gets reconstructed by humans, and whether the next intervention belongs in the route, the memory, the data, or the operating model.
€950 · 2.5 hours · nothing is sold in the room
FIELD CONNECTION • hUMAN-ai sYSTEMS
Go deeper only when the thought needs another room.
Authentic AI Marketing opens the commercial door. Human-AI Systems holds the deeper architecture, artifacts, and system thinking behind the work.
DEEPER SYSTEMS. RICHER CONTEXT.