The gap between what is recorded and what happened
Let me start with something that might sound strange coming from someone who spends a lot of time in data: I have never trusted a productivity number I could not explain.
That is not scepticism for its own sake. It is the product of standing next to enough operations to know the gap that routinely exists between what a system records and what actually happened. In a warehouse or store-based fulfilment environment, that gap can be enormous and the ways it hides in plain sight are worth understanding properly.
Fulfilment is rich in data, and that is part of the problem
Modern fulfilment operations generate an extraordinary volume of events. Every pick, scan, dispatch, exception and status transition can in principle be captured. With the right data architecture, you can zoom out to a network-level view or zoom in to a single operator's session within the same hour. That potential is real and for the right kind of analysis it is genuinely powerful.
AI inference is increasingly becoming part of this picture and when the underlying data is clean it acts as a genuine multiplier: surfacing patterns across thousands of sessions that no analyst could detect manually, flagging anomalies in real time and drawing correlations between variables that would take weeks to identify through conventional reporting. But as any number of commentators have noted, AI does not resolve the signal problem. It amplifies whatever signal exists. Feed it polluted data and you will get faster, more confidently wrong answers. The architecture that enables great AI-assisted analysis is the same one that exposes all the problems described below.
The catch is that all of this event richness exists in relation to a system and the system only knows what it is told. When operators work around the system, absorb exceptions off-screen or simply get pulled in a direction the software was not designed to track, that activity becomes invisible. The data does not show a gap. It just shows whatever the system recorded before things went sideways and after they resumed. The gap in between often contains the most operationally important information you have.
Five ways the signal gets polluted
In my experience, there are five recurring mechanisms that degrade the reliability of performance data in fulfilment operations. None of them are exotic, but all of them are consistently underestimated.
Untracked task absorption
An operator steps away mid-session to help with something that is not in the system: a stock query, a labelling issue, a customer in an aisle asking for assistance. The pick clock keeps running and their rate tanks. Without context, data surfaces this as underperformance.
Process divergence
Operators develop workarounds. Sometimes because the system-driven process does not suit the physical environment, sometimes because a shortcut is genuinely faster. Either way, the system is recording inputs to a process that is no longer being followed.
Hidden task complexity
Two operators are running the same task type. One has drawn a favourable run of activities, the other is working through a set that is slower and more demanding. The top-line numbers do not reflect that difference, so what looks like a performance gap is often just an uneven task distribution that the data has no way of flagging.
Off-system exception handling
When a pick cannot be fulfilled due to a wrong size, damaged item or location mismatch, the exception handling can often happen outside the system. The operator makes a judgment call, moves on and the data records a clean completion against what was actually a partial failure.
KPI gaming
A task is assigned to one operator but two are actively working it, with a second person assisting without being logged against the session. The individual's activity rate looks strong but what the data does not capture is that the team's collective output has been diluted to generate that number.
Individually, any one of these is manageable. Together, and in an operation running thousands of individual discrete events a day, they compound into a performance picture that can be misleading in both directions. You will find operators who look like underperformers and are actually your most motivated employees, absorbing the most complexity. You will find results that look strong on paper and are being held together by things that are not sustainable.
When data and reality diverge
Having run large-scale fulfilment operations, I have seen this play out repeatedly. Teams of over a hundred people, processing tens of thousands of orders a day, with data infrastructure that looked solid on paper: pick rates, throughput by zone, order cycle time, exception rates, the standard suite.
What that data could not tell us, reliably, was why certain sessions underperformed. It could surface the outcome but it was often not enough to explain the root cause. And the causes mattered enormously, because the right response to an operator struggling with unfamiliar zone complexity is completely different from the right response to an operator who has quietly developed a habit that bypasses an established process.
The hard lessons came from treating a data signal as a people problem when it was actually an environmental problem, and from treating what looked like a process compliance issue when it was operators compensating for a system limitation that had not yet been fixed. In resource-constrained environments where the capacity to implement improvements and counter-measures is always limited, that kind of misdiagnosis is costly, both operationally and in terms of what got prioritised to build next.
The correction was never to build a better dashboard. It was to get better at reading the gap between what the data said and what was happening on the floor.
When a company hits its number, the ones who do not celebrate are usually the ones who know how much sticky tape was holding the result together.
The most valuable feedback does not always come from the most forthcoming people
There is a dynamic in most operations that I have come to watch for deliberately. When a big result is hit, a record dispatch day, a productivity milestone, a cost target, there are usually two groups in the room. One group is celebrating and the other is quieter.
The quieter ones are almost always the people closest to execution. They know exactly how the number was achieved and they know which parts of it are repeatable and which parts involved the team heroically compensating for something broken. They are the ones who know a particular zone was manually spot-checked because the inventory data was unreliable. They know a senior operator went above and beyond their paid hours to help provide their team relief. They know the number was real, and they also know it was not quite what it appeared to be.
At the surface level, this can be interpreted as cynicism and detraction that could be the first drops of poison in the team's cultural well, but at its core it is actually operational literacy and it is one of the most undervalued sources of diagnostic intelligence available to any operations leader.
The people doing the work usually have a precise mental model of what is actually limiting performance. They have felt the friction of every broken process, every inventory discrepancy, every UX failure in an interface. They do not always frame it in the language of root cause analysis but the signal is there. The job is to create the conditions where it can actually surface and to know how to draw it out when it does not come forward on its own.
What good data practice actually looks like
None of this is an argument against data. A fulfilment operation running blind on anecdote alone is not the answer. The point is that data and direct operational engagement are not alternatives to each other and in fact they are partners in a diagnostic process, and conflating one with the other will cost you accuracy in both directions.
Use data to identify where to look, not as the final verdict
A pick rate anomaly is a starting point for a conversation, not a performance finding in its own right. An exception rate spike is a prompt to go and watch what is happening at the pick face and ask qualifying questions, not a reason to update a process document.
Be deliberate about which metrics you trust
If your system cannot distinguish between a session slowed by inventory complexity and a session slowed by operator pace, you need to be honest about what the metric is actually telling you and what assumptions you are baking in when you act on it. Not all metrics are equally reliable and treating them as though they are leads to consistently wrong conclusions.
Build the conditions for honest feedback
The elephant in the room is obvious: it means building enough trust with your frontline team that the quiet ones in the room have the space, safety and authority to tell you what they know. This is some of the most valuable diagnostic infrastructure you can invest in building.
Dashboards and reports are the right starting point. They give you scale and pattern recognition that field observation alone cannot provide. But the operations that improve most reliably over time are the ones where leaders know how to read past the number to what the number is actually representing and where the people closest to the work feel safe enough to say when the sticky tape is all that is holding things together. That conversation is where the real diagnostic work happens.