Human Review Becomes a Queue: Design for the Day It Fills Up

Start with simple arithmetic. For a chosen time window, estimate how many review requests arrive and how many focused review minutes each class needs. Compare that demand with the minutes reviewers can actually give the queue. This estimate does not predict exact wait times. It shows when the queue cannot clear at the planned rate.

For a long-running queue, a classic result can connect three averages. John D. C. Little’s 1961 proof states L = λW: the mean number of requests in a defined system equals the mean arrival rate multiplied by the mean time each request spends there. His result assumes finite means, strictly stationary corresponding stochastic processes, and an arrival process with a nonzero mean that is metrically transitive. The INFORMS paper abstract states those assumptions. Define the system boundary and request population first. L, λ, and W must describe the same class of requests over the same period, from arrival to decision or closure.

Little’s law gives a long-run average. It does not predict a one-hour reviewer outage, a burst, or the wait of a particular request. It does not rank risk classes or set a safe deadline. Keep short outage drills separate from any long-run average that may fit the paper’s conditions.

Suppose, only for illustration, one reviewer can give a queue 30 focused minutes during a half-hour period. A request takes about eight minutes to inspect and record. That reviewer can finish about three such requests in that window. If seven arrive, about four remain waiting or under review. The numbers are synthetic. Real work varies, and an average can hide long or difficult cases.

Count useful capacity, not headcount. A reviewer listed on a rota may be handling a customer call, incident, break, or another approval. A second reviewer may not have the context or authority to decide the case. Record the time people can give, the kinds of requests they can decide, and who covers an absence.

The NIST AI Risk Management Framework (AI RMF 1.0) says that risk tolerance depends on the context and use case (§1.2.2, pp. 7–8). It also warns that poor risk prioritization can waste scarce resources, and says policies and resources should reflect assessed risk and potential impact (§1.2.3, pp. 7–8). Those sections support a risk-based allocation question. They do not give a queue formula, universal review percentage, or safe timeout.

The same framework notes that human roles and responsibilities need clear definition, and that some systems require human oversight while others may not (Appendix C, pp. 40–41). It says human-AI interaction results vary. That is a reason to define the human’s actual job in this workflow. “A person is in the loop” does not explain what the person may decide, what evidence they need, or what the system does while they are unavailable.

Set an overload trigger before the queue fills

The workflow owner should set a maximum safe wait for each decision class from the actual policy or business deadline. Record arrival, first-review, and disposition times from a source the operator can inspect. Use the age of the oldest unresolved request in each class as the queue metric. The locally approved maximum wait becomes its threshold; there is no universal safe number.

Name the operator who watches that threshold. If the oldest request reaches its limit, that operator should stop new actions that require the same approval and route urgent work to an explicitly authorized backup. Keep each open request visible until a person decides, the requester withdraws it, or the policy says it expires. A threshold breach is an action trigger, not permission to silently reduce the queue.

Separate urgency from arrival order

First-in, first-out order is easy to understand. It can be fair for requests that carry similar consequences and deadlines. It is a poor rule when one request blocks a reversible draft and another request may trigger an external, difficult-to-reverse action.

Define a small number of work classes from the decision’s consequences. Do not classify work by a score unless the score has a clear owner, evidence, and a tested role in the decision. The following classes are design proposals, not a universal policy:

A class must describe an observable decision, not a vague feeling of urgency. Record the action, what could happen if it waits, whether it can be undone, who can decide, and the time at which the request should stop being acted on. Domain rules may require different handling. A system with a legal, contractual, or safety deadline must use the rule that governs that work.

For incident-response work, NIST SP 800-61 Rev. 3 gives a narrower planning example. Its ID.IM-04 profile says incident-response and related operational plans should identify the resources and management support needed to carry them out (Table 2, §3, pp. 20–21). This guidance concerns cybersecurity incident response. It does not set capacity rules for customer-support, finance, or other approval queues.

Do not let a full queue silently widen authority. If policy requires approval before an action, an empty seat does not count as approval. If a different task was already authorized to run without human review, keep that separate from requests waiting for a required decision. Exhausted capacity changes availability; it does not create permission.

Give waiting work an honest deadline

Every queued request needs a visible state and a next step. “Pending review” should identify who owns the decision and when the system will check again. “Escalated” should name the route and the reason. “Expired” should say that the request can no longer proceed under its original context. “Returned” should explain what the requester needs to update.

A timeout is a time limit on waiting. It is not a decision about the request. When the limit passes, the workflow should follow a written rule: keep waiting and notify an owner, route to an authorized backup, return the request, or expire it. The correct choice depends on the action and the governing policy. Automatic approval is not a neutral timeout behavior.

A review may also become stale before its deadline. The request’s price, recipient, customer status, evidence, or policy might change while it waits. Set out which changes require a fresh decision. A reviewer should not approve yesterday’s context by opening a notification today.

If review capacity disappears for an hour, a useful system can still answer four questions:

1. Which actions must remain stopped until someone decides?
2. Which requests still have valid authority and enough context to wait?
3. Which requests will lose their deadline or value before a reviewer returns?
4. Who receives the urgent cases, and what happens if that person also cannot respond?

When there is no valid reviewer, say so. Keep the action stopped if the required approval has not arrived. Send time-sensitive work to a named backup only if that person has authority and context. Otherwise, tell the requester that the decision is delayed and give the next check time. Do not invent a second approval path while the first one is overloaded.

Measure service without hiding missed work

Track arrivals, completed decisions, waiting time, and requests that leave without a decision. Break these counts down by work class and reviewer role. A single average can hide a small number of old requests with serious consequences.

Review at least these measures:

• Incoming requests per interval, including short bursts.
• Focused reviewer minutes available by decision class.
• Review time by class, including unusually long cases.
• Number and age of requests still waiting.
• Requests that were returned, canceled, expired, or escalated.
• Requests acted on without the required decision.
• Decisions that needed correction because the waiting context changed.

A shorter queue is not proof that oversight improved. The queue can shrink because reviewers decided faster. It can also shrink because the system rejected more requests, stopped flagging exceptions, or moved work to an untracked channel. Check what happened to every request that left the queue. Compare completed work with expired, canceled, suppressed, and bypassed work.

The strongest counterargument is that capacity planning can become a reason to ration attention. A team may label difficult cases “low priority,” set deadlines that nobody monitors, or automate decisions so the dashboard looks healthier. This can leave a short queue and more unresolved harm. The answer is not to preserve every request forever. It is to define who can defer, reject, or change a required review rule, and to keep a record of the requests that did not receive a decision.

The causal claim also has limits. A queue worksheet may expose overload, but it cannot prove that the reviewer’s decision is correct or that a proposed backup will respond. A smaller review burden may help a team focus. It may also reduce practice and make a person less prepared for a rare, difficult case. Treat that effect as a question to monitor, not a proven result. NIST’s human-AI discussion describes variable interaction outcomes; it does not measure operator readiness under a review queue.

Run an overload drill before a real spike

Use the worksheet below with one workflow. Fill it with observed requests and measured review time when those records exist. Mark estimates as estimates. Then simulate one hour with no primary reviewer.

Ask a person outside the workflow to challenge the rules. Can they tell what waits, what expires, and what gets escalated? Does any request become authorized only because its timeout passed? Can the team find every request that left the queue without approval? If the answers depend on a manager’s memory, the queue policy needs a named owner and a recorded rule.

The worksheet is a proposed planning aid, not an official standard or validated queueing model. It does not predict performance, guarantee timely review, or replace legal, safety, security, or sector-specific requirements.

A useful first Armalo Layer reference is the technical mapping for AI agent trust infrastructure. That published page proposes questions about authority, evidence, review, and consequences. It is an editorial mapping, not a standard or a claim that a particular product implements these controls. Use the capacity worksheet on its own if the mapping adds no useful decision.