In “How long does it take?” the previous post in my series on Little’s Law, I focus on the importance of a metric called residence time. It’s a fundamental concept you need to know about in order to understand how Little’s Law and flow metrics really work under the hood.
Also, it has huge value as a day-to-day operational metric in it’s own right.
But the concept is probably unfamiliar to most people — even those well-versed in the principles of flow and flow metrics. Recently, a number of people have asked me if residence time is just another name for concepts they already know: cycle time, lead time, flow time, and so on.
Yet another common question has been whether this term refers to the time a process a spends in a specific work flow state like development or QA - this is also called cycle time or flow time in various parts of our community1.
It isn’t.
Residence time is a fundamentally different concept, and it is worth understanding what it is. In this post, I’ll explain the relationship between residence time and the flow metrics we already know very well.
The standard flow metrics
Let’s look at the familiar metrics we use today—lead time, cycle time, flow time etc. All are elapsed-time metrics. Each measures the time between two events that mark the beginning and end of some process in a system:
Project approval → delivery to customers
Customer request → fulfillment
Feature is available for acceptance testing → Feature is accepted
Defect is logged → fix deployed to production
PR is opened → PR is merged
Different groups use different name like lead time, flow time, cycle time etc for these metrics, and also the same terms with different meanings: there is a fair amount of inconsistency in the way we use these terms today in the industry.
But whether we call it a lead time, cycle time or flow time, the important thing is not the specific name: it’s the definition of the start and end events and their domain semantics.
Also, what matters is that they are measuring different underlying processes.
In classical queueing theory, a queue serves as an abstraction for many such processes2. The events are arrivals and departures at a queue, and the elapsed time between arrival and departure is generically called sojourn time.
“Sojourn” is a neutral term: it simply means time for an item to “travel across” the queue, or to be processed. All the elapsed time metrics we see in practice are all just different flavors of sojourn time. So we’ll adopt sojourn time as our generic term for such metrics.
The average sojourn time for processing a set of items is important because is the W in Little’s Law3:
But residence time is not a sojourn time. It’s a completely different concept.
The importance of observation windows
Sojourn time is always item-relative. You measure the elapsed time for a given item from start event to end event.
You can compute an average of sojourn times of a set of items easily if you have a set of start and end events for those items. But this implies you can observe the whole journey of all the items and know when they start and finish.
In reality, we often observe processes mid-stream. Some items are already in progress when we start observing, or haven’t yet finished by the time we stop. Their start or end times might be unknown. Other items start or finish during the window. Other start and finish during the window.
This matters because when we measure the three quantities L, λ and W in Little’s Law they have to be measured over the same observation window before you can even consider applying any of the machinery of the law.
What kind of average sojourn do you get if you can’t observe the whole journey and are simply observing the process over some arbitrary finite window of time?
The standard approach is to measure average sojourn time for the items that end in the observation window. Pretty much all the standard definitions of lead time, cycle time, flow time etc in use today assume this. In Fig 2 these would the average of the green bars, but these averages are not being measured across W2 - you need to expand the observation window to capture all the events involved.
We discussed the problems this creates in our post last week
How long does it take?
This is the first in a series of posts exploring specific applications of sample path analysis in complex adaptive systems. We introduced this topic in broad strokes in Little’s Law in Complex Adaptive Systems.
The problems raised here are not theoretical problems. The reason that the three flow metrics as we measure them today don’t obey Little’s Law is because they are not being measured across a consistent observation window4.
Enter residence time
Residence time is a new kind of elapsed time metric that can be applied to any finite observation window. Residence time is not item-relative but observer-window-relative. If I observe a process over the window [0,T] residence time measures the portion of each item’s elapsed time that overlaps with that window.
In Fig 3 these are the durations shown in green. Because residence time is “clipped” by the observation window, it’s always less than or equal to the full sojourn time for any given item5.
We can define two different kinds of elapsed time averages for given a set of items.
Their average sojourn time across some set of items (over some large unknown horizon) that includes all their start and end points: W,
Their average residence time in a finite observation window [0,T] as w(T).
Now we have two averages which are both well-defined but are potentially defined over different observation windows.
Given any sojourn time, we can define a related observation-window relative residence time.
Now the question becomes are there any observation windows where you get the same average value when you measure both quantities simultaneously6.
Residence time is a key measurement in sample path proofs of Little’s Law, so measuring processes using residence time mirrors how those mathematical proofs show how and when Little’s Law holds.
The relationship between a sojourn time and a residence time
As we discussed in The Many Faces of Little’s Law, average residence time, w(T) plugs directly into Little’s Law at any finite observation window.
This holds unconditionally, for any observation window, as long as L(T), Λ(T), and w(T) are also measured over the same window. This means that residence time is always well defined over any finite window, and we can also define a corresponding well-defined version of arrival rate and number in the system such that the finite version of the law holds for any finite observation window.
This is a big difference between residence time and sojourn time.
Little’s Law says that if w(T) converges to a finite limit over sufficiently long observation window T, then that limit value W is also the average sojourn time of the items over those sufficiently long windows.
More precisely, sample path analysis says the relationship between average residence time and average sojourn time over a long enough observation window is as follows.
If a process is convergent, the average residence time w(T) approaches a finite limit as T grows. Over that window, average sojourn time W also converges to the same limit. ie in a convergent process, the average residence time equals the average sojourn time over a sufficient large observation window. That’s when we can replace w(T) with W in the Little’s Law formula say Little’s Law “applies.” to the process.
Fig 5: Convergent metastable process If the two values converge to a single value and stay there, then the process is stationary and we can simply use that limit value as the average sojourn time W in Little’s Law.
But more likely, you’ll get processes that behave like the one in Fig 5. These are convergent but the values of w(T) and W(T) come together and drift apart as the process evolves, but as long as they stay within a tolerance of each other, we can use residence time as a very accurate proxy for sojourn time.
This kind of process is called metastable, and in a complex system this is what we should expect residence times and sojourn times to look like most of the time.
If a process is divergent, the averages never converge no matter how long you observe the process. So you can never apply L= λW to that process. But you can still apply the finite version involving residence time at all times! And measuring residence time and sojourn time together, makes this divergence visible early.
This is why residence time is by far, the operationally more useful metric. Sojourn time is useful to report metrics after the fact, but residence time is what lets you really understand the process dynamics from moment to moment.
Operator vs participant view
There is another way to look at the relationship between sojourn time and residence time.
Sojourn time is the participant’s view—how long the customer experienced in the process.
Residence time is the operator’s view—how much of that elapsed time was visible within an operational window.
When these two align on average, we say the flow metrics are coherent.
The bottom line
Residence time is more than another name for cycle time. It’s a complementary, but necessary lens to measure any given elapsed time metric reliably when processes sojourn times are much larger than the standard measurement window and/or the processes move continuously in and out equilibrium - as most software development processes do.
Sojourn time tells you how long an item spent from start to finish. Residence time tells you how long that item was “present” in your observation window whenever you observe it over any period of time.
So a good way to think about it is as shown in Fig 7:
For any sojourn time metric you can define for a flow process, there is a corresponding residence time metric. The relationship between average residence time and average sojourn time over sufficiently large observation window let us determine how close or how far away the process is from a equilibrium state per Little’s Law.
This applies to lead times, cycle times, flow times etc - any sojourn time metric you can define.
It is also important to remember that convergence/divergence is not a system property but a process property. In applying sample path analysis, we think of lead time, cycle time etc as representing different processes at different granularities (potentially nested within each other).
Each of these processes may be convergent or divergent independently, and it is possible that the they impact each other’s convergence/divergence. Sample path analysis will not all explain these interactions but will show the impact of those interactions via convergence/divergence of each process as measured over a common observation window.
Thus the first step to understanding the dynamics of these processes is to understand whether each of them converges or diverges.
Then we can start asking why they behave the way they do.
We discussed how this type of causal reasoning might work in
The Causal Arrow in Little's Law
Most people think Little’s Law is about stability. But what it says about causality is even more important.Thanks for reading The Polaris Flow Dispatch! Subscribe for free to receive new posts and support my work.
The Causal Arrow in Little’s Law.
Check that post out for more on how we can use these metrics to reason about systems.
Let me know in the comments if this is still confusing or whether I can help make this clearer.
The term residence time is used in performance analysis of computer systems to refer to the time an item spends in a particular queue in a network model of a computer system. Some operations books also use this to define the time spent at a specific service station. In that sense it could be said that we are inventing a new meaning for this term here. But we have two counter-arguments.
First, the term residence time is not commonly used in any of these senses in the software development flow community - we use cycle time or flow time for this, and this is already confusing as it is.
Second the most important sense in which residence time is used in those other operations management contexts is that they assume that one can unconditionally apply Littles Law to a single node/station using residence time. This assumption does not hold for flow time or cycle time in software, but the definition of residence time as the observation-window relative elapsed time preserves this semantics.
So the key thing about residence time that differentiates it from all the other operational elapsed time metrics is the fact that we can always plug it into the finite version of Little’s Law and in combination with the cumulative arrival rate, apply Little’s Law to reason about flow at all points in time.
In particular, processes where we want to understand the ratio of time spent waiting to time spent processing - queuing theory is the study of waiting in processes.
Dr. Little himself called W “waiting time” in his writing, but queuing theorists needed to distinguish between time “waiting for service” and “in service”, so they reserved sojourn time for W and queue time for “waiting for service”. So this confusion around naming these things goes back decades - not a new phenomenon. Bottom line is define start and end events that demarcate initial and terminal conditions for your process and call the time between these sojourn times generically. You can rename them whatever you prefer in your context, provided you do it consistently in your context.
The standard measurement of average WIP also suffers from similar problems - but in general the problem of verifying whether Little’s Law applies is entirely a problem of figuring out how to construct a consistent measurement window over which we can measure all three parameters of Little’s Law in a well-defined manner.
Although average residence time for a finite window can be greater than the average sojourn time for the items that finished in that window. This may seem paradoxical but it it’s really not. Whenever we look at an item that has not completed, its residence time is less than its sojourn time - but some of that sojourn time lives in the future. In any finite observation window, the residence times of incomplete items may be much bigger than the sojourn times of the completed items in that window. So the averages residence time in a window can (and often is) larger than the average sojourn time for that window.
The non-obvious part here is that the averages may match even if not all the items being measured in both sets match up perfectly, but as long as they are being measured over the same observation window, the discrepancies become irrelevant over sufficiently long windows - this is what Little’s Law allows us to claim.