A central tool in managing the team’s work is the Kanban board. The team owns their Kanban board and everything on it. It is there to visualise the team’s work, to provide a sense of control and create a sense of flow. So for example, if there is a bottleneck in testing, it’s there for the whole team to see and the team can collectively prioritise eliminating it.
In short, the Kanban board is a powerful tool for creating a sense of collective ownership in the team. But there are some ground rules that apply otherwise this effect is lost:
The tickets on the board represent the team’s work and what can be delivered by the team. The board must not contain work done by external parties (i.e. anyone not in the team).
The team must ruthlessly prioritise, rework or eliminate stuck tickets. Even one exception can lead to more exceptions, and before long flow is lost.
Bottlenecks mean “all hands on deck” to solve the problem and regain flow. Plus, it makes no sense to keep pushing more work into an already long queue. WIP limits can be applied to highlight bottlenecks, but usually common sense prevails.
Following these ground rules means that any member of the team, at a glance, can see the status of the team’s flow, and know that if there is a problem with the flow then it is a) within the power of the team to solve that problem, and b) that is more important than pulling the next ticket from Todo.
If the team do not follow these three ground rules, then it will be hard to create a sense of ownership when there are a whole bunch of exceptions to the rules that erode the team’s sense of control. It becomes someone else’s problem.
Put another way, if the team use Kanban properly then it will create a sense of flow that everyone in the team can identify with and which transcends the individual roles and specialist competencies that a cross-functional team naturally has. But fail to do this, and Kanban is just another messy backlog with unclear priorities that the team eventually stops paying attention to. Then it just becomes easier to pull the next ticket from Todo instead.
In an earlier post I showed how a team can get started with Kanban by visualising and organising their work using post-its, and concluded with a look at managing WIP (work-in-process). In this article I want to broaden the discussion to look at some of the other features of Kanban that can help the team manage their work.
WIP is a problem for most teams, especially when release cycles are long (1 month or longer), but there are other factors that can affect flow and that can also contribute to a lot of WIP.
Categorising work
The Kanban board represents what the team are expected to deliver; therefore the work must be well-defined with clear acceptance criteria (as distinct from the discovery phase where ideas are still being explored and the scope is unclear). There is usually a week’s worth (or a sprint’s worth) of tickets ready to be pulled from the “To do” column.
Something I encounter frequently in teams is where developers pre-assign tickets to themselves that they know they will work on, but haven’t started yet. When I asked why they do this I learned that they want to know what was in their pipeline, what was coming up next. So, assigning the issue to themselves was their way of labelling the issue as being part of their domain, e.g. web development, or back-end development.
We talked about other ways of tagging the issue; Jira provides the Components field and Labels field to help categorise issues, and on the Kanban board it is possible to create “Quick filters” that will filter on these fields. For instance, the front-end developers were able to create a filter to that only showed “web” components. This turned out to be a great idea, something the whole team could use to find certain types of work on the board.
Using filters in this way also has the advantage of highlighting all of the work related to a specific component or label. And whereas a ticket can only be assigned to one person, there can be many components and labels set on a ticket, allowing it to be filtered in different ways.
Identifying bottlenecks
Even if the team manage to get WIP under control, it will still happen that bottlenecks will occur. After all, every piece of software development is unique and will have its own unique challenges with delivering it.
As a coach, I can talk to the team about what appear to be bottlenecks in the flow – with Kanban the problem is there for all to see. If the bottleneck is in testing, the developers may not see it as their problem to solve, until I point out that continuing to build more stuff that needs to be tested just adds to the bottleneck, and they will be waiting that much longer for the code to reach production. This usually gets the team thinking.
There are two things the developers can do in this case, either help out with testing or, do some technical improvement that does not require QA resources. In the best case, the developers use their skills to help automate some of the testing, a win-win for the whole team.
Flagging blockers
As I stated at the beginning, when a ticket is added to the board it should have a clear Definition of Done and clear acceptance criteria. But it will still happen that there is hidden complexity in the ticket and work on that ticket stops until the scope is clarified. What I see happen quite often is that the team moves stuck tickets to a separate “On hold/Blocked” column. This breaks the flow of work, now the team have to remember where the ticket was blocked, was it during development? Testing?
A better approach is to flag the ticket in the column where it is stuck. For instance, Jira provides a convenient “Add flag” option to highlight tickets that are stuck without changing its status. When the blockage is removed, the flag can be removed and work continues from where it left off. The ticket is also visually striking when is flagged, it demands attention, which it should.
Hidden complexity is just one reason why a ticket cannot move forward. There are other reasons, but they are all a result of the same thing: external dependencies. For instance, hidden complexity means going back to discovery with stakeholders, and stakeholders are not part of the team, they are an external dependency. This is not a bad thing, but it is important that the team understand their domain of control, and what can slow them down.
Understanding your domain of control
The essence of the Kanban board is that when a ticket is added to, the team can say “Yes, we can deliver that”. The work is clearly defined, but more importantly all the resources needed to deliver that piece of work are in the team: designers, developers, testers, devops, etc.
If that is not the case, then the team are relying on third parties (external to the team and/or external to the company) to get the feature into production. And every time the team need help from a third-party to move the ticket forward, they are essentially blocked because they have no control over the priorities a third-party has. So while some people work closely with the team, they are still not part of the team and so they block the team because they answer to another master with other priorities.
If a lot of tickets are being blocked, the tendency is to start working on something else, rather than solving the blockers, which just adds to WIP. Instead the team must relentlessly focus on removing the blockers, whether it means adding the necessary resources to the team, doing more in-depth discovery, or doing a more radical re-evaluation of the team purpose.
Managing risk
Finally, the Kanban board can be used to manage risk. In a nutshell, a lot of WIP means longer lead times, which increases the risk that priorities change before the feature is shipped, with the result that features are abandoned halfway through development which is an expensive way to run a business.
Regardless of what development process and release cycle the team uses, if there are a lot of tickets on the board it means that when something new is added, it will have to wait until all the work already on the board goes into production before it can be shipped.
Since the work on the board is supposed to be well-defined, it should be possible to make some ballpark estimate how long it will take to deliver everything on the board. Let’s say, your estimate is 3 months. Ask yourself, how confident are you that you will be able to deliver everything on the board before priorities change?
I recently read an article about how to help someone get back to work after a long absence, perhaps due to illness or burnout. There was lots of good advice, such as keeping colleagues informed about adjustments to working hours and limiting responsibilities among others things. But what struck me was how a lot of the advice reminded me of the Agile way of working:
create clearly defined tasks
allow space to work on one thing at a time
provide support with prioritising
do not set short deadlines
set a clear plan for the week and a review at the end of the week
ensure delegation of tasks is done via a single channel
This could be from the Kanban playbook. To put it in Agile terms:
Tasks should have a clear definition of done
Developers should pull tasks, not have them pre-assigned
The Product Owner prioritises all work
Focus on outcomes not deadlines
Set clear goals and use daily stand-ups to ensure progress
Nobody outside the team can assign work to the developers
So you could say that creating flow does not just improve the team’s efficiency, it also contributes to the continued well-being of your employees.
The Kanban board is used to visualise the team’s work. This is usually a mix of Bugs, Tasks and Stories. Good stories should follow the INVEST criteria. If the team are using Jira, then it also allows them to create sub-tasks for Tasks and Stories. Sub-tasks are a useful way for the developers to create a “Todo” list for the implementation, e.g. “setup database”, “create service”, etc. without exposing the gory details to the rest of team.
Whenever the team is looking at the flow of value across the board, these implementation details are usually not interesting, and that is why sub-tasks are usually not shown on the board. However, when a developer is discussing their current progress (e.g. during standups), this information can be a useful recall aid. This is especially true if the team are creating vertical stories which usually requires multiple developers (front-end and back-end) to work on the, and therefore the story cannot (should not) be assigned to any one person. Instead, it is the sub-tasks that provide context.
A Jira Kanban board can also be filtered per user; so if sub-tasks are shown on the board, then the team can apply the user filter to quickly see the sum of what any one developer is working on: sub-tasks, tasks, stories, etc.
Displaying sub-tasks on the board is easy to configure, but there are some other changes that the team might need to make as well. For instance, how to hide “Done” sub-tasks without hiding stories that are due for release. I will cover each of these in the following sections.
Displaying sub-tasks
Every Kanban has a Filter Query that controls which issues are displayed. If only certain issue types are displayed, then the filter must be updated to also include sub-tasks. In that case, go to the Board settings, General and edit the Filter Query to include “All Sub-Task Issue Types”. For example:
project = "ACME" AND issuetype in (subTaskIssueTypes(), Story) ORDER BY Rank ASC
If the sub-tasks are using a different workflow, then it is presumably a simpler workflow than the Stories they are a part of. Just make sure that any unique sub-task workflow states are added to the board columns. This can be configured under Board settings, Columns.
Immediately, the team will be able to see all sub-tasks on the board and can filter them per user by clicking on the avatars at the top of the board. The next step is to create a toggle to hide/unhide sub-tasks.
Toggling sub-task display
Displaying sub-tasks inevitably leads to a lot of clutter on the board. It is also important that the team can maintain focus on the flow of Stories and not just sub-tasks. To facilitate this the team want to be able to hide sub-tasks at will.
Under Board Settings, Quick Filters create a new filter called “No Sub-Tasks” and set the query to be
issuetype not in subTaskIssueTypes()
This Quick Filter will appear at the top of the Kanban Board and when pressed will hide temporarily hide all sub-tasks, making the board appear as it was before sub-tasks were added.
Definition of Done
Sub-tasks should have a simple lifecycle. The developer who performs the sub-task is responsible for its testing and integration into the feature branch. Only when all sub-tasks in the Story are completed can the acceptance criteria for the Story be tested. However, the sub-tasks will linger on in the the Done column forever unless they are explicitly removed.
Jira Kanban boards provide a “Kanban board sub-filter” for hiding issues that are part of a release (by setting the “Fix version”). However, it is not desirable to make sub-tasks part of a release; other options exist. Here is a summary of all of the alternatives:
Include the sub-tasks in the Release. This unfortunately pollutes the list of Stories included in the Release, and makes the Release notes unusable.
Build an Automation to create a dummy release just for sub-tasks, that is scheduled to run, say, every week. This is a reasonable workaround, but pollutes the release history and (perhaps not so important) puts the stories and sub-tasks in different releases.
Use the “Hide completed issues older than” option under Board Settings, General. This is a blunt instrument; the problem is that it makes no distinction between Stories and sub-tasks and could end up hiding Stories that are Done but delayed for release.
Adjust the board Filter Query to exclude sub-tasks after time elapsed (e.g. 1 week). This is the least invasive way to effect what is essentially a visual change needed to control what issues are displayed on the board.
I recommend the fourth option; it is easy to set up and modify and does not impact any other aspects of the issue lifecycle, such as Fix versions. To do this, the Filter Query can be modified to not show older sub-tasks; in this example 1 week:
project = "Acme" AND (issuetype in (subTaskIssueTypes(), Story) OR (issuetype in subTaskIssueTypes() AND (status != Done OR resolved >= -1w))) ORDER BY Rank ASC
Summary
Displaying sub-tasks on the team Kanban board allows the team to see in one place exactly all the issues the developers are working on. The new “No Sub-Tasks” Quick Filter allows the team to retain their existing overview of Stories, Tasks and Bugs while allowing them to toggle the display of sub-tasks to support different conversations.
If you are using releases and Kanban boards in Jira, then you will most likely have a problem with issues not showing on the Kanban board. Specifically, issues are hidden if that have been released but their status is something other than “Done”. This can easily happen as Jira does not check if all issues are done before executing the release. The problem is the Kanban board sub-filter:
fixVersion in unreleasedVersions() OR fixVersion is EMPTY
This means that if the release version of an issue is set then it will be hidden as soon as the release is made in Jira regardless of the issue’s status. From the team’s perspective this is probably perfectly fine, these issues were done in practice it was just that there status was incorrect in Jira. In the worst case real work is hidden. Another problem is that even if the work is done, leaving the issues open skews the data Jira relies for the various graphs and metrics that it provides, e.g. team velocity.
To fix this, what we want is to only hide issues that are released and have status “Done”. To do this we update the filter to:
(fixVersion in releasedVersions() AND status != Done) OR
fixVersion in unreleasedVersions() OR fixVersion is EMPTY
Now all issues are displayed until the team actually sets the status to “Done”, which is the more intuitive behaviour. And if the release has been made before the status is set to “Done” then the issue will disappear immediately from the Kanban board.
Expect that old issues will reappear on the board when the filter is applied, but this doesn’t take long to clean up. Alternatively the filter above can be used to search for these old issues and close them first, but it is probably easier and better to let the developers do it themselves.
When adopting an Agile development process, the team should start with visualising their work using a Kanban board. At the start, the items on the board will probably cover a whole range of work including hopefully a few User Stories. What can happen then is that if the team uses these items to try to create flow they will probably end up trying to model their entire development process (from idea to product) on the Kanban board, or worse still, try to apply Sprint planning to it.
This is especially true for platform teams that move to become cross-functional teams. The problem is that a Kanban board can only really handle the flow of one type of object (i.e. model one process) and that object has to have a clear “Definition of Done”. But hey, the whole point of using Kanban is to visualise the work so the team can do something about it to improve flow.
As I said, the Kanban board works best if you supply it with right-sized work items, and in software development these items are usually User Stories. But using Kanban to manage the flow of User Stories is only going to capture one part of the work that the team do to deliver the right thing to customers. The other part is the preparation of these valuable right-sized User Stories and their acceptance criteria, what is known as the Product Discovery process. Product Discovery and Product Development form a highly integrated dual-track development process.
Product Discovery follows a different process to User Stories: ideas are being analysed and discarded or morphing into something else. (It is a fluid process, but Kanban can still be used even here to visualise it). So even though the team could be spending 50% of their time breaking down the problem, defining an MVP and creating good User Stories, this effort will not be visible on the User Story Kanban board. Or more accurately, the time spent is not part of the User Story lifecycle, since the story can’t start its journey across the Kanban board until the discovery process is complete.
This is not to say that the team cannot visualise discovery work, it just means that it cannot be attached to any particular User Story. Instead the team can represent it using another object: the Task. Unlike Stories, Tasks do not deliver customer value. Tasks could cover any kind of activity: a spike, an analysis, purchasing a license, setting up an environment, etc. Even if there is no customer value involved, the team should still strive to create well-defined or time-boxed tasks: what is the spike attempting to prove? When is the setup complete? In other words, Tasks must also have a clear “Definition of Done”.
While the Kanban board can be used to track Tasks, the team should only use it for well-defined tasks. Activities like meetings and discovery sessions form part of the team’s work that should not be quantified using Kanban. After all, the goal is to deliver valuable right-sized User Stories to the customer, not to document the completion of Tasks.
Lastly, the team can benefit from being able to visualise which User Stories belong to which MVP. In Jira (for example) the team can group related User Stories using the Epic Issue Type. Thus, an Epic can be used to represent an MVP. If the team want to track activities related to breaking down the MVP, then they can also be associated with the Epic. Finally, the team can also create a separate Epic Kanban Board which track the flow of Epics: they are bigger objects that move more slowly but should still have a clear Definition of Done.
In this article I will discuss how to get started with Agile in the most hands-on way possible, with no discussion of frameworks and methodologies. I believe it is important to understand the essence of Agile first, as it is easy to be overwhelmed with all of the techniques and tools that have evolved from it (Scrum, XP, SAFe, etc.).
The goal then is to create an iterative software development process that can be improved upon continuously. The only tool you are going to need is a stack of post-its and some wall space or a whiteboard where the team can work together.
Start small, which means starting at the team level. Learning to work in an Agile way will also require some experimentation as every team works differently. The point being that you will need to create some slack in the team’s schedule if you want to change the way they work. Finally, you or the team lead will take on the roll of Agile Team Coach.
I should also mention that there’s lots of help out there: blogs, forums and books. One excellent resource is the Agile and Lean Software Development Group on LinkedIn. Now let’s get started!
Step 1: Visualisation
First the team should start by visualising the their work, this is especially true in software development which by nature is very abstract. By visualisation, I am not referring to traditional documentation which tries to capture an entire scope such as requirements or test cases. What you want to visualise here is what the team is doing right now. For this you use post-its. Every team member writes down what they are working on, big or small, together with their initials in the corner, and sticks it onto a whiteboard or wall.
Now the team have an opportunity to discuss the work, make adjustments, add or remove post-its. The team can try to group related activities for instance. Spend about 5-10 minutes at this, no more; just enough time to smooth out the rough edges.
Step 2: Create flow
The next thing the team need to consider is what the definition of “Done” is for each post-it note. By “Done” we mean that the team is finished with the work item; it could be putting software in production, writing a manual, upgrading a database, etc. In reality, a lot of teams starting out with Agile do not have a clear definition of Done for their work items, so don’t sweat it too much yet. Fixing this will be part of the improvement process mentioned later on.
Create three columns on the whiteboard or wall and label them: Backlog, Doing, Done. Now each team member places each of their Post-its into one of the three columns. Finished? Great! You have now created your first Kanban board. (Here we introduce the most elementary and useful of Agile tools, the Kanban Board.)
So now the team has visualised their work and created a flow of work from left to right on the board. Congratulations!
Step 3: Reflection
The whole exercise above shouldn’t take more than an hour for a team of 10 people. Stand back and take a look at it. It may be obvious that some items on the board have unclear scope and some items are very large (or small). We’ll come back to these issues later.
One final exercise, sum the number of post-its in the Backlog and Doing columns and divide them by the number of members on the team. This will give you some indication of how much multitasking is going on and how much overhead is being created due to context-switching.
Step 4: Focusing on the goal
OK, the team have taken the important first steps in becoming Agile. And they will continue taking small steps, applying well-proven techniques that will improve the flow of work. But let’s discuss the goal; where is the team trying to get to? In the book The Phoenix Project, Bill is inspired by Lean manufacturing techniques used on production lines. Bill’s goal becomes the creation of a factory production line for his IT Department. As stated in Agile Principle #8:
Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
In other words the team shall create a process that they can reuse to build whatever software solutions the organisation or customers need now and in the future. The team can now take their new Kanban board and visualised work flow and use it to build a software factory!
Step 5: Breaking down the work
In our first iteration the team’s work items had unclear scope and different sizes. The team should deal with the scope problems first. This can be solved by breaking these work items into smaller items, each with a clear definition of “Done”. Spend 1-2 hours on this step, starting with the most important work items.
A classic problem is that a work item involves input from people outside the team. The Kanban board should not contain items that are assigned to “outsiders”. If this external work is a prerequisite for completing a team work item, then it should be added as a dependency to a team work item only. It is essential that the team have control over the work items on their board, even if they are currently blocked by external dependencies. The Kanban board should be used to focus on the team’s work!
Sizing of work items is about creating items that are of roughly equal size. As a rule-of-thumb, a work item should take about 2-3 days to complete, up to a maximum of two weeks. There are techniques for standardising work sizes, but for now I recommend a simple consensus from the team on whether an item is large or small or somewhere in between. Remember, if the definition of “Done” is software in production, then this must include coding, testing, etc.
In the worse case, a work item is so badly scoped and sized that it may not be possible to continue working on it in its current state and some more analysis (of requirements or architecture) is needed. If work stops altogether on such items then it should be moved into the backlog. This is one of the hardest things to do in Agile, but really knowing when a work item is ready for execution is one of the great benefits Agile brings.
A clear definition of Done for each item together with creating items of roughly equal size will build team confidence. By breaking down work items into smaller chunks and visualising them on the Kanban board it becomes possible for every team member (and stakeholders!) to understand what the team is going to deliver. And getting items to Done will make everyone happy.
Step 6: Limiting Work in process (WIP)
At this point the team have broken down the work into similar size chunks and this probably means that there are many more post-its on the board. (There are many tools available for creating digital Kanban boards, but this is still a low priority for now; wait 2-4 weeks before taking that step.) What the team needs to focus on next is WIP. This exercise should take about 30-60 minutes.
Earlier the team calculated how many work items were being done per team member. Ideally, each team member should be working on one item at a time, i.e. sequentially; so for a team of 10, the number of work items in the “Doing” column would be 10. In practice the figure is higher and the team need to think about what that number is.
In Agile terms, we are talking about the team capacity. We use this figure to set a work in progress (or process) limit (WIP limit). In other words, the team cannot start a new work item until they have finished a work item that is already in progress (unless an item is blocked). Remember, the team have a clear definition of Done for every work item, so they are supposed to be able to complete them before starting something new.
WIP limits are extremely important in creating flow. It follows that if the team tries to complete 20 work items at the same time it will take twice as long as if they were working on just 10 items.
For now, there is just one column with work in progress (“Doing”). The team should try to estimate how many man-days of work is in that column. Anything more than 30-40 days (3-4 days x 10 people) worth of work should be moved to the Backlog, and this means prioritising what needs to get done first. Prioritising is the responsibility of the Product Owner or Business Manager responsible for the product being developed, so naturally they need to be involved. Agile creates visibility for both the team and stakeholders!
Step 7: Daily stand-ups
Book 10-15 minutes with the team every morning for a stand-up in front of the Kanban board to discuss the day’s activities. The stand-up is for the team only, but guests can be invited on occasion. Longer discussions should be saved for break-out sessions with those involved. The focus of the stand-up is to make sure everybody knows what they are doing, if there are any blockers that need to be escalated, and to check that the Kanban board is up-to-date.
In case it’s not obvious, the Kanban board has now become the most important tool the team have for organising and visualising their work. Well done!
Conclusion
The team have made great progress! They have managed to visualise their work, create flow, size their work items and limit their work in progress. This demonstrates the concept of Continuous Improvement (“Kaizan“) as preached in Lean manufacturing, meaning that the team are constantly looking for ways to improve the flow of work.
In Agile we use Retrospectives to specifically discuss how well the flow of work is, well, working. All the team are involved in suggesting improvements, and then some or all of the team are responsible for implementing at least one improvement right away. Process automation (e.g. test automation) is a classic example of improving flow.
There are many, many other techniques that are used as part of Agile such as User Stories, Storyboarding, Minimum Viable Products (MVPs), backlog refinement and measuring velocity to create an iterative software development process. Scrumis a subject onto itself. But these are topics for another article.
Further reading
I highly recommend the following books:
The Phoenix Project by Gene Kim
User Story Mapping by Jeff Patton
Lean from the trenches by Henrik Kniberg
Accelerate by Nicole Forsgren, Jez Humble and Gene Kim
In the book The Phoenix Project there is a part where Bill is discussing lead times with Wes and Patty. Even though a certain task only takes 30 seconds, the lead time is still hours due to the time spent waiting for a resource to become available (Brent). Bill draws a graph to demonstrate the problem: if a resource is fully (100%) utilised then wait times become very long.
This statement threw me for a bit until I did some research. The graph definitely nails the problem with trying to maximise utilisation of resources. I mean, it is counter-intuitive for a manager to allow resources to idle. So what is the graph actually showing?
The graph shows that when the average utilisation goes over 80-90% then wait times become very long. So over time, a very high average utilisation will cause the queue to grow very long, increasing lead times dramatically. In other words, if a resource is very busy then it cannot cope with a workload that varies. The team must be allowed some slack so that lead times are still reasonable even when the workload is sometimes higher than average. In short, there is a trade-off between workload variability and resource utilisation.
This is well-understood in the manufacturing industry, but it is intrinsically true for software development. Everything that is built in software development is a one-off, unique. This creates huge variability in the job times of the development process; no two projects are ever the same. The challenge then is to reduce this variability so that we can create a more predictable workload and push up utilisation.
In Agile we use techniques such as storyboarding, MVPs and backlog grooming to manage variability and ensure that we maintain flow. WIP limits and Velocity are our KPIs that let us know how well we are succeeding in maintaining flow. Flow refers both to managing the variability of the arrival time of work (i.e. breaking down the work into smaller deliverables) and the execution time of the job (e.g. sizing of User Stories).
The science
Back to Bill’s graph. Where does it come from? It is actually based on Kingsman’s formula which is from the domain of queueing theory. In layman’s terms the wait time is made up of three parts:
So what Bill is actually saying is that, given a certain variability and a certain job time, then the wait time will be a function of the utilisation as shown in the graph above. Bill wants to focus on utilisation, so he normalises the other parameters (variation and job time) as follows:
Variation is the norm in software development, so it has to be dealt with. There are several ways to mitigate variability. Reducing utilisation is one option, but we can do better than having our developers and testers idling.
Kingsman’s formula shows that adding more work centres reduces sensitivity to variation (as well as increasing capacity obviously). However, this is probably more feasible in manufacturing than in software development, because a work centre (i.e. a development team) often has domain expertise, i.e. no two teams have exactly the same capabilities. But this approach may be more applicable in larger organisations.
We have already mentioned reducing variability using Agile techniques such as storyboarding, MVPs and backlog grooming, and this should be the primary focus of the team coach in creating and optimising flow.
Another option available to development teams is to have a technical backlog containing work that is lower priority that can be used to fill idle time and bring utilisation closer to 100%. The kind of tasks in this backlog should be small and independent. For example, it could involve refactoring, writing automated tests, learning about a new technology, and so on.
In summary, it is the combination of these techniques that allows development teams to be fully utilised. What Lean teaches us, is that the same discipline and structure that is used to optimise manufacturing flows, applies even more so to software development.
Tracking the team’s velocity gives us insights into both utilisation and the amount of planned work vs. unplanned work. We can also track the velocity of both Stories and Epics to see how good we are at sizing our MVPs. (An Epic is always an MVP in my book; this makes it clear what the definition of Done is for an Epic).
Skipping the queue
One of the early problems Bill had to deal with was departments trying to skip the queue. This is the result of a chronic failure of the development process. If lead times become unacceptably long (due to high utilisation, high variability or both), then eventually people will try to find shortcuts. This just makes a bad problem even worse, and represents a total breakdown in the chain-of-command. That kind of short circuit has to dealt with before any other improvements have a chance of succeeding. Hence the need to start by visualising all of the work in process.
Inventory
I was almost going to write that inventory doesn’t cost anything in software development, after all it is virtual. We don’t have to purchase raw materials and we don’t have to store anything in warehouses. (Yes, GitHub costs something, but it is a negligible cost in this context.)
But there is still inventory in software. The raw materials are just ideas, one-liners that take up virtually no space at all and until the team commits to building (analysing, developing and testing) something, the backlog can be reorganised and priorities changed as often as desired.
The rest of the inventory is in the queues between work centres, e.g. when handing over from development to test. This inventory does represent an investment in time and effort, e.g. breaking down the problem, defining an MVP and coding a solution. The cost of having this inventory is that the knowledge about the solution disappears over time; no amount of documentation can replace the shared understanding that existed when the team were working actively on the solution. Furthermore, TTM is probably the single most important factor for success nowadays. So to sum up, a lot of inventory, or WIP, is bad in software development. Watch those WIP limits!
Cycle time
Here is an example of a good article describing Lead times and Cycle times. However, the difference between the two is not very clear in my opinion. A new Initiative will contain an unknown amount of work, that’s why we analyse it and break it down into reasonably sized chunks; we are reducing variability and minimising risk. A task (e.g. User story) is only added to the backlog when it is somewhat well-defined and so the Lead time for every deliverable is a reasonably well-understood and managed parameter. Otherwise, Lead times just becomes guess work and that is not so useful.
But the backlog doesn’t only contain planned work, it also contains unplanned work; bugs and outages which must be dealt with immediately. This increases the Lead time for planned work in ways that can be hard to manage. While unplanned work cannot be avoided completely, it can be mitigated using a small iterative release process, i.e. continuous delivery, continuous improvements to the delivery process, as well as detective and preventative security controls.
So ideally, Lead Time only applies to tasks that are MVP-sized and, we should also have a WIP limit on new work to control Lead time. It does not make sense to fill up the backlog with Tasks that will be delivered years from now. Doing this, we achieve an understanding of the team’s capacity and that long Lead times indicate the need for an increase in team capacity, the need for more teams, or a change in priorities.
My agile development team was involved in storyboarding and backlog grooming for all new tasks, not just development and testing. The team were constantly managing the flow of deliverables at all stages on the Kanban board, both when there were too few and too many tasks in a queue. So the difference between “Task created” and “Work started” was really very small, and therefore Cycle time should be uninteresting.
In queueing theory there is a formula known as Little’s Law which is used to calculate the Cycle time. So does this formula still have relevance even if we are not interested in Cycle time?
The term “Cycle time” is somewhat non-intuitive. But if you think about it, the cycle time is also the average time it will take to deliver everything that’s on your Kanban board right now. For example, if your WIP is 10 and your average throughput is 2 tasks/day, then your cycle time is 5 days/task. Or put another way, the team can deliver everything on the Kanban board within the next 5 days. Now that’s a rather powerful statement. So Cycle time is also the Turnover time for all WIP.
The better the team get at breaking down the backlog into equal-sized chunks (i.e. minimising variability), the more relevant the turnover figure becomes.
And so if Cycle times converge with Lead times, then we are much more sure of our commitments to the business side of the organisation. Roll-on Big Room Planning!
