Empirical Process Control Part 2: Why And How EPC Helps Agile Product Delivery Teams
10 April 2017 |
Stuart Munton | About a 6 minute read
In the first part of this blog a metaphor was presented regarding building a wall. It outlined the processes, constraints, and options that needed to be considered in the construction of a wall around a garden. This metaphor was to abstract away from the details involved in software engineering, in this Part 2 post, the details of why and how EPC will help Agile product delivery teams are to be considered.
Empirical Process Control (EPC) is a way to review and understand your past performance and make informed decisions about the future. With the correct practices in place, EPC can provide real time directional information, and with the correct discipline it takes nearly zero additional effort (note – a good way to go this is by the use of an Agile Lifecycle Management (ALM) application, but more of that to follow in the ‘How’ section…)
When teams are migrating from traditional project management approaches (e.g. PRINCE2) into Agile product delivery approaches (e.g. Scrum, Kanban) there is a shift in emphasis from a ‘command and control’ way of working where a Project Manager is responsible to a ‘self-organising’ way of working where the team are responsible. This change has significant benefits in terms of ownership in the team and subsequent throughput, but without EPC being understood and applied this change can also inadvertently lead to a system with reduced stability and control.
Some externally visible symptoms of this reduced stability and control are:-
- Teams going through cycles of under / over-committing
- Teams delivering work, but not the work to which they originally committed
- Teams not being able to forecast more than a sprint ahead
Back in the early days of Agile delivery, these were assumed to be some potential consequences of Agile delivery, but as knowledge of the Agile delivery methods had matured we have learnt that these things are not unavoidable, but it requires discipline and control – it requires EPC.
W. Edwards Deming – “Without data, you’re just another person with an opinion.”
Lord Kelvin – “When you can measure what you are speaking about, and express it in numbers, you know something about it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind; it may be the beginning of knowledge, but you have scarely, in your thoughts advanced to the stage of science.”
When teams first move into using Agile practices, they have a tendency to get tied up with the process, practices and doing parts of the framework they are using. This is fine to start with, but if left unchecked this becomes the new normal, teams believe that by ‘Doing Agile’ by following the practices, then they will get all the outcomes promised (see – “Twice the work in half the time” by Jeff Sutherland – Co-creator of Scrum).
Teams are not ‘Being Agile’ or ‘Being Empirical’ – they are following a cargo cult approach to Agile delivery with limited understanding of why practices are completed. The view being that by enacting the Scrum events that somehow Agile magic will happen.
Teams miss the fact that they need to look at the data to understand performance and forecast near future performance (and quite often there is no longer a Project Manager to do it for them…).
Scrum is based on Empiricism and EPC – as stated in the third paragraph of the Scrum guide…
“Scrum is founded on empirical process control theory, or empiricism. Empiricism asserts that knowledge comes from experience and making decisions based on what is known. Scrum employs an iterative, incremental approach to optimize predictability and control risk. Three pillars uphold every implementation of empirical process control: transparency, inspection, and adaptation.”
http://www.scrumguides.org/scrum-guide.html#theory – ©2016 Scrum.Org and ScrumInc. Offered for license under the Attribution Share-Alike license of Creative Commons,
Empirical Process Control provides Scrum teams with the approach to inspect, adapt, and by definition control delivery. The control comes from looking for variances in the expected outcomes (the forecasts) and then attempting to understand and rectify the issues.
From a Scrum perspective – EPC can be understood further by looking at the team’s velocity* across a number of Sprints. If you have a team that is stable (i.e. people are not joining and leaving the team, scope is not randomly and dramatically changing) then it should have a stable velocity. This is a transparent value, that can be inspected each Sprint. If the team has delivered 50 Story Points of work in the past four Sprints, then EPC will forecast that this will be the same velocity in the next Sprint. This seems blatant common sense, but often this is overlooked when a team is coming close to a product release date (often arbitrarily picked by middle / senior management).
*Velocity = the quantum of work that a team can complete in a set period – a metric ‘for the team, by the team’
EPC requires that a Scrum team have transparent delivery metrics, understanding the value of metrics, analysis of the data to see pattern / variances. This is where the Agile Lifecycle Management (ALM) applications come in very handy…
ALM applications (e.g. Jira, Rally, Leankit, Basecamp) are designed to help teams work and deliver together as a team. Work is broken up into small parcels and generally a defined flow is applied across them to guide the team to getting the work to ‘done’. We tend to generically call these parcels of work ‘Stories’ (but in a purist view they are ‘Product Backlog’ items and can come in many different types).
ALMs can work across many different styles of Agile (e.g. Scrum, Kanban, SAFe etc) and the selection of the working method will have a difference in how the controls in EPC that is applied. Empirical data is both directly and indirectly captured in the ALM, this needs to be reviewed, consolidated and trended to provide both stability and control. EPC with ALMs provides both the basis for short term forecasting (see the Velocity chart in Jira) plus a basis for systemic control (see the Control chart in Jira).
To link back to our metaphor – instead of reviewing at the end of each Sprint (day in the metaphor), you are able to review software delivery at the end of each story (block in the metaphor). Each block in the wall told a tale, it had a specific size, and a specific amount of time to complete. If our stone mason had been able to apply EPC and gathered data in an ALM, then he would have been able to respond quicker, apply more control, and deliver in a more stable way, all good outcomes…and maybe the wall will finally be built in the end.
React Native Engineer (London)
Champion software quality and technical vision for AND and our clients, work on large-scale projects and help junior and mid developers grow in their roles.
Technologies you will be using
Tech Lead (Reading)
Bring your expert tech knowledge to the table to influence the direction of projects, whilst coaching and your team through engineering best practices.I'm Interested
DevOps Lead (Reading)
Bring your delivery expertise to the table, leading the pack as ambassador on operational requirements, influencing and continuous development.I'm Interested