The First Way of DevOps: Flow
by Eric Hankinson, on Nov 1, 2018 12:20:09 PM
In the previous article, we started talking about DevOps principles and stopped after a brief introduction to “The 3 Ways” as mentioned in the book The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win. We’ll be continuing on by talking about the first way, Flow.
What Do We Mean By Flow?
When we look at the Value Stream Map, we should consider the performance of the entire system we’re in, not just a specific department or team within our organization. Focusing on a little silo won’t give us enough perspective on where bottlenecks are or where improvements can be made across the system as a whole.
We should understand the flow of work in our value stream. Work can originate from features, defects, requirements from other teams (InfoSec, Legal, etc), support tickets, etc. Your value stream should always be moving forward. If you identify areas where flow is going backwards in your value stream, that should be treated as a problem that needs addressed.
As The Ford Motor Company says, “Quality if Job Number 1.” All members of the organization that participate in the value stream should focus on improving quality. Most of us would like to only do something once. Sometimes that is not always the case. What is important is making rework visible and accountable. If the individuals causing rework don’t “feel the pain” then they are likely to continue impeding flow and any fixes they contribute are unlikely to correct the cause of the rework.
The key lies in having a deep understanding of your system. In a Systems Thinking approach, this doesn't mean that the smartest person in the room knows everything. High-performing organizations will lean on collective intelligence. You need to have a foundation of knowledge in order to make informed decisions. Deming referred to this as “Appreciation of the System.”
What Are Some Things That Interfere With Flow?
Visibility around problems and processes can be an issue. Software development work is often "invisible" because there is no tangible product you can hold in your hands. Where is flow being impeded in your current processes? Do you know if there are any work constraints?
You should strive to make work as visible as possible. I’m not talking about all the developers using giant screens with large fonts. Large visual work boards are one way. An Obeya, or visual room, is another way.
A good place to start is a simple idea board. Have individuals put up ideas to make their own work easier or faster. It may not immediately “move the needle” but the idea here is to get everyone comfortable with making their ideas visible to both peers and leadership in the organization.
Involving individuals in establishing qualitative and quantitative measurements for each process in the Value Stream will help organize a visual board. For example, departments within an organization may decide that they want to practice the 5Ss and want to track how well the department is doing.
Controlling the signal to noise ratio of your visual communication will go a long way in being useful to the organization.
Beware, though... an organization can get carried away with metrics and measurements for their visual boards & rooms. Controlling the signal to noise ratio of your visual communication will go a long way in being useful to the organization. Identifying critical success factors that help the organization reach yearly goals allows the collective to select the process measurements that can be visualized to radiate progress towards those goals.
WIP, or Work In Progress, varies from team to team and organization to organization. In the case of software development organizations, interrupting developers is easy. The consequences of the interruptions are generally invisible. Aso, something as simple as being on multiple projects at a time actually decreases efficiency and raises risk.
One way to help with this problem is to limit the WIP for project teams. A simple visual way of doing this is with a Kanban Board. The project team can define, agree on, and enforce WIP limits on each column of the board. This makes it easier for everyone on the project team, and others that may just be passing through the project team space, to see problems that prevent completion of work.
Long Development Cycles
Many organizations tend towards long development cycles. This can be anywhere from 6-24 months of development before user feedback is solicited. Some would argue that 2 months is too long for this.
One of the causes of long development cycles, especially within very large organizations, is BRUF and Analysis Paralysis. Then, once you get past that, the developers work first, then testing comes later. The testers test everything - regression, integration, edge cases. Then, some day, the code sees the light of day and gets into a Production environment that the customer is allowed to access. This is where the organization figures out if they even built the right thing or not.
Clearly, this is not a sustainable approach, but it is a pretty common one. How do we change this approach in order to increase our flow? One way is to look at implementing a GRIT approach. Rather than documenting All The Things at the beginning, we can (and should) treat requirements gathering as an iterative process slightly ahead of the development iterations.
Instead of relying on a QA team or department to do all of the testing, the developers can adopt the practices of TDD and ATDD as part of their workflow. This would then allow for automated unit & integration tests which would free up dedicated QA to focus solely on edge cases and the “hard” problems that can only be found via manual testing as the “easy” stuff is now found prior to anything reaching QA.
Restructuring how you handle requirements gathering and managing code testing are just two ways to cut down on the development cycle time. This will allow the organization to deploy to Production sooner.
Getting Code Into Higher Environments
How long does it typically take for a development team to push their code to Production? Are multiple departments involved? There could be hundreds or thousands of operations between the code entering into source control and the final deployment into Production. Depending on the number of individuals involved, there is N(N-1) / 2 possible communication paths in the project. If you think about it, if multiple teams are involved in the deployment of software to Production, it can be kind of like the game of Telephone.
How can we mitigate these kinds of problems? If we can reduce the number of handoffs, that would increase our flow. Automating portions of the work (code build, code deployment, server configuration, etc) can certainly help. Also helpful: reorganizing project teams so that they can be cross-functional to directly deliver value themselves versus having many external teams that are relied upon.
Other Project Barriers
Partially done work, extra unintended features, task switching due to changing priorities, defects found later in the development cycle, and team heroics can all contribute to impeding flow.
So, what can we do? Being introspective and looking to eliminate waste in our value stream can help. Partially done work loses value over time. The idea of “gold plating” generally adds complexity & testing effort and thus should be actively thwarted. Using TDD/BDD/ATDD techniques to shorten the time between defect creation & detection can diminish the difficulty of resolution. Lastly, don’t put your teams into a position where unreasonable acts become the norm.