I’m jotting down a few notes on Scaling Agile software development as Bucharest Agile group invited me to talk about doing this. I have already warned them that I am very skeptical about attempts to apply agile practices on large endeavours. While preparing for our conversation, I thought it might be helpful for me to blog about the reasons why I’m not a fan of Scaling Agile as this may make our conversation easier to follow and help the group to come up with some questions.
When we apply Agile principles, we strip away process so that software developers can work more collaboratively with business people to identify what is the most valuable thing for them to deliver next. We focus on building working software and releasing as early as we can to help us figure out what to build based on feedback from users. Working this way is much harder when a lot of people are involved!
A bunch of things break down as you scale up. The biggest one is not being able to maintain interpersonal relationships through which rich information flows, these are replaced with weaker lossy forms of communication and misunderstandings about what is the right thing to build next follow.
Typical things that become difficult at scale are access to business people and infrastructure controlled by others outside immediate team. Meetings get long and tedious, we start sending a representative from each team, which introduces more secondhand information, emails and documentation.
When a project is big and is being changed by many hands it becomes much harder to understand the whole, we start to introduce hierarchy with a select few looking at the bigger picture and paying attention to separating concerns to allow different teams to work in parallel. As a result, choice is removed from the team and it can feel in teams that edicts come down from on high through a series of chutes and screens that mask the reasoning behind them.
Often the initial attraction of Agile approaches to a business is to reduce delivery timescales and enable developers to work faster with a lightweight approach. Working in small teams allows individuals to feel more engaged because they have some influence on how things are built. When a lot of people are working in parallel coordination becomes harder. Ideally we need skilled people who have enough experience to work independently without the guide-rails of process.
Yes, I have seen plenty of organisations with large systems being maintained, expanded and extended by multiple teams. When each team has to interface to many others decisions take longer to make. Where a large number of people are working on a complex thing, it takes a lot of effort to keep up with what’s happening other areas and most reduce their focus to the work at hand. Once they reduce their focus and interfaces between teams become walls to hide behind. When myopia is easier then teams tend to make local suboptimal changes rather than considering the whole.
A large body people working on a large body of code often feels crazily chaotic or oppressively bureaucratic to work within. In these conditions developer motivation is often dimmed, people leave taking away valuable knowledge of how the system evolved and why things are. New people who need to learn how things work join and struggle to make sense of it all, the situation gets worse.
There are a few agile practices that can help in scaled up situations - such as having reliable automated builds, decent test coverage and teamwork but these are also painful to maintain with a lot of people involved. I’ve seen many large organisations attempt to apply Agile at scale. It’s hard and although it can bring more humanity to work in-the-small, I’m not sure there is much evidence that it is actually better than traditional ways of organising the work.
I appreciate that there are large systems out in the world that need to be supported and evolved to better meet user and business needs. I remain unconvinced that throwing a lot of people at the problem makes things any faster. I believe deadlines driving this behaviour are defined without realising the impact on the maintainability and quality of existing software. Code is an asset and so is the knowledge that software developers carry - businesses can run into costly mistakes if they ignore their value.
Attempting to scale to go faster is driven business greed and ambition - not a bad thing in the commercial world but it’s like seeing a someone blindfold attempting to run along a cliff edge while juggling priceless antique glass vases. Mistakes at scale are costly!
When it seems that scaling up is the only way to go, challenge yourself to think of a smaller simpler thing to build sooner that gives faster feedback. Try to limit your total Work-In-Progess by reducing the amount of development being done in parallel across the organisation. When a smaller group of people is involved in development, choices about which way to go next can be made more quickly and there's less waiting around. Ultimately, this goes back to Agile Manifesto values - focus on better interactions betweem individuals to produce valuable software sooner -- not on a process to enable lumbering projects at scale.