Scrum vs Waterfall: a possible compromise
September 2016
As an IT project manager with more than 18 years of experience, I face every day with a fundamental question: What is the best project management approach when you receive a project charter from your key user?
As commonly accepted, the Scrum methodology works like a charm in a context where customers have no clear idea for deliverables, or where you are dealing with a feasibility study; but it also has some (critical) limits when applied to a complex scenario with the process already in place—big problems that have to be extricated and so on.
Some examples in an IT context may be “performance with big database” or “define a process for backup and restore quickly and efficiently more than 4,000 mailboxes.” Another example (not only found in IT) is how to dedicate seven people to a single project when I lead a team of 12 developers and I receive more than two project charters per week.
At the opposite end, a strong and pragmatic approach—one directed by the 47 processes of the Project Management Professional (PMP) certification from the Project Management Institute—is sometimes too casehardened to be accepted by customers.
When I started studying Scrum, My first reaction was that this methodology would not work at all in my context (a company with more than 4,000 people, and a member of an international network with more than 200,000 workers). I guess I’m not the only one—many PMP-certified people turn up their noses reading about so much flexibility in defining time, cost and scope…where are our solid strongholds?
One of the aspects of Scrum that is very hard to accept (from my point of view) is the solution design. During my career, I experienced what it meant to create a wrong software architecture. If you do not provide the right data structure to your database, you will soon face performance problems—or you will discover that it’s impossible to obtain a quick webpage.
Alternatively, you could discover that an entity was completely forgotten and you don’t have the required information for adding it at runtime. How can you build a mansion if your basement has been designed to support a little house? Unfortunately, you cannot change the basement once it has been covered!
Notwithstanding these critical aspects, my developers love some processes from Scrum theory. First, the “working software over comprehensive documentation” explained in the Agile Manifesto: Any developer hates writing functional business blueprints, technical specifications and tons of user acceptance tests before starting with a sane “public static void writeHelloWorld()” in his development program. And any project manager will spend a lot of time asking for documentation on the software, approved specifications, a clear statement and planning before (not after) the work is done.
Moreover, they love the iterative development—another reason to start writing code, immediately creating a working mockup that becomes progressively similar to the expected deliverables. No matter if there is no global coordination, no future vision, no security analysis and no planned integration test. “Do you need an estimation for task 1.25.3.2? Okay, no problem, just a few days of work and I will send you an email…when the task is finalized!”
I have been a developer for a long time, and I perfectly understand this point of view. Today, roles have changed and—as a project manager—I am entitled to keep the project (and developers) on the right track. It doesn’t matter how many languages or platforms you know.
Nevertheless, some Scrum principles are also found in the project manager perspective. One of my favorites is value-driven prioritization: the concept that gives higher priority to what is really important for customers. Another one is self-organization: I am strongly convinced that we act like a football team, where the coach suggest a strategy but players score goals. Then, the development team must be involved, starting from the initial stage of the project: A WBS, activities list and task duration provided by the team are more realistic (and probably more efficient) than the ones provided by a PM.
So, is there a possible compromise between a waterfall approach and agile? Or are they are destined to fight each other for the rest of time?
In my opinion, there are some principles of the PMP that should not be called into discussion; but we can find some mediations between these two opposites:
- Time: When you are working in a big organization, you couldn’t accept that your team works on a unique project without a clear indication of when the work will be done and resources released. So the project starting date and a predefined ending date must be clearly declared for any project. Works that could be completed before the end date are okay; what is over will be part of a further release.
- Basement (or scope dimension): Remember that you cannot build a house over the sand because, sooner or later, you could experience a unique thunderous collapse. So please, my dear customer, write a project specification explicating if you are going to manage 50, 500, 5,000 or 500,000 items with our web application.
- Change management plan (and clear contract clauses): The Scrum approach loves changes (perhaps) because we cannot pretend that our customer—especially if it is not IT confident—is able to describe all the requirements at the initial stage of the project. However, we cannot accept a fixed-price contract without a clear list of deliverables. An acceptable solution should be a fixed price plus a variable percentage (i.e., 20-30%) that should be paid at the end of the project. This percentage must be declared in the contract, and no one should be surprised when I ask for extra funds.
- Roles and responsibilities: There must be one—and only one—key user that collects requirements from other stakeholders and works very strictly with a project manager. No matter if the team is a Scrum team or a standard team: We need a person that drives the project. He or she has much more influence than a ScrumMaster but is less directive than a strong PMP project manager. He or she is a team leader with sound project management capabilities.
- Iterative development: We can accept iterative development during the initial part of our project, but not at the end. We will spend the first part of the executing phase creating a functional mockup, and we will have very frequent comparisons with the key user. We will create the best prototype our customer has ever seen, but at some point we will require a formal “okay” to proceed. Then we will consider this prototype as a visual version of a scope statement of work: No changes are admitted once this approval has been provided.
- Functional and technical documentation: There are several acceptable levels of quality in documentation. We can assume that detailed comments on a single class or methods are not strongly required at the initial project phase—but providing a brief description of the macro entities and how they are related to each other when a process begins (or when it is scheduled) could not miss. At the end of the project, the development team should complete the documentation and the key user will verify and approve what’s provided.
- Quality: Quality is never an option. The development team must conduct a unit test before involving the key user. An integration test has to be tracked and documented. Quality control results are part of the official project documentation and must be archived for further reference. Quality assurance processes are still running and last the entire project lifecycle.
- Risk management plan: Accepting scope that isn’t well defined is a risk in itself. Working with a variable part of the contract is a way to mitigate this risk, but we must put all our attention to the risk register. Try to identify critical points as soon as possible—and verify how to mitigate or transfer risk outside the project.