During my years in the IT industry — I’ve heard some bazaar ideas. Collectively we love fads and throw around lingo with scant regard to its provenance, proper meaning or implication — everyday in IT is a buzzword bingo day. Occasionally (but only very occasionally) I stop to think “is that such a good idea!?”. Prompted by some strongly felt views and the juxtaposition of Waterfall and Scrum, “Water-Scrum-fall” is one of those moments. In this post I thought I would share my thoughts and research on the matter, covering off questions such as “what is water-Scrum-fall”, “why does it exist?”, and “could we be stung by it?” ….
First off, What is “water-Scrum-fall”? …
Waterfall
Waterfall emerged from common practice during the early days of the IT industry and is a sequential lifecycle that reflects a gated delivery approach from requirements, design, build, integration, and test through to customer acceptance. Emerging from common practice, identifying a definitive definition is somewhat more difficult. Winston Royce is widely credited with its definition in 1970 (although he did not invent it!), albeit only so he could highlight its failings. It is more or less an abandoned approach with little “public” support; it has no official definition nor education programme.
Scrum
Scrum is an agile framework for managing teams popularised and defined by Ken Schwaber and Jeff Sutherland. It is based on observations from Takeuchi and Nonaka into post war manufacturing advances in Japan. Schwaber and Sutherland are still evolving the method with a clearly written definition and educational programs (Scrum Guide — reference 3) . Use of Scrum is wide spread within the IT industry (and beyond).
“water-Scrum-fall”
Water-Scrum-fall is widely credited as being defined in 2011 by Dave West of Forrester Research in an article (reference 2) highlighting the reality many organisations find themselves in with their journey towards agile. In a eery rerun of waterfall’s history, Dave West defines water-Scrum-fall in an attempt to warn others about its use, only to get the credit for coining the approach. In his article, he vaguely defines the approach as being a fusion of “water” with regard to the requirements, “build” being conducted using Scrum, and “fall” being the infrequent release of software at the end of the process. Dave West describes water-Scrum-fall so he can warn:
“it’s imperative that application development and delivery professionals push back on the water-Scrum side of the model. Spending too much time upfront will not increase the quality of the release; on the contrary, it is wasteful.”. On the “fall” part of the model Dave West warns: “Application development professionals should challenge the status quo of infrequent releases and push to better integrate release processes into the development team.”
I would summarise his arguments as “it’s not agile” and it doesn’t align well with a “DevOps” approach — two fairly fundamental drivers for most organisations today.
Researching this post, I have come across a number of different definitions of water-Scrum-fall, with differing amounts of “waterfall” formality:
Other definitions are more like this …
Which everway you look at it, it looks largely like waterfall in disguise. It has all the same problems, like Humphrey’s Law “Users don’t know what they want until they see working software”, Wegner’s Lemma “It is impossible to fully specify or test an interactive system designed to respond to external inputs.”, and Ziv’s law “software development is an inherently uncertain process, and hence it can be hard (impossible?) to fully define a specification upfront”, to name but a few. You are also at the mercy of Kahneman’s/Tversky’s Planning Fallacy, underestimating how long projects will take, and overestimate how quickly you can change things, and Hofstadter’s law, namely “It always takes longer than you expect, even when you take into account Hofstadter’s Law”.
Why does it exist?
From what I have experienced, the key reason for adopting a water-Scrum-fall approach broadly relates to the contractual aspect of needing to fix the delivery outcome. This could be for a number of reasons including a legacy approach to procuring IT systems, limited availability of the user communities, a product based delivery or a complex Systems Integration job, and/or a lack of knowledge around modern delivery techniques.
Could we be stung by it?
It’s based on poor foundations, namely Waterfall
We are now two decades into the twenty first century and the arguments around the failings of waterfall have been dredged over for at least the last 50 years. Through the application of logic, common sense, or retirement, I come across fewer and fewer people within the industry who would now advocate such an approach. Thankfully collective wisdom seems to have prevailed and tradition management approaches in the IT and wider industry seem to be in steep decline. Meyer states “(Waterfall )… serves as a textbook example of how NOT to organise a software project” (reference 6). As waterfall and traditional management is displaced, so Agile and Lean are making inroads into wider corporate life, for example Management 3.0, Holacracy.
I’m pleased that Royce’s and the wider IT industry’s criticisms of the classic waterfall lifecycle as being ”risky and invites failure“ (reference 1) have rightly hit support for waterfall. Promoting late feedback is an inevitable consequence of the lifecycle model, and even those who may be more wary of agile are still likely to be favourable to iterative and incremental delivery — an approach Royce suggested. With the evolution to SaaS and DevOps, waterfall and other sequential approaches should be considered “legacy” approaches for a bygone age — not fitting well with today’s needs and challenges.
Key proponents in the 80s for the Waterfall approach were Yourdon and DeMarco, with the latter writing seminal books such as “Structured Analysis and System Specification”, and “Controlling Software Projects: Management, Measurement, and Estimation.” In 2009 Tom DeMarco (reference 5) reflected back on the advice given during the 80s and noted that he is now
“… advocating a management approach, … that might well steer the team toward agile methods, at least toward the incremental aspects of the agile school.”
And later on …
“Software development is and always will be somewhat experimental. The actual software construction isn’t necessarily experimental, but its conception is. And this is where our focus ought to be. It’s where our focus always ought to have been“.
Spot on DeMarco!!
Given that industry consensus is that waterfall was not the best idea it ever game up with, what makes people think that water-Scrum-fall is any better?
Water-Scrum-fall is simply not defined!
In my experience, being “vague” is a great way to side step criticism, however you also miss the opportunity for constructive peer review. Water-Scrum-fall’s vagueness is arguably it’s best defence. Inspection and adaption is at the heart of improvement, and is definitely at the centre to Agile methods. That cannot be said for water-Scrum-fall which appears to have no definitive and formal definition — hence it makes it difficult to robustly critique. Given the level of investment required to “fill in the blanks” around water-scrum-fall’s definition, why would anyone go to that trouble rather than picking a better defined alternative?
Scrum express forbids being used in this constrained manner
Notwithstanding a clear definition of the approach, it is not hard to see how the waterfall fusion with Scrum could jeopardise certain aspects of Scrum’s framework. Unsurprisingly being aware of such reputational dangers, the Scrum Guide expressly prohibits customisations that break its core guidelines:
“Scrum is free and offered in this Guide. Scrum’s roles, events, artefacts, and rules are immutable and although implementing only parts of Scrum is possible, the result is not Scrum.”(Reference 3)
There is a significant risk that Water-Scrum-fall undermines the primacy of the Product Owner, the empowerment of the delivery team, and the approach to deliver business value in usable increments after each sprint. The use of scrum in fixed price/fixed scope environments has been discussed as problematic from the early days of Scrum, as noted by Ken Schwaber in 2003 (reference 4):
“… adding a waterfall phase to the front of Scrum …. was terrible, and what possible benefit could Scrum provide after it had been so corrupted!”.
Schwaber goes on to advocate selling the competitive advantage of Scrum rather than working in such a waterfall manner; his feeling on “water-Scrum” leaves little to the imagination.
A key aspect to Scrum is team empowerment to deliver business value. The waterfall element of the water-Scrum-fall ultimately binds the team and limits the opportunities for it to deliver true business value. Faced with predefined and time bounded constraints, someone else’s design, and ultimately not carrying the full responsibility for delivering the finish product, it is unlikely any team would feel fully committed, courageous or respected. Certainly this is not quite the “competitive advantage” that Schwaber/Sutherland’s advocated.
It lacks Empiricism
Empiricism is great and Scrum is explicit about its allegiance to empirical process control, an approach founded in chemical engineering — this is clearly a positive step forward for those who wish to see a more disciplined and controlled approach to software delivery. Scrum is about delivering business value faster, more dependably with the empirical principles of transparency, inspectability and adaptability promoting a more thought-out and repeatable approach.
In contrast it is hard to see how to effectively integrate transparency, inspectability and adaptability into water-Scrum-fall without compromising it’s simplistic sequential lifecycle. Whilst water-Scrum-fall at least adds a degree of empiricism to the build phase (by hijacking Scrum), from a coherence point of view it is still a long way behind approaches likes the 25 year old Unified Process.
Illusion of control / Role protection / Ignorance
In my experience, the types of projects often used for water-Scrum-fall typically involve a considerable degree of time and/or cost related commercial risk to the supplier. These risks can take many forms — scope creep, complex dependencies, cost; the reaction being to increase governance and oversight for these projects. This is a very natural response, but it begs the question “why stop at water-Srum-fall?”. If you really believe in the waterfall lifecycle — why not adopt a full fat waterfall approach so you can have the illusion of absolute control!
Not only does Water-Scrum-fall provides this feeling of being in control (at least the illusion of), but it provides roles for displaced waterfall managers. This latter point is something Craig Larman’s law (part 2) warns us about:
“any change initiative will be reduced to redefining or overloading the new terminology to mean basically the same as status quo”
Then there is plain old ignorance. Managers know waterfall/classic project management techniques and they may even know Scrum (certainly enough seem to go on the Scrum Certification course) — but they don’t have the IT experience to work outside these two polar opposite approaches.
Consequently they can imitate — but they can’t Innovate. Scrum is not the only incremental and iterative development approach, and if the “build” phase is suitable to an iterative way of working, then so should the “water” and “fall” parts!?
In my experience many Project Managers see Scrum Masters as a stepping stone for those wanting to become junior PMs. This in itself demonstrates a degree of ignorance about the fundamental differences between Scrum and waterfall.
It doesn’t tackle risk
Software development operates in a space dominated by Conklin’s “Wicked problems”. With Wicked problems, traditional preplanned delivery approaches don’t work well (reference 7) — for example, Rittel and Webber pointing out that baselining requirements is a receipe for disaster — risk is inevitable. If risk is a key driver of water-Scrum-fall adoption, it would make sense then that the central thrust of the water-Scrum-fall approach was risk reduction — risk becoming the primary statement of business value (in the absence of any other business value prioritisation). In this context, the mechanisms behind backlog prioritisation becomes clear, namely prioritise work by risk. However as water-Scrum-fall is not defined, this hugely important consideration is not covered. Whilst I’m pleased other methods tackle risk as a central concern, water-Scrum-fall remains ignorant to it.
There are much better alternatives!
Since Royce’s 1970s paper, the IT landscape has been littered with many better defined delivery approaches, most with much broader industry support. Scaled agile approaches such as LeSS, Nexus and SAFe all have clear strategies for delivering complex and risky projects including those with significant inter dependencies and risk. The early iterative and incremental approaches of the Unified Process and DSDM equally have better delivery models for driving risk out early for complex projects. Therefore, there are many better alternatives to use without having to throw away 50 years of IT process improvement by reintroducing waterfall by the backdoor.
Conclusion — So where does that leave us?
If in 1959 Alex Issigonis, the investor of the mini, had described his car as “sound — but has some major flaws so only use it for a couple of trips”, would Mini still be a successful brand today? I think not, however that was essentially the message from Dave West who is widely credited with naming the broad “water-Scrum-fall” approach. Waterfall simply refuses to die — and water-Scrum-fall seems like the latest attempt to keep it alive.
I fail to see the attraction of water-Scrum-fall. It is an ill defined approach based on an earlier, ill defined, and discredited lifecycle. It requires a new name as it likely breaks Scrum well articulated rules. In its vagueness of definition, there is no concept of business value (which in a fixed everything world, I assert should be “risk”) and therefore it is unclear the benefits one will derive from using the approach to guide work. Many of these issues could be rectified with more effort around formalising water-Scrum-fall’s definition, but why bother? There are many better documented, tried and tested, and mature approaches out there. Other than for reasons of ignorance or to further the illusion of control — it is hard to see the appeal of water-Scrum-fall, especially in a world where others have long since embraced the competitive advantages of Agile and are well on the way to DevOps.
For some, unwanted contractual constraints are an everyday fact of life. Despite best intentions and advice, sometimes these constraints cannot be avoided, the result being a world of fixed scope, fixed cost, and fixed time constraints. It is precisely in these more challenging environments where we should not consider abandonment of best practice for the promise of “snake oil”. Whilst Iterative and Incremental methods and Agile offer no golden bullet (nothing will), they do offer better control, faster reduction of risk and repeatability through empiricism.
Dave West advises asking the following question of management around the use of “water-Scrum-fall” (reference 2) — “Do you want the technology to help us adapt quickly to threats and opportunities?” If the answer is yes, then “tune the approach to favour discovery and action over planning and execution” — i.e use Iterative and incremental methods over waterfall / water-Scrum-fall. If the answer is no, you may be consigning yourself and your company to history.
Other Posts:
References:
- Managing The Development of Large Software Systems, Royce 1970 — http://www-scf.usc.edu/~csci201/lectures/Lecture11/royce1970.pdf
- Water-Scrum-fall is the reality of agile, SD Times, Dave West 2011 — https://sdtimes.com/agile/analyst-watch-water-scrum-fall-is-the-reality-of-agile/
- The Scrum Guide, Schwaber and Sutherland 2017 — https://www.scrumguides.org/docs/scrumguide/v2017/2017-Scrum-Guide-US.pdf
- Agile Project Management with Scrum, p147, Schwaber 2003
- Software Engineering: An Idea Whose Time Has Come and Gone?, IEEE 2009 — https://www.infoq.com/news/2009/08/demarco-software-engineering-
- Agile!, Bertrand Meyer 2014
- Wicked Problems, Wikipedia, https://en.wikipedia.org/wiki/Wicked_problem
