Engineering, Management, Posts

Between Waterfall and Agile – The Grey Area

What is the development theory that Steve Jobs used to develop the Apple II computer? Do you think that when Thomas Edison invented and mass-produced the light bulb, and many other inventions, he was thinking about what project development model to apply? What model was on the mind of Henry Ford when he revolutionized the automotive industry (besides the model T)?

If you have great visionary leaders who have a good understanding of the market, and talented people who master the state-of-the-art technology, with a bit of luck, you can develop a best-in-class product, and disrupt the market, without the need for any management/development theory. Following your success, people (not as smart as your people) will try to extract lessons from your good practices, theorize them, and sell them to other companies as remedies to their problems. Good practices will always help to improve the outcome, but will never fix the problem of narrow-minded leadership and unmotivated people.

This sets the ground for the sequel.

Image for post

Product/Software Development

The development of a product/software comprises the following key activities:

  • Requirements Gathering from the customer/user or market specialist. They define what the product/software is expected to do/be. Requirements are not technical solutions.
  • Architecture & Design: this phase, which can be split into two phases in complex systems, translates the requirements to technical specifications. Architecture deals with the high-level design, and the definition of the interfaces and interactions between subsystems. Design deals with the components, functions, subsystems, etc.
  • Implementation/coding of the specified design. This task can be done by the designer, by another developer, or automated.
  • Testing: there are two types of tests, verification and validation (V&V). Verification checks that the design and implementation work as expected (e.g. unit testing, white-box testing). Validation checks that what we designed and implemented actually fulfills the requirements (mainly black-box testing).
  • Release: in the case of software development, this phase entails the deployment and maintenance activities. In the case of a hardware product, it entails mass production.

There are different ways to perform these activities. The best way would depend on the type of the product and the market, the aggressiveness of the competition, the experience of your team, and so on.

The development practices fall broadly into two models: Waterfall (sequential, rigid, and one-shot) and Agile (adaptive, iterative, and incremental). In reality, there is a vast gray area between the two, and one can (or must) customize a model that is best suited for the project. Here are some models:

  • Waterfall
  • V-Model
  • Sashimi Model
  • Incremental Waterfall
  • Unified Process
  • Spiral Model
  • Agile Mindset and Practices

Most of the models above are inspired by the software industry, by can be easily adjusted to other industries. For more details refer to Agility — Beyond Software Development. Here, software and product are used interchangeably.

The Waterfall Model

The Waterfall model is a sequential process, as illustrated in the following figure.

Image for post

This model can be very efficient in the following cases:

  • if the requirements are well defined and understood, and
  • if the teams are skilled and have experience building similar product/software, and
  • if no change (in requirements, design, or else) is required or might occur during the development.

Waterfall is for predictive and repeatable (business-as-usual) projects. Its major drawbacks are:

  • No flexibility: once the design started, the customers cannot introduce any change in the requirements.
  • Mistakes are costly: design errors and requirements misunderstanding are detected late, during the testing phase, when the error cost is the highest.
  • Not customer-centric: the first release comes late in the process, and the customer/user needs to wait until the end of the development to test the product.

The V-Model

The V-model is an advanced version of the Waterfall that overcomes some of its shortcomings. By emphasizing the tests earlier, defining their acceptance criteria during the Architecture and Design phase, the team could avoid some errors and misunderstandings. This requires more upfront work. The V-model is presented below:

Image for post

Besides reducing the mistakes (and their costs), the V-model has the same cons as the Waterfall model: no flexibility, nor customer feedback before the release. Thus, it remains better suited for the same type of projects as for Waterfall (predictive and repetitive).

The Sashimi Model

The Waterfall and V models are sequential and not adaptive; every phase relies on the previous one. They don’t allow for adjusting (shortening) the project schedule. They might result in poor resource utilization; for instance, during the design phase, the implementation team and testing team will be waiting and cannot work on the project.

Image for post

The Sashimi model is a way to overcome these problems. It is based on the Waterfall model but allows for the overlapping between different phases when possible. The design team can start working on the requirements that were defined, while the marketing team is working on the rest of the requirements. The V&V team can test some functions while the designers are working on other functions, and so on.

This could shorten the development time and allow for better use of the skills and resources. The drawback of this process is that it may result in rework, and waste of time.

The Incremental Waterfall

Another variant of the Waterfall model is the incremental model. In this model, the project is divided into several parts that can be performed incrementally. This could allow for the overlap between increments (parallel workflow) to improve resource utilization.

Image for post

Some projects are based on previous projects (product replacement, Kaizen, or next-generation), in this case, most of the upfront work and design are already there. The team can implement in increments to speed up the development and deliver value earlier. This is shown in the following figure.

Image for post

Using the incremental model, the organization may profit from early value delivery, and get feedback and make necessary changes between increments if possible. The weakness of this model is that it may cause rework and increase the cost. This model allows for some adaptivity in the process, a value that is embraced by the Agile mindset.

The V, Sashimi, and Incremental models are variants of the Waterfall model. They work best for predictable projects, with rather experienced teams. The next two models (Unified Process and Spiral) are more suited for high-risk projects and allow for more adaptivity.

The Unified Process

The Unified Process (UP) is a framework that can incorporate other methods and models. It focuses on risk mitigation for big and risky projects. By allowing for iterative and incremental development, it increases the chances of success. It is a flexible approach, centered mainly on the architecture and use cases.

The Unified Process divides the project into four phases (with overlap):

  • Inception
  • Elaboration (milestone)
  • Construction (release)
  • Transition (final production release)

Even if the requirements are not all known early, the UP welcomes new requirements during the development.

Image for post

By adopting the UP framework, the organization may profit from earlier value delivery. However, it is a rather complicated framework that may consume more resources. It is ill-suited for small projects.

The Spiral Model

The Spiral model is recommended for big projects with very high risks of failure (e.g. aerospace projects). It is a risk-driven cyclic model that comprises four steps:

Image for post
Source
  • Determine objectives
  • Identify and resolve risks
  • Develop and test
  • Plan the next iteration

These four steps are iterate over and over until we achieve the end product. The spiral model is iterative and incremental. It is adaptive and allows for changing requirements at any time during the development. It is a risk-driven that embraces the “fail fast to learn fast” mindset. It can incorporate other models and can be considered on the Agile side of the spectrum.

Agile Mindset and Practices

OnFebruary 11th — 13th, 2001, at The Lodge at Snowbird ski resort in the Wasatch mountains of Utah, USA, seventeen experienced software developers met to talk, ski, relax, and try to find common ground — and of course, to eat. What emerged was the Agile Software Development Manifesto, with 4 values and 12 principles that are summarized in the following figure.

Image for post

Agile Practices

Image for post

Agile is a Software development mindset that embraces change. It is neither a process nor a model, but a set of values and principles. It is a flexible approach for Software development that helps organizations to adapt fast to the market change. We can’t talk about the Agile process or Agile model, we rather talk about Agile Practices. They are proven practices to organize teams, manage workflow, track progress, and deliverables, etc. so that Software projects are conducted the Agile way.

The most popular Agile practice is Scrum, which is often mistaken for Agile. Other practices are Kanban, Extreme Programming (XP), Feature Driven Development (FDD), Crystal, Test Driven Development (TDD), Dynamic Systems Development Method (DSDM), etc.

Scrum is today an established project management practice, with defined roles (Product Owner, Scrum Master, Scrum Team) and rituals (sprints, daily scrums, sprint retrospectives, etc.).

Image for post

For more details refer to Agility — Beyond Software Development.

References

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s