Waterfall based development methodologies are rapidly being replaced by Agile ideologies. Programmers and programming teams wonder what this means for developing their skill sets and tackling the vast array of projects coming down the pike with Agile methodologies. Businesses are wondering if they should build Agile teams to meet the challenges of the future.

Until recently, the waterfall methodology of development and deployment had taken a strategic hold in the majority of enterprises. Enterprises have moved into the world of Agile, where different concepts are leveraged and the development process is handled in a completely different fashion. Agile development ideologies are proving to be a better way to deal with rapidly changing objectives and speeding the time-to-market for business applications, services and other customer- (or internal-) facing offerings. The ideas behind Agile extend beyond basic application development, and can benefit numerous projects by improving the process to achieve deliverables and garner improved productivity.

The birth of Agile.

Agile was created in February 2001, when 17 software developers met in Utah to discuss development methods. These developers were bogged down by documentation driven software development processes and the silos created by these methods. They wanted to encourage collaboration and the sharing of ideas. The result of this gathering was the Manifesto for Agile Software Development.

Manifesto for Agile Software Development

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

• Individuals and interactions over processes and tools.
• Working software over comprehensive documentation.
• Customer collaboration over contract negotiation.
• Responding to change over following a plan.

That is, while there is value in the items on the right, we value the items on the left more.  To break this Manifesto down further:

• Individuals and interactions: self-organization and motivation are important, as are interactions like co-location and pair programming.
• Working software: working software is more useful and welcome than just presenting documents to clients in meetings.
• Customer collaboration: requirements cannot be fully collected at the beginning of the software development cycle, therefore continuous customer or stakeholder involvement is very important.
• Responding to change: Agile methods are focused on quick responses to change and continuous development.

Agile defined further with methodologies.

Without defined processes, methodologies and steps, Agile became difficult to implement. This led to the creation of more specific methods in the Agile movement. The Scrum Methodology is the most adopted methodology used to implement Agile development projects. Scrum emphasizes collaboration, functioning software, team self-management, and the flexibility to adapt to emerging business realities. In this management framework for incremental product development, Scrum teams create product increments during fixed-length iterations called Sprints, which are typically 1-2 weeks. These increments and iterations allow feedback to be incorporated more quickly than it could be with waterfall development.

Other Agile methodologies include:

• Extreme Programming (XP): Encourages a workflow in which the customer works closely with the development team to prioritize needs (referred to as “User Stories”). Requires close teamwork and continuous testing to deliver software at frequent intervals, typically every 1-3 weeks.
• Kanban: Promotes continued collaboration by emphasizing continual delivery while not overburdening the development team. This methodology emphasizes use of the best possible team workflow, using visualization to determine what this is.
• Scrumban: Combination of Scrum and Kanban that uses Scrum iteration and Kanban processes.
• Lean: Focuses on delivering value to the customer and the efficiency of the “Value Stream” (mechanisms that deliver “Value”). Eliminates waste by focusing on feedback from small teams as opposed to a hierarchy and delivering prioritized features in small batches.
• Feature Driven Development (FDD): Model-driven, short-iteration process that works in two-week iterations to design each feature.
• Dynamic Systems Development Method (DSDM): Works off of requirements and prioritization. Requirements are planned early in the project and are delivered in short iterations. Rework is a standard consideration and all development changes must be reversible.

There are several different Agile methodologies, including Scrum, XP, Kanban, Crystal and a few others, with Scrum having the lionshare of the market.

Waterfall vs. Agile

The waterfall development model uses an approach that relies on a defined sequence of accomplishments to derive an endpoint. As the name implies, a development product’s life cycle flows steadily towards completion, much like how water travels down a waterfall.

The waterfall development method has its roots in industries, ranging from construction to assembly lines, in which sequential processes are the key to success. Waterfall methodologies also took hold in the early days of programming, where changes during development projects were costly or impractical. Applying a waterfall methodology to a programming project meant that all the requirements gathering and design work was done before any coding took place.

Although the ideology behind waterfall isn’t old, it was first described in the software development realm by Winston Walker Royce, who wrote a paper in 1970 explaining the methodology applied to software development projects. Waterfall has been the leading methodology for software development projects since its introduction and in most cases, adheres to the seven-step process outlined in Royce’s 1970 publication.

The Agile development methodology eschews a linear, sequence-based approach, and uses an incremental, iterative approach. Agile-based projects do not start with extensive planning and design, which means changing requirements can be incorporated into the development process. Cross-functional teams become an important piece of the Agile method puzzle. They incorporate planners, designers, developers and testers working in concert to create success iterations of the product. Those iterations are completed over fixed time periods (referred to as timeboxes) with the goal of creating a working product, which can be demonstrated to stakeholders for feedback, comments and change requests to introduce in future iterations.

Why Agile methodologies work.

To weigh the differences between waterfall and Agile methodologies, it is critically important to know how their characteristics can be interpreted based upon an individual’s point of view. For example, a software developer may find the idea of flexibility and reduced test cycles a pro, while a project manager may consider those two elements a con.

The most common pain points of waterfall methodologies include: long waits for releases, inability to make changes throughout the development process, and having to completely restart the development process due to changes in needs. Fortunately, these can be resolved with Agile methodologies.

The driving forces behind Agile methodology adoption.

Agile methodologies are becoming the go-to ideology to embrace productivity while allowing businesses to react to changes, market demands, and shifting priorities.

Agile methods offer more flexibility than waterfall derived projects. Agile is also customer facing, meaning that customer requests can be implemented on the fly, allowing the creation of software that adapts as needed and can change with each iteration to meet new challenges. For example, if a company using waterfall methodologies is developing software with a green background, then decides they need a blue background, this would be a major delay for the project. Using an Agile methodology, this would be a small change seamlessly integrated into development. The Agile methodology offers several perceived benefits, which is why the adoption of it as a development practice is growing. The benefits include:

• Reduced Waste: Continuous improvement proves to be a faster process, reducing waste in the form of time spent on design, redesign and addressing change.
• Increased Speed: Agile’s ability to bring forth rapid change, especially in the form of course corrections, means that deliverables can be constructed quicker.
• Improved Productivity: Agile is a more effective way for teams to work together, meaning that productivity will increase due to improvements in communications, as well as insight quickly garnered by cooperative goal setting.
• Enriched Decision Making: Agile teams share wisdom and help to drive faster decisions based upon needs and not original design goals.
• Improved Confidence: With better decisions and a proactive approach, Agile improves the confidence of an organization. It makes it easier for the product to pivot, which allows the organization to react better to changing market conditions.
• Improved Trust and Safety: Agile has a different approach toward failure: it emphasizes learning from mistakes and leverages a blameless post-mortem, which improves the attitude towards failures. Ultimately, learning from failures and eliminating blame leads to improved trust among team members.

An HP online survey of 601 development and IT professionals shows that 51 percent of respondents are leaning towards Agile, while another 16 percent are already using Agile as their sole development method. Only 2 percent surveyed were pure waterfall environments, while only 7 percent were leaning towards waterfall. This is a clear indication that Agile is taking hold and is quickly becoming the new normal.

The path to Agile.

Adopting an Agile ideology is more than just claiming teams are Agile. Managers and decision makers should understand the components of what makes up an Agile ideology, especially when Scrum is the target ideology. One of the biggest struggles businesses encounter when shifting to Agile is the cultural shift. While having an Agile evangelist is an effective first step, the need to permeate the culture requires several major shifts.

• Goals shift from making money to customer satisfaction.
• Structure shifts from management overseeing to management enabling self-organized teams to contribute all they are capable of.
• Workflow shifts from bureaucratic to cyclical with direct customer feedback.
• Values shift from efficiency and predictability to transparency and continuous improvement.
• Communication shifts from hierarchical to lateral.

Scrum not only outlines the process, but also defines the roles of those involved. A Scrum team may consist of:

• Product Owner: Shares the vision of the product, prioritizes the functionalities to be built, and makes key decisions on behalf of the team. The Product Owner is also responsible for maintaining the product backlog, bridging the gap between the developers and other stakeholders, managing the end-user (or customer) expectations, and managing the budget.
• Scrum Master: Primarily responsible for solving problems team members may be facing while building the product. The Scrum Master does not have to completely understand all requirements, but must still be able to find solutions to situations. Ultimately, the Scrum Master is responsible for creating the best possible working conditions for team members so they can meet the goals of each sprint effectively.
• Scrum Development Team: A cross-functional team that is responsible for developing the product. The team consists of developers, business analysts, testers and other liaisons. The goal of the team is to work together and in tandem while building the application. Team activities are aligned with the targets associated with the specific sprint. Team members must identify the complexity of assigned tasks and allocate the appropriate resources. Team members are expected to communicate the status of project and issues they are facing to Scrum Masters on a daily basis. Team members also are responsible for demonstrating tasks completed to product owners during sprint reviews.

Agile Terms and Ideologies

Product Backlog:A living document that contains the features desired in a product, as well as the bugs, technical work, and acquired knowledge. The product backlog is managed by the Product Owner, and is continually updated and prioritized.

Sprint: The developer team works in sprints, a classification used to describe an iteration task set. At the start of the sprint, the developers and the Product Owner agree on the items from the backlog they will complete during the sprint. That subset of items from the Product Backlog becomes the Sprint Backlog. During the sprint, the Product Owner is not allowed to change the items in the Sprint Backlog.

Daily Scrum: During a sprint, the developer team meets to coordinate work for completing the items in the Sprint Backlog. Ideally, the team will discuss who is working on what and whether any blocking issues have been discovered. Meetings are kept short and simple, consisting of what someone has done since the last meeting, what is planned for today, and whether there are any impediments. The idea is to share knowledge directly related to the project.

Stories and Tasks: Items in the Product Backlog or Sprint Backlog are referred to as stories. Stories are created by the Product Owner and represent a specific business need. Stories are often broken down into specific tasks so accomplishments can be measured. Without defined tasks, stories can become never-ending stories simply because there are no measurable milestones. Stories are owned by the Product Owner and are all about business value, while tasks are owned by the developer team and are all about implementation details. A story might take several days or weeks to complete; a task is something a developer can complete in less than a day.

Burndown Charts: In Scrum, Burndown charts are used to represent the remaining work on a project. A Release Burndown is built calculating the remaining number of uncompleted Story Points for the entire Product Backlog every day, while a Sprint Burndown chart is used to identify remaining work on a particular sprint.

Agile success stories.

HealthFitness

The workplace health company implemented Agile methodologies to create a new portal. HealthFitness was losing potential and existing clients due to an outdated online portal tool. They needed to update their technology quickly while having the option to consistently optimize it. In 2012, they assembled Scrum teams to develop a new portal. The portal is now launched and, as of the end of 2015, maintains 50 of the company’s clients.

Spotify

The music streaming service used Agile methodology to scale their development teams. Spotify has adopted Agile since their launch. They got back to the spirit of flexibility and experimentation at the foundation of Agile and created their own Agile methodology to meet the needs of their large development team. Spotify’s teams are called “squads” and each has autonomy to complete their responsibilities, which are start-to-finish, from design to deployment and continued improvements. They have scaled their Agile methodology to 30 teams in three locations around the world, supporting more than 75 million users and 1.5 billion playlists across 58 markets.

Cerner

The health information technology company transitioned to Agile methodologies to accelerate time-to-market. It typically took Cerner’s development teams 30 months to introduce product innovations. In the era of rapid healthcare reform, the company realized the need to improve their go-to-market time to stay competitive. Their transition to Agile resulted in a 75 percent reduction in time-to-market, 24 percent increase in productivity, 14 percent decrease in development costs, and 50 percent reduction in critical defect resolution turnaround time.

FBI

After almost a decade of mismanagement and waste at the FBI, its CTO turned the agency’s maligned case management implementation into an Agile project. When the tragedy of 9/11 struck the United States, the FBI’s only secure means of transmitting photos of the hijackers were faxes and overnighted CD-ROMs. The need to digitize was made painfully obvious not only to the bureau, but to the nation as a whole. As a result, the Virtual Case File (VCF) project was launched in 2000 to usher the bureau into a new era of case management. After floundering for five years and costing the federal government nearly $170 million, the project was scrapped for its inefficiency.

Some of the reasons cited for the project’s failure include poor technical architecture, repeated changes in specifications, and micromanagement. In 2006, the FBI revived their efforts with the Sentinel Project using waterfall methodology to manage the project. By 2010, they had only completed two of the project’s four phases and spent $405 million of the $451 million budget. The bureau’s new CTO transitioned the project management style from waterfall to Agile upon being briefed on the project’s performance in 2010. After the transition, the project was complete within 12 months for $30 million – a cost savings of more than 90 percent.

Your path to Agile starts now.

It’s time to begin your transition to the next stage of Agile implementation. At Modis, we help you make exceptional connections to find the Agile-versed talent you need. For more information about Agile or to find top tech talent for your company, contact your local Modis representative today.

Sources

• Agile Manifesto. History: The Agile Manifesto. http://Agilemanifesto.org/history.html
• Manifesto for Agile Software Development. http://Agilemanifesto.org/
• Version One. The 10th Annual State of Agile™ Report. https://versionone.com/pdf/VersionOne-10th-Annual-State-of-Agile-Report.pdf
• Scrum Methodology. http://scrummethodology.com/
• Version One. Agile Methodologies. https://www.versionone.com/Agile-101/Agile-methodologies/
• SolutionsIQ. What is Scrumban? http://www.solutionsiq.com/what-is-scrumban/
• Hewlett-Packard. Agile is the new normal: Adopting Agile project management. http://www8.hp.com/h20195/v2/getpdf.aspx/4AA5-7619ENW.pdf?ver=1.0
• Forbes. How To Make The Whole Organization Agile. http://www.forbes.com/sites/stevedenning/2015/07/22/how-to-make-the-whole-organization-agile/#55e4d665135b
• ReQtest. How Spotify does agile – A look at the Spotify engineering culture. http://reqtest.com/blog/how-spotify-does-agile-a-look-at-the-spotify-engineering-culture/
• Harvard Business School. The Spotify ‘Squad’: How to successfully lead a global organization WITHOUT an operations team. https://rctom.hbs.org/submission/the-spotify-squad-how-to-successfully-lead-a-global-organization-without-an-operations-team/
• Version One. Cerner Case Study. https://www.versionone.com/our-customers/case-studies/cerner/
• CIO. How the FBI Proves Agile Works for Government Agencies. http://www.cio.com/article/2392970/Agile-development/how-the-fbi-proves-Agile-works-for-government-agencies.html
• Congressional Testimony, US DOJ Inspector General Glenn A. Fine. https://oig.justice.gov/testimony/0502/final.pdf
• Software in 30 Days: How Agile Managers Beat the Odds, Delight Their Customers, And Leave Competitors In the Dust. Case Study: The FBI’s Sentinel Project. http://my.safaribooksonline.com/9781118240908/navpoint-10?link=106b6df7-7b31-425e-a539-e02c3ec13fb4&cid=shareLink