The Era of Epic Fails is Over
The intentions of public sector leaders to modernize and make government operations more transparent and efficient have been trampled not only by limited budgets and aggressive timelines but also by repeated project failures. This makes funding agencies and stakeholders question whether these projects are good investments to pursue and, if so, how best to mitigate risks and protect taxpayer dollars. Such struggling IT implementation initiatives are represented in many different sectors including health and human services, transportation, finance and administration, tax, and defense. The nature of the programs may be different but the underlying systems development approaches have been the same.
“The annual cost of failed U.S. Government IT Projects is estimated to be as high as U.S. $20 billion.”
As a result, government leaders are rethinking traditional system implementation methods and have begun encouraging the use of agile development for public-sector projects nationwide. Agile methodology is an incremental and iterative development methodology that allows the project team to respond to changing requirements and solutions by offering continual opportunities to evaluate and adjust the direction of the project. The concept is not new. In fact, Agile concepts have been used very successfully in a wide range of industries, for example, in manufacturing since the 1950s and in software development since 2001. Agile methodology boasts significantly lower development costs, on schedule deployment, greater end user satisfaction, and an increase in team productivity. The core components include user-centered designs, swift delivery, multiple iterations, flexibility, working product first and foremost, and self-organizing teams. The benefits of this approach have been proven in both the private and public sectors, and are likely to increase the IT implementation success rate across public agencies. This discussion reveals the need for a change in the IT project environment, explains how agile works and why it fits government.
Why Do IT Projects Fail in Government?
Among the numerous articles and reports discussing IT project failures, a wide array of reasons and possible explanations for the phenomenon are posited. It is most pertinent to highlight the contributing factors in order to point to a new approach to implementation that has a role in solving this expensive inefficiency. Some of the most common cited factors include:
- Changing requirements
- Unclear scope
- Project too large in scope
- Significant glitches and malfunctions eventually revealed during testing
- Overly monolithic approach
- Poor planning
- Staff turnover
- Leadership that lacks the necessary experience to lead such projects
The “big bang” approach for migrating from a legacy system to a new one encapsulates many of the above stated issues for why projects fail, including the enormous size and changing scope. It is a project that is contracted with a broad scope and a plan that spans several years. In theory, this approach is to allow for flexibility for large projects as the requirements become clear once the analysis phase begins. It also leaves room for evolution of IT advancements during the years of implementation.
In reality, scope creep occurs and the multi-year implementation is ultimately not able to keep up with the fast-paced progress of IT. Frequent direction shifts due to unclear scope primarily caused by evolving government regulations create tremendous delays. This schedule breakdown increases the already high cost of implementation and exacerbates the potential for the released system to be irrelevant.
A key factor to project breakdown is discovering system glitches and malfunctions during the testing phase; the final phase of the development cycle. While the nature of testing is to confirm the technology works, in many large government implementations, the errors are substantial and require significant rework. The troubleshooting and fixing substantive errors perpetuates delays and require extensions on expensive resource contracts for an often already behind schedule project, once again driving up costs without
A key factor to project breakdown is discovering system glitches and malfunctions during the testing phase; the final phase of the development cycle.
TRADITIONAL WATERFALL METHOD
The culmination of the identified factors in project impediments and failure is particularly prevalent in the traditional way that government approaches IT implementation – that is the waterfall method.
This traditional and familiar style tends to be approachable because it appears to be easier to understand, monitor, and procure for. It is organized into defined phases that are dependent on one another for progress. While the waterfall method is organized and comprehensive, it is also large, complex, and often slow, making course corrections difficult and costly, and response times significant.
Unfortunately, this method also perpetrates cramming in too much scope early in the project, bogging down the essentials later, making it difficult to deliver a working system on time. Additionally, the model of slowly moving from phase to phase and testing only at the end, inhibits teams from fixing issues early enough to reduce impact because of the late discovery.
Private Sector Methodology that Works for Government
With annual costs of $76 billion and project failure rates at nearly 50%, public agencies are incorporating a private sector methodology that has shown proven success, and is consistent with the goals and priorities of government (Computerworld; GAO). Agile is more than just a new set of skills and tools; it is a perspective and transformative approach to project delivery and implementation. It boasts substantial cost savings and delivery in a fraction of the time by focusing on keeping code simple, testing often, and delivering functional bits of the application as soon as they are ready.
- A comparison study (Rico,2008) between agile and traditional methods found agile projects rated significantly higher.
20% more successful in the area of cost
50% more quality
91% more on schedule
97% more productive
400% greater satisfaction rate
- Organizational agility is responsible for 25% more successfully completed projects.
- The Department of Defense reported an 87% improvement in their time to market utilizing agile for their Defense Medical Human Resources System internet.
More Than Just a New Set of Tools
The agile concept is not new. In fact, it dates back to the 1950’s with Toyota’s lean manufacturing. However, it took hold as a viable method in the technology world in 2001, in particular with the publication of the Agile Manifesto. The core values of the manifesto are simple:
INDIVIDUALS AND INTERACTIONS
over processes and tools
over comprehensive documentation
over contract negotiation
RESPONDING TO CHANGE
over following a plan
Agile promotes cross-team collaboration, business interaction contribution throughout the process, adaptability, continuous improvement, and quick delivery. Rather than fixed scope based on ambiguous design requirements and varying time and resources as seen in the traditional waterfall methods, agile promotes a variable scope with fixed time and resources. Contrary to traditional project management methodology and training, project managers must expect variable scope in order to be effective in an agile environment.
This difference allows for pivots and adjustments to be made as issues are discovered without disrupting the whole process, and also eliminates the need to wait until one phase is entirely complete before shifting to the next phase.
The Case for Agile in Government
The Federal government has made a significant effort in adopting, promoting, and utilizing agile as they modernize systems. In light of the costly mishaps that occurred in releasing HealthCare.gov, government agencies have turned to this alternative method because of its modular and significantly reduced time-to- value approach.
“To reduce the risk of such problems [incurred cost overruns and schedule delays while contributing little to mission-related outcomes], the Office of Management and Budget (OMB) recommends modular software delivery consistent with an approach known as Agile, which calls for producing software in small, short increments” (GAO-12-681, 2012).
Government sponsored digital projects generally require significant design and development based on complex business requirements, as well as the adaptability for future enhancements and compliance. Because of this, these projects will benefit from agile approaches.
This approach – used successfully in the private sector and at the Federal Government level… promises to deliver better technology that meets the needs of users quickly and frequently.
As with any tool, a one-size-fits-all approach is a flawed strategy, and agile is no different. The actual questions of its applicability are centered on understanding how to integrate this proven practice in a relevant way, being mindful of what aspects of the methodology will work within the government’s regulatory environment, governance structure, and bureaucracy. The Federal government has made efforts in the last several years to cultivate an environment that welcomes agile practices and has encouraged public agencies to prioritize them during procurement. To encourage effective utilization of agile, they developed guides such as the TechFAR Handbook and the Digital Services Playbook. These offer tangible and government applicable recommendations for incorporating agile development into procurement and implementation while navigating regulations.
The TechFAR Handbook’s includes overall government goals of modular contracting and agile software development.
Government’s Move Toward Adopting Agile
To help with these new initiatives, the General Services Administration (GSA) founded 18F, an office inside GSA run like a startup, devoted to transforming government technology practices and assisting agencies in creating successful solutions. As a major step toward bringing agile development and procurement into government, 18F established a GSA Blanket Purchase Agreement in 2015 for vendors who offer agile services and utilize an agile approach during development.
State governments, such as California, are following suit. California’s Health and Human Services Agency (HHS), partnered with Department of Social Services (DSS), 18F, and Code for America to produce a series of smaller agile style RFPs rather than a large “big bang” multi-year implementation. This partnership plays a crucial role in overhauling California’s legacy Child Welfare System (CWS).
“The business-as-usual procurement for a new Child Welfare System would be risky…It would probably be late, over budget, and lack functionality or optimal usability. These are the reasons why a new Child Welfare System would be the candidate for a new modular, agile approach to delivering government technology.” (Hon, 2015)
Examples of Successful Government Implementation Using Agile
As government continues to focus on increasing efficiency and transparency by updating their systems and practices, agile development provides a proven solution that reduces the rate of failed digital services projects, drastically lowers costs, and addresses the immediate and long-term usability needs of the end-user and general public.
Federal and state agencies cannot continue the status quo of the big-bang approach in today’s progressive information technology environment and continued refinement of regulatory compliance to meet evolving needs. Public confidence will continue to erode until project statistics reverse its current 90+ percentage failure rate and billions of dollars in lost expenditures.
Agile provides tangible benefits such as quicker releases, lower costs, flexibility, reduction of risks, and is a proven methodology used in the private sector for decades that works well for public agencies. This is already represented in government agile projects nationwide. Like any transformation, there are gaps of knowledge, learning curves, and traditional habits that will emerge as potential obstacles; however, the pay offs for moving in this direction are clear. Agile is a critical part of the future of government.
We hope this discussion will initiate further exploration of agile development and encourage more widespread usage when seeking viable solutions during procurement and project implementation.