Let's start off with the obvious: Healthcare.gov, as a web service, is a catastrophe.
Much has been written about CGI, the firm responsible for the site's backend code, and about the government's procurement process leading to CGI's selection. It is absurd that a project so vast—three years in the making with tens of millions spent already—could fail in such preventable ways.
The anger and frustration over the massive bungling of such a vital project threaten to make us lose proper perspective and learn the wrong lessons, however. So let's revisit the basics: What went wrong, how do we do better, and how do we think about government's fundamental capabilities?
From an end-user and technical perspective, a federal health insurance marketplace isn’t so different from a standard e-commerce website. But most of the work involved in building a site like this is not technical; it's all about client management. The project spanned numerous seemingly unrelated federal agencies, and many individual states, with a large set of specifications. These stakeholders undoubtedly provided conflicting and contradictory requirements.
Effective web startups think in terms of "agile development" and building a "minimum viable product." We experiment with bite-size pieces of a product to validate our technology and our market assumptions. If something isn't working, we make adjustments. But how can you be flexible when your final product is bound on all sides by federal law and agency regulations? What are the bite-sized pieces and optional components for an overhaul of an entire nation's health care system?
We would love to believe that a modern Silicon Valley-style tech company could have been hyper-efficient in building the equivalent of Google for health care. In today's reality, for better or worse, this is a technocratic fantasy. Outdated federal procurement practices and politicking certainly limited the set of companies considered to manage this project, but the government would still have been faced with a challenge even without these issues. The intersection of companies that produce genuinely good technology, and companies that are willing and able to navigate the built-in bureaucratic complexity of a project like this, is vanishingly small. The truth is that good software developers don't want to deal with the headaches of bureaucracy, and they have enough appealing options that they don't have to.
The complexity of this project was defined by the law itself. Perhaps one of the Republicans' most effective arguments against the bill was that it was poorly understood. A single-payer system (Medicare for all) would not only have been a simple thing to understand, it would have been an order of magnitude easier to implement. Sometimes the simplest solutions really are the best.
Even considering the inherent complexity of the project, the scope of problems with Healthcare.gov point to a staggering degree of technical incompetence and poor management. Certain aspects of the site, including the design and some of the front-end code, were completed effectively. But much of the back-end code, and the communication between the various application layers, has been fraught with bugs and scaling issues.
Perhaps most significantly, Healthcare.gov requires users to wait for the application to interact with numerous third-party legacy software systems, run by various federal and state agencies and insurance companies. This design introduces numerous points of failure and bottlenecks. Not only should these constraints have been simplified or better accommodated from the start, but a system so complex needs extensive automated and human testing. It is not clear if there were any automated tests, and we've learned that manual testing was begun a mere two weeks before the site went live (and was unsuccessful, at that).
It doesn't need to work this way. Building software is fundamentally different from building physical goods. There is no need for factories and specialized equipment. Even co-location and transportation are irrelevant. Management processes that work well for producing physical goods achieve poor results when applied to software (consider the classic "waterfall" model). Government procurement and oversight practices ought to reflect these realities.
A single change can provide the most reliable defense against repeating the current situation: All public-facing government software should be open source.
The value proposition of open source is not a hypothesis—we all use open-source software every day. The vast majority of software powering the Internet at all levels—operating systems, web servers, application frameworks and browsers—are built in full public view. Open-source code is generally at least as secure as proprietary code (and there are excellent arguments, and some academic research, showing it is more secure, though it is not a panacea). Building software in the open doesn't magically make it better, but we know that it can be done very effectively, and it provides a context of transparency and accountability. Effective, transparent, and accountable—isn't that exactly what we want from the government?
A black-box codebase allows everyone to make excuses. The administration can blame the contractor for bad technology, while the contractor can blame ever-changing guidelines. When development is done through open source, the world can see the history of what really happened. The developer community is notoriously curious and engaged. There is no chance the current implementation of Healthcare.gov could have been built out in the open without the entire country being aware of the debacle long before it went live. It is clear why the administration and private contractors would want to avoid that sort of accountability, but this is exactly the accountability that we, as citizens, ought to demand.
Despite the challenges, government will, and must, continue to develop software. We are used to thinking of software as a product, but government software isn't about products—it's about infrastructure.
We are still in the first few decades of the Information Age. All aspects of our society—our social interactions, commerce, wars, and even our philosophies and morality—have been fundamentally altered by how we consume and produce information. The most successful companies in the world are those that best use information to create and service markets. And in the coming decades, the most successful nations will be those that most effectively use information to service the needs of their populations.