Issue #16 February 2006

Open source development: The diversocracy

by Elliot Lee

It is common knowledge that the free/open source software development community is structured as a meritocracy. A meritocracy is a system of government which allocates power according to ability (merit) rather than other factors such as wealth, social position, or number of votes. The idea that open source is a meritocracy has become embedded in the community's collective consciousness.

But is open source development really a meritocracy? We've made that assumption so long without really examining its truth. In a perfect world, a meritocracy hands the reigns to the best people who will produce the best outcome in each situation. In the real world, though, only history can judge what the best outcome is, and only psychics claim to know authoritatively which people are the best for each task. Any real world organization can only approximate a meritocracy at best.

It's also fairly common to assume that within the open source world, a meritocracy means that the best technical solutions win. Ask GNOME and KDE advocates which desktop is technically superior, and you'll realize that there are as many 'best technical solutions' as there are users. Each user has a unique set of needs for which a unique solution is appropriate. The success of open source projects hinges not on the abstract idea of technical superiority, but on the ability of the project leaders to balance all the constraints, including technological trends, users' needs, and available resources.

There's no doubt that contributors to open source projects aim to make good decisions, and that in the long run smart people wind up being given the most power. Yet this pseudo-meritocracy doesn't explain the success of the open source development model. Many proprietary software projects can also claim at least some of these characteristics. What is the real advantage of the open source development model, then? The essence of its advantage, one that cannot be touched by proprietary software, is that the contributors are numerous and diverse.

The development of open source software is a meritocracy insofar as it ignores irrelevant differences between people. The color of someone's skin, their age, their social status, their location--none of these affect the opportunities for contributors to make a difference. Proprietary software companies, on the other hand, have to deal with situations like the one where a team of US programmers, fearing further outsourcing, stonewalled requests from a team of Indian contractors. Many software companies do not even understand how to do distributed software development at all, a skill that is a key success factor for open source development. This factor is simple but crucial: by opening up the software development process to the scrutiny of the world ensures that contributors focus on results instead of irrelevant differences such as status. The effectiveness of the open source development model is only partially explained by its similarity to a meritocracy, however. The real power of the model is in the diversity that a meritocracy facilitates.

Those familiar with the world of finance will have heard of portfolio diversification. The idea is to invest in a wide range of assets that represent all possible investments. By doing this, you can expect long term risk-adjusted investment returns that are superior to the returns from investing in a single asset (such as Microsoft stock). This theory can be summarized by an old saying: don't put all your eggs in one basket!

A similar concept can be applied to software development. Proprietary software companies invest all their resources in relatively few projects with relatively well-defined target markets. Open source development diversifies the world's collective software development resources, reducing the risk of failure.

Remember the Insight desktop project? How about UDE? Fox? Those of us who have spent time within the open source community sometimes deride projects like these because they aim to provide much of the same functionality as GNOME and KDE. Yet it is much better that we've a few great desktops and a few peripheral ones, than for us to have risked everything by having everyone work on a single desktop which could have turned out a failure. The ease with which the failing projects can be abandoned as their shortcomings become apparent, and the knowledge gained through these failures, makes the successful projects even stronger.

Moreover, because every user has a slightly different set of needs, open source can reach more users than proprietary software by having an abundance of offerings in the same space. No sane corporation would think of offering three products that are nearly identical. Yet open source developers do this every day and get away with it! Yes, there are drawbacks to this approach, and there is certainly a time and place for standardization, but we have made the conscious decision to accept the drawbacks, in exchange for the benefits of this culture of collaboration. The future of the open source community hinges on its ability to embrace diversity and deal with the drawbacks. Yes, users have problems making choices when there are so many, and sure, it's a pain to deal with flame wars and infighting between various projects, but we as a community are learning to deal with these problems, and it's clear that the open source vision is a healthier and more robust future than the one where a few large software companies decide what software people will use based on profitability projections.

Is open source a meritocracy? It doesn't matter. Open source is probably more of a meritocracy than most other human endeavors, but that is not its real power. Its real power comes from its ability to include a diversity of contributors around the world.

For another perspective on open source as a meritocracy, see Anca Metiu's research.

About the author

Elliot Lee has been a software engineer at Red Hat since 1996. His open source contributions include release engineering for Fedora Core, co-founding the GNOME project, and maintaining assorted open source libraries and utilities. He currently is leading out in work on Fedora infrastructure and management.