A recent report claims that one of the fundamental benefits of open-source development, the co-called Law of Many Eyes is wrong. The idea behind the law is that since anyone can read the source code and find problems with it, they can then either fix them or report them back to the community. The end result is that you get better software.

The study, by Fortify Software, a company that makes development tools for checking security, found that many popular open source software programs contain significant security holes. I can't take this study too seriously. After all, what else is Fortify going to say? "Open-source's Law of Many Eyes works great. You don't need our products?" I don't think so.

Here's what I think. I think the Law of Many Eyes, or as Eric Raymond phrased it in his seminal work on open source, The Cathedral and the Bazaar, "given enough eyeballs, all bugs are shallow," does work. All you need do is watch how quickly open-source projects progress and how quickly they fix bugs to know that.

However, there's nothing magical about the Law. Just because bugs are easier to find and fix, doesn't mean that all bugs are found quickly or fixed quickly. People who actually work in open source already know this.

For example, in late June, Mark Shuttleworth, Ubuntu Linux's founder, wrote in his blog that while Ubuntu is doing a good job of finding and fixing bugs both its own Linux distribution and with other associated programs like GNOME and Mozilla, it could do better. Shuttleworth wrote, "When I delved into the data to see how we do with pushing bugs upstream, I found a somewhat mixed picture. In many cases, we do very well indeed. We have a very good relationship with GNOME, for example, with a very high percentage of bugs appropriately forwarded to the relevant upstream bug tracker. In other projects, it's harder to make a definitive statement. The percentage varies based on whether the Ubuntu team members have good relationships upstream, or whether there's a person acting as an ambassador from Ubuntu to upstream.

In other words, cross-project communications can be a problem. If one set of eyes sees a problem, but what they see is never reported to a project's developers, then the Law of Many Eyes is less effective.

Shuttleworth concluded, "We need to improve the tools that support these kinds of cross-project conversations. Launchpad [a set of integrated development tools] that support collaboration and community formation] does currently allow us to track the status of a bug in many different bug trackers, and there are quite a few distributions and upstreams that are now either using Launchpad directly or exchanging data efficiently. We'll keep working to improve the quality of exchange across the whole ecosystem, including those projects that don't use Launchpad themselves."

I could cite other examples, but the point is that open-source developers already know that the Law by itself doesn't automatically make things better. It only works when it's actively used. Thanks to efforts like Launchpad, it does become easier to put the Law into practice.

At the same time, Fortify is right about one point. Open-source developers could better use of security debugging software. Some projects already do this. For example, I know that Samba, the CIFS (Common Internet File System) server, makes use of Coverity's code scanning tools. More open-source projects could, and should, make use of these tools.

The Many Eyes approach isn't broken. It just needs to be taken for what it really is, just an excellent, but not miraculous, way of fixing software. With platforms like Launchpad making it more effective and security bug checking software, we can expect to see better, more secure open-source programs. Software, I might add, that I expect will continue to be better than its closed-source counterparts.