The Power of the Open Source Periphery

Like a lot of people in the tech world, I try to read a variety of publications (both online and physical versions) to keep up with what is going on in the dizzying world of technology. Recently, I read a great article in a recent issue of Wired entitled Accept Defeat: The Neuroscience of Screwing Up. This article explains in some detail, with examples, the problem of 'failure blindness' in scientific research.

This phenomenon causes scientists and researchers to ignore anomalies in their results they didn't expect, which are therefore categorized as 'failures'. The article goes on to state that one of the best ways of overcoming this bias is to involve other people from outside of your immediate speciality in discussions about your results.

For me, this triggered the thought process of why it's important to encourage participation in your communities (Open Source or internal) from folks on the periphery of your project - non-specialists who have something to contribute that could be the key to solving a problem or advancing the state of the art. Being an effective community leader/shepherd/guide/manager is as much about how you enable 'outsiders' to contribute as it is allowing the core group to work effectively together.

The Wired article has a great example of how two different groups of scientists went about solving a thorny issue in their research efforts. The story centered around two different labs studying E. coli, and a problem they had with keeping proteins they were measuring from sticking to a filter.

“One of the labs was full of people from different backgrounds. They had biochemists and molecular biologists and geneticists and students in medical school. The other lab, in contrast, was made up of E. coli experts. They knew more about E. coli than anyone else, but that was what they knew..."

"The E. coli group took a brute-force approach, spending several weeks methodically testing various fixes. It was extremely inefficient, and they eventually solved it, but they wasted a lot of valuable time..."

"The diverse lab, in contrast, mulled the problem at a group meeting. None of the scientists were protein experts, so they began a wide-ranging discussion of possible solutions. At first, the conversation seemed rather useless. But then, as the chemists traded ideas with the biologists and the biologists bounced ideas off the med students, potential answers began to emerge. After another 10 minutes of talking, the protein problem was solved..."

Open Source has long had a maxim that 'with many eyes, all bugs are shallow'. I think it's time we extend that idea to include general problem solving and new features/directions in both internal and external Open Source efforts. To that end, remember to encourage and nurture that 'outsider' mentality in your project - don't be afraid to step back and see things from a different perspective. Note that this kind of paradigm shift can happen in closed 'internal' projects, but given the open and dynamic nature of the way Open Source projects/communities function, it should be easier to implement this culture there.

A big part of the power of the Open Source ethos is meritocracy, and checking of egos at the door (as much as possible). Embracing the 'periphery' of contributors can push innovation and productivity to new levels, establishing a positive feedback loop which encourages more collaboration and ultimately, better software.