Large scale small projects, Part I

Integrating teams isn't an easy task. There's much written about it, and it's usually one of the major complaints in management. Software development teams are no exception to this rule. But until recently, most projects could be split in two relatively well-defined classes: small (and thus more manageable) projects, and big projects. Big projects impose such a management overhead that they can only be adequately funded and executed big big corporations, bigger in fact that the simple difference in project scale would suggest. Although projects of all size fails, failure in big projects means a much bigger loss, and a much more visible impact on its members.

For small software projects, intra-team communication used to be a relatively simple task. Everyone works together, so much of the necessary communication occurs using standard tools; live meetings, cheap phone calls, and the usual visit to a colleague's desk. Now, with globalization and cheap Internet connectivity, something new is happening. It's possible now to have a "large scale small project". This apparent contradictory name defines very well what's happening: it's a project which shares some of the complexity of a large scale project, but without having necessarily the same resources or the same contraints. Projects in this classification are usually big in at least one of the measuring dimensions: project scope, team size, and communication overhead. On the other hand, they are small by one defining dimension: the resources available to fund the project are minimal, or often, negligible (as in the case of an open-source project).

The dynamics of these new "large scale small projects" is challenging. Some teams are already handling big projects, the Linux kernel and the Apache Web Server being the most notable examples. Projects with a big scope often found innovative ways to structure themselves, taking advantage of the inherent modularity of the problem at hand. In the case of the Linux kernel, a good deal of the complexity lies in specific, highly-modularized parts of the system. Smaller teams working on these parts are being successfull at managing their own subprojects with good results.

Globalization also has a big impact. Open-source projects often attract developers worldwide. But it's becoming more and more common for developers to work remotely, even for close (paying) projects. This leads to a new kind of project, where many of the participants never meet face to face (with the notable exception of a few projects which are able to organize and host annual conferences). For projects with thousands of participants, mailing lists are the norm. Bigger projects also have an additional advantage of greater visibility, which creates a sustained momentum that helps the project to keep going.

Small projects with geographically dispersed members are the most difficult case to deal with. These projects have the inherent complexity of a much bigger one, but with less resources. The availability of cheap Internet connectivity helps a lot; tools such as text-based messaging, using IRC or ICQ, and cheap voice communication software such as Skype can greatly improve the efficiency of the communication at unbeatable cost. On the other hand, there's little than can be done as far as timezone differences are concerned. Linguistic and other cultural barriers also play a role here. Some cultures are renowed for a formal and introspective approach to work; personal aspects are entirely left beside the working environment. Other cultures prefer a more relaxed expansive approach, and talking about personal or family stuff is entirely acceptable, as long as it does not cause any loss of productivity.

Once you get it all running - overcoming differences in timezone or cultural traits, for example - there's still a huge task left to do: how to effectively share the knowledge inside a distributed working environment. This is a great undertaking, big enough to be left for a coming article...