What Works in Software Development?

Lately I've been reading a bit about software project management. This meshes with my college course on system analysis, but at a higher altitude. Anyone in the software business for long knows that conventional wisdom claims that most software projects fail spectacularly, and most of the rest fail in a less-spectacular manner. But somehow the industry trudges on.

Let's take another look at the landscape of software projects. For every giant-scale project out there, how many projects have been so small as to slip undetected from conventional wisdom? It depends in part where you draw the line between programming and not.

Constructing a spreadsheet is a form of computer programming. If you count up all the spreadsheets in the world, how many million-dollar failures can you count? Common sense points to a very small number -- perhaps none.

Similarly, it happens often in professional computing that a small job must be done with some regularity. This is a perfect opportunity for automation in the small. A competent programmer or system administrator tackles one tiny irritant, such as updating the files on a web server automatically, or making sure the back-up process doesn't accidentally overwrite yesterday's tape, or changing the format of a data file. These micro-jobs are measured in hours or days, not months or years. But they have an impact. They don't tend to fail. They free up labor for other tasks. And nobody notices.

There's a general rule we can infer.

Tackle the smallest possible feature that will add business value today.

Related link: http://www.stsc.hill.af.mil/crosstalk/2003/04/weinberg.html

* By Unix, I mean not just official Unixes, but all related remakes, reimaginings, and reboots, including at least Linux, Hurd, and Plan 9 from Bell Labs.