Some Empirical Findings

There is a fairly recent American Scientist article that presents some of the latest empirical findings in software engineering. (Such stuff is important since the analysis of programming methodologies tends to be dominated by theoretical arguments.) The findings are not presented in list form. For convenience, here is a partial list. Please read the article for more detail and the authors' perspectives on the findings.
  • More lines of code mean more errors in the code, all other things being equal. (Such is the state-of-the-art in quantifying software quality!)
  • Pair programming is approximately as productive as solo programming. (That is, the useful output of a pair is roughly comparable to the output of two individual programmers.)
  • It is not possible to reliably identify great programmers based on their personality type. That is, there still is no personality test that can be used to skillfully identify outstanding programmers from mediocre ones. (By focusing on personality types, the authors of the article seem to imply, at least to me, that it is generally already known and accepted that in-depth knowledge of languages, design patterns, or standard algorithms; or having formal certifications or degrees from prestigious universities, won’t distinguish the outstanding from the merely mediocre either.)
  • Trying to imitate traditional engineering practices doesn’t have a positive return-on-investment. For example, it seems that UML tools, while useful for contemporaneous communication of ideas for the software design, are not worthwhile for generating or even documenting the actual software design. (As noted in my last post, this has been my observation being both a mechanical engineer and a software engineer. Traditional engineering and software engineering are two very different hats.)
  • Code imitates life: “The organization that designs a system will produce a design whose structure is a copy of the organization’s communications structure.” For example, if a programming project involves four teams from four different groups in the organization, the program created will have four major parts. And the probability of bugs increases with greater organizational (not physical!) separation between team members.
  • Open-source software does not produce software any less vulnerable to security flaws than closed-source software. (To me, this seems to imply that open-source software is not inherently any less bug free either.)

1 comment:

  1. George said: For example, it seems that UML tools, while useful for contemporaneous communication of ideas for the software design, are not worthwhile for generating or even documenting the actual software design.

    I've seen this play out recently. The implementation passed the test suite, but the test suite didn't match the design (because of an error in translating UML+friends into test code). Big design up front sounds great in theory, but if it doesn't get automatically turned into unit tests (for interesting levels of complexity it *must* be automatic), then it just adds another verification problem to the one you already had.

    ReplyDelete