The Rapidly Increasing Scope for Software Quality Improvement

The nature of a computer is fundamentally different than that of any other manufactured device. No other device has software. This is a game changer.

For example, the headline in msnbc today reads:

Cars of today are essentially computers on wheels — and that’s a problem. Locating the source of electronic glitches is like “looking for a needle in a haystack.” Full story.

But the story doesn't mention another important consequence. In Alan Cooper's book The Inmates are Running the Asylum, he poses the following questions:

What do you get when you cross a computer with an airplane?
Answer: A computer.
What do you get when you cross a computer with a camera?
Answer: A computer.
What do you get when you cross a computer with an alarm clock?
Answer: A computer.
What do you get when you cross a computer with a car?
Answer: A computer.
What do you get when you cross a computer with a bank?
Answer: A computer.
What do you get when you cross a computer with a warship?
Answer: A computer.

The point Cooper is making is that software quality assurance (SQA) is becoming increasingly more important. And just because Toyota knows how to make a quality car, it does not mean they also know how to make a quality computer. And their failure to fully appreciate that cars are now "computers on wheels" is costing them enormously. Perhaps even the existence of the company.

This enormous risk potential is there for almost every manufacturer. Worse(?), science and even mathematics are becoming more and more dependent on computer software. For example:

What do you get when you cross climate science with a computer?
Answer: A computer.

Our confidence in climate forecasts depend in an ever increasingly fundamental way on how well they do their climate model software quality assurance.

The Measure of Software Quality Improvement

It seems there are many definitions for the term "software quality". On the bright side, this can be liberating. I can add my own definition:

Software quality is the degree of belief that the code will actually execute its intended functions, in the intended operating environment, for the intended life-cycle, without significant error or failure.

Notice that quality is subjective. It is a degree of belief, a Bayesian probability. "A measure of a state of knowledge." This is important for IVV since it means there is a big difference between "software quality" and "consensus software quality."

But that's another topic. The topic here is the proper measure of software quality improvement. (Engineers like quantitative measures and making things better.) So here I would like to note that improving software quality requires conditioning a Bayesian prior. But this, in turn, requires new evidence. The more new evidence presented and the stronger the evidence, the more software quality improves.

This means that the sole test of software quality is, not surprisingly, testing. Testing provides an objective foundation for quality that keeps quality's subjectivity from (hopefully) getting too far from reality.

But that's another topic too. Here I am noting that, IMHO, the strength, power, or intensity of new evidence is best measured in decibels. Bayesian probability ratios are often expressed in decibels. For example, see How Loud is the Evidence?

The decibel is a log scale that simplifies overall power gain/loss calculations and is convenient for ratios of numbers that differ by orders of magnitude. Additionally, log scales have been useful in describing the potential of certain physical phenomenon (e.g., earthquakes) and human perceptions (e.g., hearing). Thus, log scales can be useful when performing risk assessments and other related quality assurance activities.

For more information on evidence measured in decibels, see Chapter 4 of Jayne's Probability Theory: The Logic of Science.

Finally, an analogy. If evidence of software quality is measured in decibels, it suggests software quality assurance can be thought of as singing the quality song about the software. Consensus software quality then is where we all sing the same song, or at least sing our part in a symphony. :-)

(One of my complaints about the state of quality of say, climate science, is that every participant feels she/he must carry the melody.)