Preface |
|
xvii | |
Acknowledgments |
|
xix | |
Authors |
|
xxi | |
|
PART I Fundamentals of Measurement and Experimentation |
|
|
|
Chapter 1 Measurement: What Is It and Why Do It? |
|
|
3 | (22) |
|
1.1 Measurement In Everyday Life |
|
|
4 | (7) |
|
1.1.1 What Is Measurement? |
|
|
4 | (3) |
|
1.1.2 "What Is Not Measurable Make Measurable" |
|
|
7 | (4) |
|
1.2 Measurement In Software Engineering |
|
|
11 | (6) |
|
1.2.1 Neglect of Measurement in Software Engineering |
|
|
12 | (2) |
|
1.2.2 Objectives for Software Measurement |
|
|
14 | (1) |
|
|
14 | (1) |
|
|
15 | (1) |
|
1.2.3 Measurement for Understanding, Control, and Improvement |
|
|
16 | (1) |
|
1.3 Scope Of Software Metrics |
|
|
17 | (5) |
|
1.3.1 Cost and Effort Estimation |
|
|
18 | (1) |
|
|
18 | (1) |
|
1.3.3 Quality Models and Measures |
|
|
19 | (1) |
|
|
19 | (1) |
|
|
20 | (1) |
|
1.3.6 Structural and Complexity Metrics |
|
|
20 | (1) |
|
1.3.7 Capability Maturity Assessment |
|
|
20 | (1) |
|
1.3.8 Management by Metrics |
|
|
21 | (1) |
|
1.3.9 Evaluation of Methods and Tools |
|
|
21 | (1) |
|
|
22 | (3) |
|
|
22 | (3) |
|
Chapter 2 The Basics of Measurement |
|
|
25 | (62) |
|
2.1 The Representational Theory Of Measurement |
|
|
26 | (14) |
|
2.1.1 Empirical Relations |
|
|
27 | (5) |
|
2.1.2 The Rules of the Mapping |
|
|
32 | (1) |
|
2.1.3 The Representation Condition of Measurement |
|
|
33 | (7) |
|
2.2 Measurement And Models |
|
|
40 | (11) |
|
2.2.1 Defining Attributes |
|
|
42 | (2) |
|
2.2.2 Direct and Derived Measurement |
|
|
44 | (3) |
|
2.2.3 Measurement for Prediction |
|
|
47 | (4) |
|
2.3 Measurement Scales And Scale Types |
|
|
51 | (10) |
|
|
53 | (1) |
|
|
54 | (2) |
|
2.3.3 Interval Scale Type |
|
|
56 | (2) |
|
|
58 | (1) |
|
2.3.5 Absolute Scale Type |
|
|
59 | (2) |
|
2.4 Meaningfulness In Measurement |
|
|
61 | (17) |
|
2.4.1 Statistical Operations on Measures |
|
|
65 | (3) |
|
2.4.2 Objective and Subjective Measures |
|
|
68 | (2) |
|
2.4.3 Measurement in Extended Number Systems |
|
|
70 | (5) |
|
2.4.4 Derived Measurement and Meaningfulness |
|
|
75 | (3) |
|
|
78 | (9) |
|
|
79 | (5) |
|
|
84 | (1) |
|
|
85 | (2) |
|
Chapter 3 A Goal-Based Framework for Software Measurement |
|
|
87 | (46) |
|
3.1 Classifying Software Measures |
|
|
87 | (13) |
|
|
91 | (1) |
|
|
92 | (1) |
|
3.1.2.1 External Product Attributes |
|
|
93 | (1) |
|
3.1.2.2 Internal Product Attributes |
|
|
93 | (2) |
|
3.1.2.3 The Importance of Internal Attributes |
|
|
95 | (1) |
|
3.1.2.4 Internal Attributes and Quality Control and Assurance |
|
|
96 | (1) |
|
3.1.2.5 Validating Composite Measures |
|
|
96 | (2) |
|
|
98 | (1) |
|
3.1.4 Change and Evolution |
|
|
99 | (1) |
|
3.2 Determining What To Measure |
|
|
100 | (10) |
|
3.2.1 Goal-Question-Metric Paradigm |
|
|
100 | (5) |
|
3.2.2 Measurement for Process Improvement |
|
|
105 | (3) |
|
3.2.3 Combining GQM with Process Maturity |
|
|
108 | (2) |
|
3.3 Applying The Framework |
|
|
110 | (7) |
|
3.3.1 Cost and Effort Estimation |
|
|
110 | (2) |
|
3.3.2 Productivity Measures and Models |
|
|
112 | (1) |
|
|
112 | (1) |
|
3.3.4 Quality Models and Measures |
|
|
113 | (1) |
|
|
113 | (1) |
|
3.3.6 Structural and Complexity Metrics |
|
|
114 | (1) |
|
3.3.7 Management by Metrics |
|
|
115 | (1) |
|
3.3.8 Evaluation of Methods and Tools |
|
|
116 | (1) |
|
3.4 Software Measurement Validation |
|
|
117 | (4) |
|
3.4.1 Validating Prediction Systems |
|
|
117 | (2) |
|
3.4.2 Validating Measures |
|
|
119 | (1) |
|
3.4.3 A Mathematical Perspective of Metric Validation |
|
|
120 | (1) |
|
3.5 Performing Software Measurement Validation |
|
|
121 | (6) |
|
3.5.1 A More Stringent Requirement for Validation |
|
|
122 | (2) |
|
3.5.2 Validation and Imprecise Definition |
|
|
124 | (1) |
|
3.5.3 How Not to Validate |
|
|
125 | (1) |
|
3.5.4 Choosing Appropriate Prediction Systems |
|
|
126 | (1) |
|
|
127 | (6) |
|
|
129 | (2) |
|
|
131 | (2) |
|
Chapter 4 Empirical Investigation |
|
|
133 | (50) |
|
4.1 Principles Of Empirical Studies |
|
|
134 | (11) |
|
4.1.1 Control of Variables and Study Type |
|
|
135 | (4) |
|
4.1.2 Study Goals and Hypotheses |
|
|
139 | (2) |
|
4.1.3 Maintaining Control over Variables |
|
|
141 | (2) |
|
4.1.4 Threats to Validity |
|
|
143 | (1) |
|
|
144 | (1) |
|
|
145 | (26) |
|
4.2.1 A Process Model for Performing Experiments |
|
|
145 | (1) |
|
|
146 | (1) |
|
|
146 | (4) |
|
|
150 | (1) |
|
|
150 | (1) |
|
|
150 | (1) |
|
4.2.1.6 Dissemination and Decision-Making |
|
|
150 | (1) |
|
4.2.2 Key Experimental Design Concepts |
|
|
151 | (2) |
|
|
153 | (1) |
|
|
154 | (1) |
|
|
155 | (2) |
|
4.2.3 Types of Experimental Designs |
|
|
157 | (2) |
|
|
159 | (1) |
|
|
159 | (2) |
|
4.2.4 Selecting an Experimental Design |
|
|
161 | (1) |
|
4.2.4.1 Choosing the Number of Factors |
|
|
162 | (2) |
|
4.2.4.2 Factors versus Blocks |
|
|
164 | (1) |
|
4.2.4.3 Choosing between Nested and Crossed Designs |
|
|
165 | (4) |
|
4.2.4.4 Fixed and Random Effects |
|
|
169 | (1) |
|
4.2.4.5 Matched- or Same-Subject Designs |
|
|
170 | (1) |
|
4.2.4.6 Repeated Measurements |
|
|
170 | (1) |
|
4.3 Planning Case Studies As Quasi-Experiments |
|
|
171 | (2) |
|
|
172 | (1) |
|
|
172 | (1) |
|
4.3.3 Partitioned Project |
|
|
173 | (1) |
|
4.3.4 Retrospective Case Study |
|
|
173 | (1) |
|
4.4 Relevant And Meaningful Studies |
|
|
173 | (6) |
|
4.4.1 Confirming Theories and "Conventional Wisdom" |
|
|
174 | (1) |
|
4.4.2 Exploring Relationships |
|
|
175 | (2) |
|
4.4.3 Evaluating the Accuracy of Prediction Models |
|
|
177 | (1) |
|
4.4.4 Validating Measures |
|
|
178 | (1) |
|
|
179 | (4) |
|
|
180 | (1) |
|
|
181 | (1) |
|
|
182 | (1) |
|
Chapter 5 Software Metrics Data Collection |
|
|
183 | (42) |
|
|
184 | (1) |
|
5.2 Data Collection For Incident Reports |
|
|
185 | (19) |
|
5.2.1 The Problem with Problems |
|
|
186 | (5) |
|
|
191 | (6) |
|
|
197 | (6) |
|
|
203 | (1) |
|
|
204 | (9) |
|
5.3.1 Data Collection Forms |
|
|
208 | (3) |
|
5.3.2 Data Collection Tools |
|
|
211 | (2) |
|
5.4 Reliability Of Data Collection Procedures |
|
|
213 | (1) |
|
|
214 | (11) |
|
|
216 | (6) |
|
|
222 | (1) |
|
|
222 | (3) |
|
Chapter 6 Analyzing Software Measurement Data |
|
|
225 | (66) |
|
6.1 Statistical Distributions And Hypothesis Testing |
|
|
226 | (6) |
|
6.1.1 Probability Distributions |
|
|
226 | (5) |
|
6.1.2 Hypothesis Testing Approaches |
|
|
231 | (1) |
|
6.2 Classical Data Analysis Techniques |
|
|
232 | (11) |
|
|
233 | (1) |
|
6.2.1.1 Sampling, Population, and Data Distribution |
|
|
233 | (3) |
|
6.2.1.2 Distribution of Software Measurements |
|
|
236 | (3) |
|
6.2.1.3 Statistical Inference and Classical Hypothesis Testing |
|
|
239 | (2) |
|
6.2.2 Purpose of the Experiment |
|
|
241 | (1) |
|
6.2.2.1 Confirming a Theory |
|
|
241 | (1) |
|
6.2.2.2 Exploring a Relationship |
|
|
242 | (1) |
|
|
243 | (1) |
|
6.3 Examples Of Simple Analysis Techniques |
|
|
243 | (16) |
|
|
243 | (4) |
|
|
247 | (1) |
|
|
248 | (2) |
|
|
250 | (2) |
|
6.3.5 Measures of Association |
|
|
252 | (1) |
|
|
253 | (2) |
|
|
255 | (2) |
|
|
257 | (2) |
|
6.3.9 Multivariate Regression |
|
|
259 | (1) |
|
6.4 More Advanced Methods |
|
|
259 | (8) |
|
6.4.1 Classification Tree Analysis |
|
|
259 | (2) |
|
|
261 | (3) |
|
6.4.3 Multivariate Data Analysis |
|
|
264 | (1) |
|
6.4.3.1 Principal Component Analysis |
|
|
264 | (3) |
|
|
267 | (1) |
|
6.4.3.3 Discriminant Analysis |
|
|
267 | (1) |
|
6.5 Multicriteria Decision Aids |
|
|
267 | (12) |
|
6.5.1 Basic Concepts of Multicriteria Decision-Making |
|
|
268 | (6) |
|
6.5.2 Multiattribute Utility Theory |
|
|
274 | (2) |
|
|
276 | (2) |
|
6.5.4 Bayesian Evaluation of Multiple Hypotheses |
|
|
278 | (1) |
|
6.6 Overview Of Statistical Tests |
|
|
279 | (5) |
|
|
279 | (1) |
|
|
280 | (1) |
|
6.6.1.2 Chi-Squared Test for Goodness of Fit |
|
|
280 | (1) |
|
6.6.1.3 Kolmogorov--Smirnov One-Sample Test |
|
|
281 | (1) |
|
6.6.1.4 One-Sample Runs Test |
|
|
281 | (1) |
|
6.6.1.5 Change-Point Test |
|
|
281 | (1) |
|
|
281 | (1) |
|
6.6.2.1 Tests to Compare Two Matched or Related Groups |
|
|
282 | (1) |
|
6.6.2.2 Tests to Compare Two Independent Groups |
|
|
283 | (1) |
|
6.6.3 Comparisons Involving More than Two Groups |
|
|
283 | (1) |
|
|
284 | (7) |
|
|
285 | (4) |
|
|
289 | (1) |
|
|
289 | (2) |
|
Chapter 7 Metrics for Decision Support: The Need for Causal Models |
|
|
291 | (44) |
|
7.1 From Correlation And Regression To Causal Models |
|
|
293 | (8) |
|
7.2 Bayes Theorem And Bayesian Networks |
|
|
301 | (5) |
|
7.3 Applying Bayesian Networks To The Problem Of Software Defects Prediction |
|
|
306 | (14) |
|
7.3.1 A Very Simple BN for Understanding Defect Prediction |
|
|
307 | (3) |
|
7.3.2 A Full Model for Software Defects and Reliability Prediction |
|
|
310 | (4) |
|
7.3.3 Commercial Scale Versions of the Defect Prediction Models |
|
|
314 | (6) |
|
7.4 Bayesian Networks For Software Project Risk Assessment And Prediction |
|
|
320 | (8) |
|
|
328 | (7) |
|
|
329 | (1) |
|
|
330 | (5) |
|
PART II Software Engineering Measurement |
|
|
|
Chapter 8 Measuring Internal Product Attributes: Size |
|
|
335 | (36) |
|
8.1 Properties Of Software Size |
|
|
336 | (3) |
|
|
339 | (9) |
|
8.2.1 Counting Lines of Code to Measure Code Size |
|
|
339 | (5) |
|
8.2.2 Halstead's Approach |
|
|
344 | (2) |
|
8.2.3 Alternative Code Size Measures |
|
|
346 | (1) |
|
8.2.4 Dealing with Nontextual or External Code |
|
|
347 | (1) |
|
|
348 | (2) |
|
8.4 Requirements Analysis And Specification Size |
|
|
350 | (1) |
|
8.5 Functional Size Measures And Estimators |
|
|
351 | (9) |
|
|
352 | (3) |
|
8.5.1.1 Function Points for Object-Oriented Software |
|
|
355 | (1) |
|
8.5.1.2 Function Point Limitations |
|
|
356 | (2) |
|
|
358 | (2) |
|
8.6 Applications Of Size Measures |
|
|
360 | (4) |
|
8.6.1 Using Size to Normalize Other Measurements |
|
|
360 | (1) |
|
8.6.2 Size-Based Reuse Measurement |
|
|
361 | (2) |
|
8.6.3 Size-Based Software Testing Measurement |
|
|
363 | (1) |
|
8.7 Problem, Solution Size, Computational Complexity |
|
|
364 | (1) |
|
|
365 | (6) |
|
|
366 | (2) |
|
|
368 | (3) |
|
Chapter 9 Measuring Internal Product Attributes: Structure |
|
|
371 | (62) |
|
9.1 Aspects Of Structural Measures |
|
|
372 | (4) |
|
9.1.1 Structural Complexity Properties |
|
|
373 | (1) |
|
|
373 | (1) |
|
9.1.3 Coupling Properties |
|
|
374 | (1) |
|
9.1.4 Cohesion Properties |
|
|
375 | (1) |
|
9.1.5 Properties of Custom Attributes |
|
|
375 | (1) |
|
9.2 Control Flow Structure Of Program Units |
|
|
376 | (26) |
|
9.2.1 Flowgraph Model and the Notion of Structured Programs |
|
|
377 | (4) |
|
9.2.1.1 Sequencing and Nesting |
|
|
381 | (3) |
|
9.2.1.2 Generalized Notion of Structuredness |
|
|
384 | (2) |
|
9.2.1.3 Prime Decomposition |
|
|
386 | (2) |
|
9.2.2 Hierarchical Measures |
|
|
388 | (3) |
|
9.2.2.1 McCabe's Cyclomatic Complexity Measure |
|
|
391 | (2) |
|
9.2.2.2 McCabe's Essential Complexity Measure |
|
|
393 | (1) |
|
9.2.3 Code Structure and Test Coverage Measures |
|
|
394 | (5) |
|
9.2.3.1 Minimum Number of Test Cases |
|
|
399 | (2) |
|
9.2.3.2 Test Effectiveness Ratio |
|
|
401 | (1) |
|
9.3 Design-Level Attributes |
|
|
402 | (12) |
|
9.3.1 Models of Modularity and Information Flow |
|
|
402 | (2) |
|
|
404 | (1) |
|
|
405 | (1) |
|
|
406 | (3) |
|
|
409 | (1) |
|
|
410 | (2) |
|
9.3.7 Information Flow: Test Coverage Measures |
|
|
412 | (2) |
|
9.4 Object-Oriented Structural Attributes And Measures |
|
|
414 | (11) |
|
9.4.1 Measuring Coupling in Object-Oriented Systems |
|
|
416 | (2) |
|
9.4.2 Measuring Cohesion in Object-Oriented Systems |
|
|
418 | (3) |
|
9.4.3 Object-Oriented Length Measures |
|
|
421 | (1) |
|
9.4.4 Object-Oriented Reuse Measurement |
|
|
422 | (1) |
|
|
423 | (2) |
|
9.5 No Single Overall "Software Complexity" Measure |
|
|
425 | (3) |
|
|
428 | (5) |
|
|
429 | (4) |
|
|
433 | (96) |
|
A.1 McCabe's Testing Strategy |
|
|
433 | (3) |
|
|
433 | (1) |
|
|
434 | (2) |
|
A.2 Computing Test Coverage Measures |
|
|
436 | (5) |
|
|
437 | (4) |
|
Chapter 10 Measuring External Product Attributes |
|
|
441 | (34) |
|
10.1 Modeling Software Quality |
|
|
442 | (7) |
|
|
443 | (4) |
|
10.1.2 Define-Your-Own Models |
|
|
447 | (1) |
|
10.1.3 ISO/IEC 9126-1 and ISO/IEC 25010 Standard Quality Models |
|
|
447 | (2) |
|
10.2 Measuring Aspects Of Quality |
|
|
449 | (7) |
|
10.2.1 Defects-Based Quality Measures |
|
|
450 | (1) |
|
10.2.1.1 Defect Density Measures |
|
|
450 | (5) |
|
10.2.1.2 Other Quality Measures Based on Defect Counts |
|
|
455 | (1) |
|
|
456 | (4) |
|
10.3.1 External View of Usability |
|
|
457 | (2) |
|
10.3.2 Internal Attributes Affecting Usability |
|
|
459 | (1) |
|
10.4 Maintainability Measures |
|
|
460 | (6) |
|
10.4.1 External View of Maintainability |
|
|
462 | (1) |
|
10.4.2 Internal Attributes Affecting Maintainability |
|
|
463 | (3) |
|
|
466 | (4) |
|
10.5.1 External View of Security |
|
|
467 | (3) |
|
10.5.2 Internal Attributes Affecting Security |
|
|
470 | (1) |
|
|
470 | (5) |
|
|
471 | (2) |
|
|
473 | (2) |
|
Chapter 11 Software Reliability: Measurement and Prediction |
|
|
475 | (54) |
|
11.1 Basics Of Reliability Theory |
|
|
476 | (8) |
|
11.2 The Software Reliability Problem |
|
|
484 | (6) |
|
11.3 Parametric Reliability Growth Models |
|
|
490 | (7) |
|
11.3.1 The Jelinski--Moranda Model |
|
|
492 | (2) |
|
11.3.2 Other Models Based on JM |
|
|
494 | (1) |
|
11.3.3 The Littlewood Model |
|
|
495 | (1) |
|
11.3.4 The Littlewood--Verrall Model |
|
|
495 | (1) |
|
11.3.5 Nonhomogeneous Poisson Process Models |
|
|
496 | (1) |
|
11.3.6 General Comments on the Models |
|
|
497 | (1) |
|
|
497 | (11) |
|
11.4.1 Dealing with Bias: The u-Plot |
|
|
499 | (3) |
|
11.4.2 Dealing with Noise |
|
|
502 | (1) |
|
11.4.3 Prequential Likelihood Function |
|
|
503 | (5) |
|
11.4.4 Choosing the Best Model |
|
|
508 | (1) |
|
11.5 Recalibration Of Software Reliability Growth Predictions |
|
|
508 | (9) |
|
11.6 Importance Of The Operational Environment |
|
|
517 | (1) |
|
11.7 Wider Aspects Of Software Reliability |
|
|
518 | (5) |
|
|
523 | (6) |
|
|
523 | (3) |
|
|
526 | (3) |
Appendix: Solutions To Selected Exercises |
|
529 | (26) |
Bibliography |
|
555 | (22) |
Index |
|
577 | |