Acknowledgments |
|
xiii | |
Introduction |
|
xv | |
|
Part I Goals, Strategies, and Tactics |
|
|
1 | (52) |
|
What Does It Mean to Be Pragmatic? |
|
|
3 | (10) |
|
What Do Effective and Efficient Mean? |
|
|
3 | (1) |
|
What Effects Do You Want? |
|
|
4 | (1) |
|
What Is the Right Level of Efficiency? |
|
|
4 | (3) |
|
|
4 | (1) |
|
|
5 | (1) |
|
What Software Testing Isn't...But Is Often. Thought to Be |
|
|
6 | (1) |
|
Five Phases of a Tester's Mental Life |
|
|
7 | (1) |
|
Other Founding Views on Testing |
|
|
8 | (1) |
|
Testing as a Form of Quality Risk Management |
|
|
9 | (1) |
|
So What Is the Test Team All About? |
|
|
9 | (4) |
|
What Does ``Quality'' Mean to You? |
|
|
10 | (3) |
|
|
13 | (6) |
|
Exercise: The Triangle Test |
|
|
13 | (6) |
|
Your Solution for the Triangle Test |
|
|
14 | (1) |
|
Your Solution for the Triangle Test (Continued) |
|
|
15 | (1) |
|
Author's Triangle Test Solution |
|
|
15 | (4) |
|
Aligning Testing with the Project |
|
|
19 | (16) |
|
Why Do Organizations Test? |
|
|
19 | (1) |
|
|
20 | (1) |
|
|
20 | (1) |
|
Common Test Phases and Objectives |
|
|
21 | (1) |
|
Testing Throughout the Organization |
|
|
21 | (2) |
|
|
23 | (2) |
|
Evolutionary and Incremental Models |
|
|
25 | (3) |
|
|
28 | (1) |
|
Regrettably Common Model: Code and Fix |
|
|
28 | (1) |
|
Testing Maintenance Releases |
|
|
29 | (1) |
|
|
30 | (2) |
|
Hardware/Software Development |
|
|
32 | (1) |
|
|
32 | (3) |
|
Understanding Test Strategies, Tactics, and Design |
|
|
35 | (18) |
|
Aligning Mission, Strategies, and Tactics |
|
|
35 | (1) |
|
Analytical Test Strategies |
|
|
36 | (1) |
|
Model-Based Test Strategies |
|
|
37 | (1) |
|
Methodical Test Strategies |
|
|
38 | (1) |
|
Process-Oriented Test Strategies |
|
|
38 | (1) |
|
|
39 | (1) |
|
Philosophical Test Strategies |
|
|
40 | (1) |
|
|
41 | (2) |
|
Regression Strategy 1: Repeat All Tests |
|
|
42 | (1) |
|
Regression Strategy 2: Repeat Some Tests |
|
|
42 | (1) |
|
Three Other Regression Strategies |
|
|
43 | (1) |
|
Tactics: Categories of Testing Techniques |
|
|
44 | (2) |
|
Strategic and Tactical Considerations |
|
|
46 | (1) |
|
|
47 | (3) |
|
Classic Principles for Test Design |
|
|
50 | (1) |
|
Phases of Test Development |
|
|
51 | (1) |
|
Synopsis of Test Strategies, Tactics, and Design |
|
|
52 | (1) |
|
Part II Risk-Based Testing |
|
|
53 | (52) |
|
Understanding Risks to System Quality |
|
|
55 | (18) |
|
Categories of Quality Risks |
|
|
56 | (4) |
|
|
56 | (1) |
|
Performance and Reliability |
|
|
57 | (2) |
|
Stress, Capacity, and Volume |
|
|
59 | (1) |
|
|
59 | (1) |
|
|
60 | (3) |
|
Installation and Deinstallation |
|
|
61 | (1) |
|
|
62 | (1) |
|
Maintenance and Maintainability |
|
|
63 | (1) |
|
|
63 | (9) |
|
Usability and User Interface |
|
|
64 | (1) |
|
|
65 | (1) |
|
Error and Disaster Handling and Recovery |
|
|
66 | (1) |
|
|
67 | (1) |
|
|
68 | (1) |
|
Configuration and Compatibility |
|
|
69 | (1) |
|
Networked, Internetworked, and Distributed |
|
|
69 | (1) |
|
Standards and Regulatory Compliance |
|
|
70 | (1) |
|
|
70 | (1) |
|
|
71 | (1) |
|
|
71 | (1) |
|
Can You Think of Other Quality Risks? |
|
|
72 | (1) |
|
Aligning Testing with Quality Risks |
|
|
73 | (12) |
|
Prioritize Risks to System Quality |
|
|
73 | (1) |
|
Testing, Customer Usage, and System Configurations |
|
|
74 | (2) |
|
Approaches for Quality Risks Analysis |
|
|
76 | (2) |
|
Informal Quality Risk Analysis |
|
|
78 | (3) |
|
|
81 | (1) |
|
Challenges of Risk Analysis |
|
|
82 | (3) |
|
Quality Risk Analysis Exercise |
|
|
85 | (20) |
|
|
86 | (5) |
|
|
91 | (1) |
|
A Funny Thing Happened to Me on the Way to the Quality Risks... |
|
|
92 | (3) |
|
|
95 | (1) |
|
|
96 | (4) |
|
|
100 | (1) |
|
|
101 | (1) |
|
Bonus Example: Grays and Blues Quality Risk Analysis |
|
|
102 | (3) |
|
|
105 | (36) |
|
Reviews and Other Static Tests |
|
|
107 | (16) |
|
Testing Requirements and Designs |
|
|
107 | (1) |
|
Reviews Costs and Benefits |
|
|
108 | (1) |
|
|
109 | (1) |
|
Reviews for Verification and Validation |
|
|
109 | (1) |
|
Reviews as a Way to Improve the Process, Improve the System, and Reduce Costs |
|
|
110 | (4) |
|
Answers to the Three Questions about Static Testing |
|
|
114 | (1) |
|
Reviews as a Way to Achieve Consensus and Understanding |
|
|
115 | (1) |
|
The Review Process, Roles, and Responsibilities |
|
|
116 | (1) |
|
Deliverables and Ground Rules from Reviews |
|
|
117 | (1) |
|
Common Requirements and Design Bugs |
|
|
118 | (1) |
|
Reviewing (and Testing) Documentation |
|
|
119 | (1) |
|
|
120 | (3) |
|
|
123 | (18) |
|
|
123 | (1) |
|
|
124 | (4) |
|
|
128 | (4) |
|
|
132 | (1) |
|
Bonus Exercise: Reviewing Triangle Requirements Title Page |
|
|
132 | (2) |
|
Triangle Requirements Review Using Wiegers's List |
|
|
132 | (2) |
|
The Requirements Bugs You Found |
|
|
134 | (3) |
|
My Solution: Requirements Bugs |
|
|
137 | (2) |
|
|
139 | (2) |
|
Part IV Behavioral Testing |
|
|
141 | (138) |
|
Equivalence Classes and Boundary Values |
|
|
143 | (16) |
|
|
144 | (1) |
|
|
144 | (15) |
|
|
145 | (2) |
|
|
147 | (2) |
|
|
149 | (3) |
|
|
152 | (2) |
|
|
154 | (1) |
|
|
155 | (2) |
|
Beyond Functional Classes and Boundaries |
|
|
157 | (2) |
|
Equivalence Classes and Boundary Values Exercise |
|
|
159 | (10) |
|
Functional Boundaries and Classes |
|
|
159 | (2) |
|
|
161 | (5) |
|
Nonfunctional Boundaries and Classes |
|
|
166 | (3) |
|
Omninet Marketing Requirements Document (v0.3) |
|
|
166 | (2) |
|
Omninet System Requirements Document (v0.2) |
|
|
168 | (1) |
|
Use Cases, Live Data, and Decision Tables |
|
|
169 | (8) |
|
Use Cases and Scenario Tests |
|
|
169 | (3) |
|
Nouns and Verbs, Adjectives and Adverbs |
|
|
172 | (1) |
|
Live Data and Customer Tests |
|
|
173 | (1) |
|
|
174 | (3) |
|
|
177 | (20) |
|
|
177 | (1) |
|
|
178 | (3) |
|
Decision Tables and Boundary Values |
|
|
181 | (2) |
|
|
183 | (5) |
|
Building a Decision Table for Testing |
|
|
188 | (2) |
|
|
190 | (7) |
|
State Transition Diagrams |
|
|
197 | (6) |
|
Describing Systems with States Using State Transition Diagrams |
|
|
197 | (2) |
|
|
199 | (1) |
|
Printer Server State Transition Diagram |
|
|
200 | (3) |
|
State Transition Diagram Exercise |
|
|
203 | (14) |
|
|
203 | (1) |
|
|
204 | (7) |
|
|
211 | (1) |
|
|
212 | (1) |
|
Grays and Blues and State Transition Diagrams |
|
|
213 | (1) |
|
|
214 | (3) |
|
|
217 | (16) |
|
|
217 | (1) |
|
A Domain Example Using Frequent-Flyer Programs |
|
|
218 | (2) |
|
Possible Domain Test Values |
|
|
220 | (1) |
|
|
221 | (2) |
|
|
223 | (1) |
|
|
224 | (1) |
|
Complex Domain Testing Example |
|
|
225 | (3) |
|
A General Rule for Complex Domains |
|
|
228 | (5) |
|
|
233 | (6) |
|
|
234 | (5) |
|
Orthogonal Arrays and All Pairs |
|
|
239 | (22) |
|
|
239 | (2) |
|
Orthogonal Arrays and All-Pairs Tables |
|
|
241 | (1) |
|
|
241 | (2) |
|
Selecting an Orthogonal Array |
|
|
243 | (1) |
|
Applying Orthogonal Arrays to a Complicated Real-World Example |
|
|
244 | (14) |
|
|
258 | (1) |
|
Other Thoughts on Configuration Testing |
|
|
259 | (2) |
|
Orthogonal Arrays Exercise |
|
|
261 | (6) |
|
|
263 | (4) |
|
|
267 | (12) |
|
General Facts about Reactive Tests |
|
|
267 | (1) |
|
Error Guessing, Attacks, and Bug Taxonomies |
|
|
268 | (1) |
|
|
269 | (1) |
|
|
270 | (1) |
|
|
271 | (2) |
|
Other Sources of Inspiration for Reactive Testing |
|
|
273 | (1) |
|
Advantages and Disadvantages |
|
|
274 | (2) |
|
A Case Study of Exploratory Testing |
|
|
276 | (3) |
|
Part V Structural Testing |
|
|
279 | (46) |
|
|
281 | (6) |
|
|
282 | (2) |
|
McCabe Cyclomatic Complexity |
|
|
284 | (3) |
|
Control-Flow Testing Exercise |
|
|
287 | (6) |
|
|
288 | (5) |
|
|
293 | (4) |
|
Data-Flow Testing Exercise |
|
|
297 | (10) |
|
My Solutions and Comments |
|
|
298 | (9) |
|
|
307 | (14) |
|
|
308 | (1) |
|
|
309 | (2) |
|
|
311 | (2) |
|
McCabe Basis Paths for Integration |
|
|
313 | (3) |
|
Enhanced Hex Converter Program |
|
|
316 | (2) |
|
|
318 | (3) |
|
Integration Basis Test Exercise |
|
|
321 | (4) |
|
|
323 | (2) |
|
|
325 | (30) |
|
Appendix A Omninet: The Internet Everywhere Marketing Requirements Document |
|
|
327 | (8) |
|
|
329 | (1) |
|
1.1 Terms, Acronyms, and Abbreviations |
|
|
329 | (1) |
|
|
330 | (1) |
|
|
330 | (1) |
|
3 Description of requirements |
|
|
330 | (1) |
|
3.1 General technical requirements |
|
|
330 | (1) |
|
|
330 | (1) |
|
|
330 | (1) |
|
|
331 | (1) |
|
|
331 | (1) |
|
|
331 | (1) |
|
|
331 | (1) |
|
3.1.7 Session Termination |
|
|
331 | (1) |
|
|
332 | (1) |
|
|
332 | (1) |
|
|
332 | (1) |
|
|
332 | (1) |
|
|
333 | (1) |
|
|
333 | (1) |
|
|
333 | (2) |
|
Appendix B Omninet: The Internet Everywhere System Requirements Document |
|
|
335 | (16) |
|
Functionality System Requirements |
|
|
336 | (4) |
|
Reliability System Requirements |
|
|
340 | (1) |
|
Usability System Requirements |
|
|
341 | (1) |
|
Efficiency System Requirements |
|
|
342 | (1) |
|
Maintainability System Requirements |
|
|
343 | (1) |
|
Portability System Requirements |
|
|
344 | (1) |
|
|
345 | (1) |
|
Omninet System Architecture |
|
|
345 | (1) |
|
Payment Processing Decision Table |
|
|
346 | (1) |
|
|
347 | (1) |
|
Kiosk State-Transition Diagram |
|
|
348 | (1) |
|
Kiosk State-Transition Table |
|
|
348 | (1) |
|
Kiosk OS/Browser/Connection Speed Configuration Orthogonal Array |
|
|
348 | (3) |
|
Appendix C Bibliography and Other Recommended Readings |
|
|
351 | (4) |
|
|
353 | (2) |
Index |
|
355 | |