Preface |
|
xv | |
Acknowledgments |
|
xvii | |
About the Author |
|
xix | |
1 Introduction to Software Testing Management |
|
1 | |
|
1.1 Product Defect: A Case Study |
|
|
2 | |
|
|
3 | |
|
|
4 | |
|
1.4 Causes of Defects in Software |
|
|
5 | |
|
1.5 Factors Creating Problems in Software Development Process |
|
|
7 | |
|
1.6 Solutions to Software Development Problems |
|
|
8 | |
|
1.7 Definition of Software Quality |
|
|
9 | |
|
1.8 Definition of Good Design |
|
|
9 | |
|
1.9 Definition of Good Code |
|
|
10 | |
|
1.10 Definition of Testing |
|
|
11 | |
|
|
11 | |
|
|
11 | |
|
|
12 | |
|
|
12 | |
|
|
12 | |
|
1.11 Software Testing Evolution |
|
|
13 | |
|
1.12 Software Engineering |
|
|
14 | |
|
1.13 Software Testing Methodologies |
|
|
15 | |
|
|
15 | |
|
1.14.1 Test Case Execution Automation |
|
|
15 | |
|
|
16 | |
|
|
16 | |
|
|
16 | |
|
|
16 | |
|
1.16 Testing Being Commoditized? |
|
|
17 | |
|
|
17 | |
2 Kinds of Software Testing Projects |
|
19 | |
|
2.1 Software Testing Types |
|
|
19 | |
|
2.2 What Needs to Be Tested? |
|
|
23 | |
|
2.3 Enterprise System Testing |
|
|
24 | |
|
2.4 Enterprise System Types |
|
|
25 | |
|
2.4.1 Banking, Finance, Insurance, and Securities (BFSI) Systems |
|
|
25 | |
|
2.4.2 Enterprise Resource Planning (ERP) Systems |
|
|
25 | |
|
2.5 Desktop System Testing |
|
|
26 | |
|
2.6 Device Driver System Testing |
|
|
26 | |
|
2.7 Stage of Software Life Cycle |
|
|
27 | |
|
2.8 Outsourced Software Testing |
|
|
27 | |
|
2.8.1 Software Vendor Perspective |
|
|
27 | |
|
2.8.2 Software Service Provider Perspective |
|
|
28 | |
|
|
28 | |
|
|
29 | |
3 Software Testing Project Strategies |
|
31 | |
|
3.1 Strategy versus Planning |
|
|
32 | |
|
3.2 Complexity Management |
|
|
32 | |
|
3.2.1 Case Study on Complexity Management |
|
|
33 | |
|
3.2.1.1 Module Description |
|
|
34 | |
|
|
34 | |
|
|
34 | |
|
|
35 | |
|
|
35 | |
|
|
35 | |
|
3.3 Technology Management |
|
|
39 | |
|
|
40 | |
|
|
40 | |
|
|
41 | |
|
3.6.1 Technological Risks |
|
|
41 | |
|
|
42 | |
|
|
42 | |
|
3.7 Strategy for Automation |
|
|
42 | |
|
3.8 Strategy for Manual Testing |
|
|
43 | |
|
3.9 Automation Tool Selection |
|
|
43 | |
|
3.10 Strategy for Creating Automation Framework |
|
|
43 | |
|
3.11 Software Product Life Cycle and Automation |
|
|
44 | |
|
3.12 Test Case Prioritization |
|
|
46 | |
|
|
46 | |
|
3.14 Software Test Project Strategy Checklist |
|
|
46 | |
|
|
47 | |
|
|
47 | |
4 Project Effort Estimation |
|
49 | |
|
4.1 Estimation by Experience |
|
|
50 | |
|
4.2 Estimation Using Test Point Analysis |
|
|
51 | |
|
4.2.1 Basic Components of TPA |
|
|
51 | |
|
|
51 | |
|
|
52 | |
|
|
52 | |
|
4.2.2 TPA Calculation Details |
|
|
53 | |
|
4.2.2.1 Dynamic Test Point Characteristics |
|
|
53 | |
|
4.2.2.2 Dynamic Quality Characteristics |
|
|
54 | |
|
4.2.2.3 Test Point Calculation |
|
|
55 | |
|
4.2.2.4 Productivity Factor |
|
|
55 | |
|
4.2.2.5 Primary Test Hours |
|
|
56 | |
|
4.2.2.6 Total Number of Test Hours |
|
|
56 | |
|
|
57 | |
|
|
57 | |
|
4.2.4 TPA at Bidding Stage |
|
|
57 | |
|
4.3 TPA-Based Effort Estimation Implementation |
|
|
58 | |
|
4.3.1 Identify Test Life Cycle Stages |
|
|
58 | |
|
4.3.2 Identify Activities for Each Phase |
|
|
59 | |
|
4.3.3 Size Estimation for Each Phase |
|
|
59 | |
|
4.3.3.1 Test Requirement Phase |
|
|
59 | |
|
4.3.3.2 Test Case Design Phase |
|
|
60 | |
|
4.3.3.3 Test Script Development Phase |
|
|
60 | |
|
4.3.3.4 Test Case Execution Phase |
|
|
61 | |
|
|
61 | |
|
4.3.4 Effort Estimation for Each Phase |
|
|
61 | |
|
|
62 | |
|
4.4 Importance of Effort Estimation |
|
|
63 | |
|
|
63 | |
|
4.6 Schedule versus Effort |
|
|
64 | |
|
|
64 | |
|
4.8 Effort Estimation Checklist |
|
|
66 | |
|
4.8.1 Checklist for Software Test Schedule Estimation |
|
|
66 | |
|
4.8.2 Checklist for Software Test Effort Estimation |
|
|
66 | |
|
|
67 | |
5 Software Testing Project Plan |
|
69 | |
|
5.1 Test Area Prioritization |
|
|
70 | |
|
|
70 | |
|
|
70 | |
|
|
70 | |
|
5.5 Methodology Selection |
|
|
71 | |
|
|
71 | |
|
|
72 | |
|
|
72 | |
|
|
72 | |
|
|
73 | |
|
|
74 | |
|
5.6.5.1 Steps for Test Case Automation |
|
|
74 | |
|
|
75 | |
|
|
75 | |
|
5.8.1 Data Maintenance in Production Environment |
|
|
75 | |
|
5.9 Testing without a Formal Test Plan |
|
|
76 | |
|
|
77 | |
|
5.10 Software Test Plan Checklist |
|
|
78 | |
|
5.10.1 Test Plan Checklist—Analysis and Review |
|
|
78 | |
|
5.10.2 Test Plan Checklist—Testing Activities |
|
|
78 | |
|
5.10.3 Test Plan Checklist—Test Environment |
|
|
78 | |
|
5.10.4 Test Plan Checklist—Organization |
|
|
79 | |
|
5.10.5 Test Plan Checklist—Test Schedule |
|
|
79 | |
|
5.10.6 Test Plan Checklist—Test Tools |
|
|
79 | |
|
5.10.7 Test Plan Checklist—Configuration Management |
|
|
80 | |
|
5.10.8 Test Plan Checklist—Test Metrics |
|
|
80 | |
|
5.10.9 Test Plan Checklist—Project Tracking for Unit/Integration Testing |
|
|
80 | |
|
5.10.10 Test Plan Checklist—Project Tracking for Acceptance Testing |
|
|
81 | |
|
5.10.11 Test Plan Checklist—Project Tracking for System Testing |
|
|
81 | |
6 Software Testing Project Risk Management |
|
83 | |
|
6.1 Risk Measurement Method |
|
|
84 | |
|
6.1.1 Create a Scale and Assign a Score to Each Risk |
|
|
85 | |
|
6.1.2 Count Number of Times the Risk Occurs in the Project |
|
|
85 | |
|
6.1.3 Risk Analysis Case Study |
|
|
85 | |
|
6.2 Risks Related to the Application Being Tested |
|
|
86 | |
|
|
87 | |
|
6.3.1 Communication Risks |
|
|
87 | |
|
|
87 | |
|
|
88 | |
|
|
88 | |
|
|
88 | |
|
|
88 | |
|
6.5 Checklist for Risk Management |
|
|
89 | |
7 Software Testing Project Execution |
|
91 | |
|
7.1 Earned Value Management |
|
|
92 | |
|
|
93 | |
|
7.1.2 EVM Implementation for Software Projects |
|
|
94 | |
|
|
95 | |
|
7.2 Defect Tracking and Life Cycle |
|
|
95 | |
|
7.3 Monitoring of Production Systems |
|
|
96 | |
|
|
97 | |
|
7.5 Checklist for Test Execution |
|
|
97 | |
8 Software Testing Project Reporting |
|
99 | |
|
8.1 Importance of Reporting |
|
|
100 | |
|
8.1.1 What Should Go in a Report |
|
|
100 | |
|
|
100 | |
|
|
101 | |
|
8.2.1 Test Report Components |
|
|
102 | |
|
8.2.1.1 Acceptance Criteria |
|
|
103 | |
|
8.2.1.2 Accessibility Testing |
|
|
103 | |
|
|
103 | |
|
8.2.1.4 Blocked Test Cases |
|
|
103 | |
|
8.2.1.5 Boundary Value Coverage |
|
|
103 | |
|
|
103 | |
|
|
104 | |
|
|
104 | |
|
|
104 | |
|
|
104 | |
|
8.3.1 Metrics and Reports |
|
|
105 | |
|
|
106 | |
9 Automated Software Testing Benefits |
|
107 | |
|
9.1 Considerations for Automation |
|
|
109 | |
|
9.1.1 Analysis of Test Case Activities |
|
|
109 | |
|
9.1.2 Financial and Execution Time Impacts |
|
|
110 | |
|
|
110 | |
|
9.2 Test Automation History |
|
|
111 | |
|
|
112 | |
|
|
112 | |
|
9.3.1.1 Wrong Customer Expectations |
|
|
112 | |
|
9.3.1.2 Problem Statement |
|
|
112 | |
|
|
112 | |
|
|
113 | |
|
|
113 | |
|
|
113 | |
|
9.3.2.1 Automation Strategy |
|
|
113 | |
|
|
114 | |
|
|
114 | |
|
|
114 | |
|
|
115 | |
|
|
115 | |
|
|
115 | |
|
|
115 | |
|
|
115 | |
|
|
115 | |
|
|
116 | |
|
|
116 | |
|
|
116 | |
|
|
116 | |
|
|
116 | |
|
|
116 | |
|
|
117 | |
|
|
117 | |
|
|
117 | |
|
|
117 | |
|
|
117 | |
|
|
118 | |
|
|
118 | |
|
|
118 | |
|
|
118 | |
|
9.4 Keyword-Driven Automation Framework |
|
|
118 | |
|
9.4.1 Steps for Creating Keyword Framework |
|
|
119 | |
|
9.5 Data-Driven Automated Testing |
|
|
120 | |
10 Customer Expectation Management |
|
121 | |
|
10.1 Difficult Proposition |
|
|
121 | |
|
10.2 Service Level Agreement |
|
|
122 | |
|
|
123 | |
|
|
123 | |
|
|
125 | |
|
10.6 Requirement Analysis |
|
|
125 | |
|
10.7 Project Process Information |
|
|
126 | |
|
10.8 Case Study: Electronics Retailer |
|
|
127 | |
|
10.9 Customer Expectation Management Strategies |
|
|
127 | |
|
10.9.1 Customer Involvement |
|
|
127 | |
|
|
128 | |
|
10.9.3 Get Approval for Delivery Methodology |
|
|
128 | |
|
10.9.4 Communicate Risks Early |
|
|
129 | |
|
10.9.5 Commit Less and Deliver More |
|
|
129 | |
|
10.9.6 Be Cool and Share Lighter Moments |
|
|
129 | |
|
|
129 | |
|
|
130 | |
11 Software Testing Practice and Offshoring |
|
131 | |
|
|
133 | |
|
|
133 | |
|
11.1.1.1 Commercial Concerns |
|
|
133 | |
|
11.1.1.2 Technical and Process Concerns |
|
|
134 | |
|
|
135 | |
|
11.2 Benefits of Offshoring |
|
|
136 | |
|
11.2.1 Traditional Approach to Testing |
|
|
137 | |
|
11.2.2 Cost of Late Detection of Defects |
|
|
137 | |
|
11.3 Proposed Organization Structure |
|
|
138 | |
|
|
139 | |
|
11.3.2 Project Components |
|
|
139 | |
|
|
139 | |
|
|
139 | |
|
|
140 | |
|
|
140 | |
|
11.4 Software Testing Consolidation |
|
|
140 | |
|
11.5 Advantages of Offshoring Coupled with Centralized Quality Group |
|
|
141 | |
|
11.5.1 Offshore Team Structure |
|
|
141 | |
|
|
141 | |
|
11.6.1 The Current State Scenario |
|
|
142 | |
|
11.6.2 The Future State Scenario |
|
|
142 | |
|
11.6.3 A 3-Year Labor Cost ROI Scenario |
|
|
143 | |
|
|
143 | |
|
|
143 | |
|
11.6.6 New Model Implementation Details |
|
|
144 | |
|
11.6.7 Pilot Project Details |
|
|
144 | |
|
|
145 | |
|
11.6.9 Benefits of New Model |
|
|
146 | |
|
|
146 | |
|
|
146 | |
|
|
147 | |
|
11.6.10 Automation Benefits |
|
|
147 | |
|
|
149 | |
|
|
149 | |
12 Software Testing as a Commodity |
|
151 | |
|
|
153 | |
|
|
153 | |
|
12.2.1 Old Organization Structure |
|
|
153 | |
|
12.2.2 New Organization Structure |
|
|
154 | |
|
12.3 Linking Test Project with Development Project |
|
|
157 | |
|
|
158 | |
|
|
159 | |
|
|
160 | |
|
|
160 | |
|
|
160 | |
|
|
162 | |
|
|
162 | |
|
12.5 Project versus Manufacturing |
|
|
164 | |
|
|
164 | |
|
|
164 | |
|
12.5.3 Mass Servicing of Software Test Projects |
|
|
165 | |
|
|
165 | |
|
|
165 | |
13 Quality, Standards, and Software Testing |
|
167 | |
|
|
167 | |
|
|
168 | |
|
|
168 | |
|
|
169 | |
|
13.3 Software Development Process Quality |
|
|
169 | |
|
13.3.1 Standards for Software Development Processes |
|
|
172 | |
|
|
172 | |
|
13.3.1.2 Software Project Planning |
|
|
173 | |
|
13.3.1.3 Software Project Tracking and Oversight |
|
|
173 | |
|
13.3.1.4 Software Subcontract Management |
|
|
173 | |
|
13.3.1.5 Software Quality Assurance |
|
|
174 | |
|
13.4 Software Testing and Quality |
|
|
174 | |
|
13.5 Quality Standards Evolution |
|
|
175 | |
|
|
175 | |
Appendix I IEEE Standard for Software Test Documentation (ANSI/IEEE Standard 829-1983) |
|
177 | |
Appendix II Software Testing Glossary |
|
181 | |
Index |
|
213 | |