| Forewords |
|
xiii | |
| Preface |
|
xvi | |
| Acknowledgments |
|
xviii | |
| About this Book |
|
xxi | |
| About the Author |
|
xxv | |
| About the Cover Illustration |
|
xxvi | |
|
1 Effective and systematic software testing |
|
|
1 | (29) |
|
1.1 Developers who test vs. developers who do not |
|
|
2 | (9) |
|
1.2 Effective software testing for developers |
|
|
11 | (5) |
|
Effective testing in the development process |
|
|
12 | (2) |
|
Effective testing as an iterative process |
|
|
14 | (1) |
|
Focusing on development and then on testing |
|
|
14 | (1) |
|
The myth of "correctness by design" |
|
|
15 | (1) |
|
|
|
15 | (1) |
|
The meaning of effective and systematic |
|
|
15 | (1) |
|
The role of test automation |
|
|
16 | (1) |
|
1.3 Principles of software testing (or, why testing is so difficult) |
|
|
16 | (3) |
|
Exhaustive testing is impossible |
|
|
16 | (1) |
|
Knowing when to stop testing |
|
|
17 | (1) |
|
Variability is important (the pesticide paradox) |
|
|
17 | (1) |
|
Bugs happen in some places more than others |
|
|
17 | (1) |
|
No matter what testing you do, it will never be perfect or enough |
|
|
18 | (1) |
|
|
|
18 | (1) |
|
Verification is not validation |
|
|
18 | (1) |
|
1.4 The testing pyramid, and where we should focus |
|
|
19 | (11) |
|
|
|
19 | (1) |
|
|
|
20 | (1) |
|
|
|
21 | (2) |
|
When to use each test level |
|
|
23 | (1) |
|
Why do I favor unit tests? |
|
|
23 | (1) |
|
What do I test at the different levels? |
|
|
24 | (1) |
|
What if you disagree with the testing pyramid? |
|
|
25 | (2) |
|
Will this book help you find all the bugs? |
|
|
27 | (3) |
|
2 Specification-based testing |
|
|
30 | (67) |
|
2.1 The requirements say it all |
|
|
31 | (14) |
|
Step 1 Understanding the requirements, inputs, and outputs |
|
|
33 | (1) |
|
Step 2 Explore what the program does for various inputs |
|
|
34 | (1) |
|
Step 3 Explore possible inputs and outputs, and identify partitions |
|
|
35 | (2) |
|
Step 4 Analyze the boundaries |
|
|
37 | (2) |
|
|
|
39 | (2) |
|
Step 6 Automate the test cases |
|
|
41 | (2) |
|
Step 7 Augment the test suite with creativity and experience |
|
|
43 | (2) |
|
2.2 Specification-based testing in a nutshell |
|
|
45 | (1) |
|
2.3 Finding bugs with specification testing |
|
|
46 | (8) |
|
2.4 Specification-based testing in the real world |
|
|
54 | (10) |
|
The process should be iterative, not sequential |
|
|
54 | (1) |
|
How far should specification testing go? |
|
|
55 | (1) |
|
Partition or boundary? It does not matter! |
|
|
55 | (1) |
|
On and off points are enough, but feel free to add in and out points |
|
|
55 | (1) |
|
Use variations of the same input to facilitate understanding |
|
|
55 | (1) |
|
When the number of combinations explodes, be pragmatic |
|
|
56 | (1) |
|
When in doubt, go for the simplest input |
|
|
56 | (1) |
|
Pick reasonable values for inputs you do not care about |
|
|
56 | (1) |
|
Test for nulls and exceptional cases, but only when it makes sense |
|
|
56 | (1) |
|
Go for parameterized tests when tests have the same skeleton |
|
|
57 | (1) |
|
Requirements can be of any granularity |
|
|
57 | (1) |
|
How does this work with classes and state? |
|
|
57 | (2) |
|
The role of experience and creativity |
|
|
59 | (4) |
|
Structural testing and code coverage |
|
|
63 | (1) |
|
3.1 Code coverage, the right way |
|
|
64 | (4) |
|
3.2 Structural testing in a nutshell |
|
|
68 | (1) |
|
3.3 Code coverage criteria |
|
|
69 | (3) |
|
|
|
69 | (1) |
|
|
|
69 | (1) |
|
Condition + branch coverage |
|
|
70 | (1) |
|
|
|
71 | (1) |
|
3.4 Complex conditions and the MC/DC coverage criterion |
|
|
72 | (3) |
|
|
|
72 | (1) |
|
Creating a test suite that achieves MC/DC |
|
|
73 | (2) |
|
3.5 Handling loops and similar constructs |
|
|
75 | (1) |
|
3.6 Criteria subsumption, and choosing a criterion |
|
|
75 | (2) |
|
3.7 Specification-based and structural testing: A running example |
|
|
77 | (5) |
|
3.8 Boundary testing and structural testing |
|
|
82 | (1) |
|
3.9 Structural testing alone often is not enough |
|
|
82 | (2) |
|
3.10 Structural testing in the real world |
|
|
84 | (6) |
|
Why do some people hate code coverage'? |
|
|
84 | (2) |
|
What does it mean to achieve 100% coverage? |
|
|
86 | (2) |
|
What coverage criterion to use |
|
|
88 | (1) |
|
MC/DC when expressions are too complex and cannot be simplified |
|
|
88 | (1) |
|
|
|
89 | (1) |
|
What should not be covered? |
|
|
90 | (1) |
|
|
|
90 | (7) |
|
|
|
97 | (20) |
|
4.1 Pre-conditions and post-conditions |
|
|
98 | (4) |
|
|
|
99 | (1) |
|
Strong and weak pre- and post-conditions |
|
|
100 | (2) |
|
|
|
102 | (3) |
|
4.3 Changing contracts, and the Liskov substitution principle |
|
|
105 | (4) |
|
Inheritance and contracts |
|
|
107 | (2) |
|
4.4 How is design-by-contract related to testing? |
|
|
109 | (1) |
|
4.5 Design-by-contract in the real world |
|
|
110 | (7) |
|
Weak or strong pre-conditions? |
|
|
110 | (1) |
|
Input validation, contracts, or both? |
|
|
110 | (2) |
|
Asserts and exceptions: When to use one or the other |
|
|
112 | (1) |
|
Exception or soft return values? |
|
|
113 | (1) |
|
When not to use design-by-contract |
|
|
113 | (1) |
|
Should we write tests for pre-conditions, post-conditions, and invariants? |
|
|
114 | (1) |
|
|
|
114 | (3) |
|
|
|
117 | (24) |
|
5.1 Example 1: The passing grade program |
|
|
118 | (4) |
|
5.2 Example 2: Testing the unique method |
|
|
122 | (2) |
|
5.3 Example 3: Testing the indexOf method |
|
|
124 | (5) |
|
5.4 Example 4: Testing the Basket class |
|
|
129 | (7) |
|
5.5 Example 5: Creating complex domain objects |
|
|
136 | (1) |
|
5.6 Property-based testing in the real world |
|
|
137 | (4) |
|
Example-based testing vs. property-based testing |
|
|
137 | (1) |
|
Common issues in property-based tests |
|
|
138 | (1) |
|
|
|
139 | (2) |
|
|
|
141 | (31) |
|
6.1 Dummies, fakes, stubs, spies, and mocks |
|
|
143 | (2) |
|
|
|
143 | (1) |
|
|
|
144 | (1) |
|
|
|
144 | (1) |
|
|
|
144 | (1) |
|
|
|
144 | (1) |
|
6.2 An introduction to mocking frameworks |
|
|
145 | (13) |
|
|
|
145 | (5) |
|
|
|
150 | (3) |
|
|
|
153 | (4) |
|
|
|
157 | (1) |
|
6.3 Mocks in the real world |
|
|
158 | (14) |
|
The disadvantages of mocking |
|
|
159 | (1) |
|
What to mock and what not to mock |
|
|
160 | (4) |
|
|
|
164 | (2) |
|
Mocking types you do not own |
|
|
166 | (2) |
|
What do others say about mocking? |
|
|
168 | (4) |
|
7 Designing for testability |
|
|
172 | (26) |
|
7.1 Separating infrastructure code from domain code |
|
|
173 | (8) |
|
7.2 Dependency injection and controllability |
|
|
181 | (3) |
|
7.3 Making your classes and methods observable |
|
|
184 | (5) |
|
Example 1 Introducing methods to facilitate assertions |
|
|
184 | (2) |
|
Example 2 Observing the behavior of void methods |
|
|
186 | (3) |
|
7.4 Dependency via class constructor or value via method parameter? |
|
|
189 | (2) |
|
7.5 Designing for testability in the real world |
|
|
191 | (7) |
|
The cohesion of the class under test |
|
|
192 | (1) |
|
The coupling of the class under test |
|
|
193 | (1) |
|
Complex conditions and testability |
|
|
193 | (1) |
|
Private methods and testability |
|
|
193 | (1) |
|
Static methods, singletons, and testability |
|
|
194 | (1) |
|
The Hexagonal Architecture and mocks as a design technique |
|
|
194 | (1) |
|
Further reading about designing for testability |
|
|
195 | (3) |
|
8 Test-driven development |
|
|
198 | (17) |
|
8.1 Our first TDD session |
|
|
199 | (7) |
|
8.2 Reflecting on our first TDD experience |
|
|
206 | (2) |
|
8.3 TDD in the real world |
|
|
208 | (7) |
|
|
|
208 | (1) |
|
|
|
209 | (1) |
|
Does TDD work for all types of applications and domains? |
|
|
209 | (1) |
|
What does the research say about TDD? |
|
|
209 | (2) |
|
|
|
211 | (1) |
|
|
|
212 | (3) |
|
|
|
215 | (43) |
|
9.1 When to use larger tests |
|
|
216 | (13) |
|
Testing larger components |
|
|
216 | (8) |
|
Testing larger components that go beyond our code base |
|
|
224 | (5) |
|
9.2 Database and SQL testing |
|
|
229 | (10) |
|
What to test in a SQL query |
|
|
229 | (2) |
|
Writing automated tests for SQL queries |
|
|
231 | (5) |
|
Setting up infrastructure for SQL tests |
|
|
236 | (2) |
|
|
|
238 | (1) |
|
|
|
239 | (15) |
|
An introduction to Selenium |
|
|
239 | (3) |
|
|
|
242 | (9) |
|
Patterns and best practices |
|
|
251 | (3) |
|
9.4 Final notes on larger tests |
|
|
254 | (4) |
|
How do all the testing techniques fit? |
|
|
254 | (1) |
|
Perform cost/benefit analysis |
|
|
255 | (1) |
|
Be careful with methods that are covered but not tested |
|
|
255 | (1) |
|
Proper code infrastructure is key |
|
|
255 | (1) |
|
DSLs and tools for stakeholders to write tests |
|
|
256 | (1) |
|
Testing other types of web systems |
|
|
256 | (2) |
|
|
|
258 | (18) |
|
10.1 Principles of maintainable test code |
|
|
259 | (8) |
|
|
|
259 | (1) |
|
Tests should be cohesive, independent, and isolated |
|
|
259 | (1) |
|
Tests should have a reason to exist |
|
|
260 | (1) |
|
Tests should be repeatable and not flaky |
|
|
260 | (1) |
|
Tests should have strong assertions |
|
|
261 | (1) |
|
Tests should break if the behavior changes |
|
|
261 | (1) |
|
Tests should have a single and clear reason to fail |
|
|
262 | (1) |
|
Tests should be easy to write |
|
|
262 | (1) |
|
Tests should be easy to read |
|
|
262 | (4) |
|
Tests should be easy to change and evolve |
|
|
266 | (1) |
|
|
|
267 | (9) |
|
|
|
267 | (1) |
|
|
|
268 | (1) |
|
Bad handling of complex or external resources |
|
|
268 | (1) |
|
Fixtures that are too general |
|
|
269 | (1) |
|
|
|
270 | (6) |
|
|
|
276 | (5) |
|
11.1 Although the model looks linear, iterations are fundamental |
|
|
276 | (1) |
|
11.2 Bug-free software development: Reality or myth? |
|
|
277 | (1) |
|
11.3 Involve your final user |
|
|
278 | (1) |
|
11.4 Unit testing is hard in practice |
|
|
278 | (1) |
|
11.5 Invest in monitoring |
|
|
279 | (1) |
|
|
|
280 | (1) |
| Appendix Answers to exercises |
|
281 | (8) |
| References |
|
289 | (6) |
| Index |
|
295 | |
Lisainfo e-raamatute kohta