Preface |
|
xiii | |
Acknowledgments |
|
xix | |
About the Authors |
|
xxi | |
1 Intelligence Analysis: "Connecting the Dots" |
|
1 | (27) |
|
1.1 How Easy Is It to Connect the Dots? |
|
|
1 | (11) |
|
1.1.1 How Many Kinds of Dots Are There? |
|
|
2 | (1) |
|
1.1.2 Which Evidential Dots Can Be Believed? |
|
|
3 | (2) |
|
1.1.3 Which Evidential Dots Should Be Considered? |
|
|
5 | (1) |
|
1.1.4 Which Evidential Dots Should We Try to Connect? |
|
|
5 | (2) |
|
1.1.5 How to Connect Evidential Dots to Hypotheses? |
|
|
7 | (2) |
|
1.1.6 What Do Our Dot Connections Mean? |
|
|
9 | (2) |
|
1.1.7 Whose Evidential Dots Should Be Connected? |
|
|
11 | (1) |
|
1.2 Imaginative Reasoning in Intelligence Analysis |
|
|
12 | (5) |
|
1.2.1 Imaginative Reasoning |
|
|
12 | (1) |
|
1.2.2 What Ingredients of Analysis Are to Be Generated by Imaginative Thought? |
|
|
13 | (1) |
|
1.2.3 Generating Main Hypotheses to Be Defended by Evidence and Argument |
|
|
14 | (1) |
|
1.2.4 Generating the Evidential Grounds for Arguments |
|
|
15 | (1) |
|
1.2.5 Generating Arguments Linking Evidence and Hypotheses |
|
|
16 | (1) |
|
1.3 Intelligence Analysis as Discovery of Evidence, Hypotheses, and Arguments |
|
|
17 | (9) |
|
1.3.1 Intelligence Analysis in the Framework of the Scientific Method |
|
|
17 | (1) |
|
1.3.2 Evidence in Search of Hypotheses |
|
|
18 | (1) |
|
1.3.3 Hypotheses in Search of Evidence |
|
|
19 | (3) |
|
1.3.4 Evidentiary Testing of Hypotheses |
|
|
22 | (1) |
|
1.3.5 Completing the Analysis |
|
|
23 | (3) |
|
|
26 | (2) |
2 Marshaling Thoughts and Evidence for Imaginative Analysis |
|
28 | (17) |
|
2.1 Sherlock Holmes and Investigation or Discovery |
|
|
28 | (1) |
|
2.2 Mycroft Holmes and Evidence Marshaling |
|
|
29 | (1) |
|
2.3 Marshaling "Magnets" or Attractors |
|
|
30 | (2) |
|
2.4 Types of Marshaling Magnets |
|
|
32 | (9) |
|
2.4.1 Believability Magnet |
|
|
32 | (1) |
|
|
33 | (2) |
|
|
35 | (1) |
|
|
36 | (1) |
|
|
37 | (1) |
|
|
38 | (1) |
|
|
39 | (2) |
|
2.5 Use of the Marshaling Magnets |
|
|
41 | (1) |
|
|
42 | (3) |
3 Disciple-CD: A Cognitive Assistant for Connecting the Dots |
|
45 | (14) |
|
|
45 | (3) |
|
3.2 Obtaining Disciple-CD |
|
|
48 | (1) |
|
3.3 Hands On: Working with Knowledge Bases |
|
|
49 | (3) |
|
|
49 | (1) |
|
|
50 | (2) |
|
3.4 Knowledge Base Guidelines |
|
|
52 | (1) |
|
3.5 Hands On: Browsing an Argumentation |
|
|
53 | (6) |
|
|
53 | (4) |
|
|
57 | (1) |
|
|
58 | (1) |
4 Evidence |
|
59 | (23) |
|
|
59 | (3) |
|
4.2 The Credentials of All Evidence |
|
|
62 | (6) |
|
|
62 | (3) |
|
4.2.2 Believability or Credibility |
|
|
65 | (2) |
|
4.2.3 Force or Weight of Evidence |
|
|
67 | (1) |
|
4.3 Assessing the Relevance, Believability, and Inferential Force of Evidence |
|
|
68 | (5) |
|
4.4 Basic Operations with Disciple-CD |
|
|
73 | (6) |
|
4.4.1 Hands On: Define and Evaluate Evidence |
|
|
73 | (6) |
|
|
73 | (3) |
|
|
76 | (1) |
|
|
76 | (3) |
|
4.5 Advanced Operations with Disciple-CD |
|
|
79 | (1) |
|
4.5.1 Hands On: From Information to Evidence |
|
|
79 | (8) |
|
|
79 | (1) |
|
|
80 | (1) |
|
4.5.1.3 Advanced Operations |
|
|
80 | (1) |
|
|
80 | (2) |
5 Divide and Conquer: A Necessary Approach to Complex Analysis |
|
82 | (36) |
|
5.1 Holistic Approach to Analysis |
|
|
82 | (1) |
|
|
83 | (1) |
|
5.3 Assessing Complex Hypotheses through Analysis and Synthesis |
|
|
84 | (2) |
|
5.4 Inquiry-driven Analysis and Synthesis |
|
|
86 | (1) |
|
5.5 Types of Reductions and Corresponding Syntheses |
|
|
87 | (3) |
|
5.5.1 Necessary and Sufficient Conditions |
|
|
87 | (1) |
|
5.5.2 Sufficient Conditions and Scenarios |
|
|
88 | (1) |
|
|
88 | (2) |
|
5.6 Problems with Argument Construction |
|
|
90 | (3) |
|
5.7 Basic Operations with Disciple-CD |
|
|
93 | (9) |
|
5.7.1 Hands On: Was the Cesium Canister Stolen? |
|
|
93 | (5) |
|
5.7.1.1 Hypothesis in Search of Evidence: Illustration |
|
|
93 | (2) |
|
5.7.1.2 Hands-On Overview |
|
|
95 | (2) |
|
|
97 | (1) |
|
|
97 | (1) |
|
5.7.2 Hands On: Development and Evaluation of an Argument |
|
|
98 | (3) |
|
|
98 | (1) |
|
|
99 | (1) |
|
|
99 | (2) |
|
5.7.3 Hands On: Analysis Based on Previously Learned Patterns and Synthesis Functions |
|
|
101 | (1) |
|
|
101 | (1) |
|
|
101 | (1) |
|
|
101 | (1) |
|
5.8 Advanced Operations with Disciple-CD |
|
|
102 | (12) |
|
5.8.1 Hands On: Abstraction of Analysis |
|
|
102 | (2) |
|
|
102 | (1) |
|
|
103 | (1) |
|
5.8.1.3 Advanced Operations |
|
|
103 | (1) |
|
5.8.2 Hands On: Hypothesis Analysis and Evidence Search |
|
|
104 | (4) |
|
|
104 | (3) |
|
|
107 | (1) |
|
5.8.2.3 Advanced Operation |
|
|
107 | (1) |
|
5.8.3 Hands On: Justifications of Assumptions |
|
|
108 | (1) |
|
|
108 | (1) |
|
|
108 | (1) |
|
5.8.3.3 Advanced Operation |
|
|
108 | (1) |
|
5.8.4 Hands On: Top-down and Bottom-up Argument Development |
|
|
109 | (10) |
|
|
109 | (3) |
|
|
112 | (1) |
|
5.8.4.3 Advanced Operations |
|
|
112 | (2) |
|
|
114 | (2) |
|
|
116 | (2) |
6 Assessing the Believability of Evidence |
|
118 | (21) |
|
6.1 Believability: The Foundation of All Arguments from Evidence |
|
|
118 | (1) |
|
6.2 Classification of Evidence Based on Believability |
|
|
119 | (1) |
|
|
119 | (3) |
|
6.3.1 Real Tangible Evidence: Authenticity |
|
|
120 | (1) |
|
6.3.2 Demonstrative Tangible Evidence: Authenticity, Accuracy, and Reliability |
|
|
120 | (1) |
|
6.3.3 Examples of Tangible Evidence |
|
|
121 | (1) |
|
|
122 | (4) |
|
|
122 | (1) |
|
|
122 | (1) |
|
6.4.1.2 Understandability |
|
|
122 | (1) |
|
|
123 | (2) |
|
6.4.2.1 Veracity or Truthfulness |
|
|
123 | (1) |
|
|
123 | (1) |
|
6.4.2.3 Observational Sensitivity |
|
|
124 | (1) |
|
6.4.3 Types of Testimonial Evidence |
|
|
125 | (1) |
|
6.4.4 Examples of Testimonial Evidence |
|
|
126 | (1) |
|
|
126 | (2) |
|
6.5.1 Uncertainties Associated with Missing Evidence |
|
|
126 | (1) |
|
6.5.2 Example of Missing Evidence |
|
|
127 | (1) |
|
6.6 Authoritative Records |
|
|
128 | (1) |
|
|
128 | (2) |
|
6.7.1 Analysis of Mixed Evidence |
|
|
128 | (1) |
|
6.7.2 Examples of Mixed Evidence |
|
|
128 | (2) |
|
6.8 Deep Believability Analysis |
|
|
130 | (3) |
|
6.9 Advanced Operations with Disciple-CD |
|
|
133 | (4) |
|
6.9.1 Hands On: Believability Analysis |
|
|
133 | (10) |
|
|
133 | (2) |
|
|
135 | (1) |
|
6.9.1.3 Advanced Operation |
|
|
136 | (1) |
|
|
137 | (2) |
7 Chains of Custody |
|
139 | (9) |
|
7.1 What Is a Chain of Custody? |
|
|
139 | (1) |
|
7.2 A Case Involving Chains of Custody |
|
|
140 | (1) |
|
7.3 A Chain of Custody for Testimonial Evidence |
|
|
141 | (2) |
|
7.4 A Chain of Custody for Demonstrative Tangible Evidence |
|
|
143 | (3) |
|
7.4.1 Chain of Custody for a Photo Given Directly to the Analyst |
|
|
144 | (1) |
|
7.4.2 Chain of Custody for a Written Description of a Photo Given to the Analyst |
|
|
145 | (1) |
|
7.5 Analyzing a Chain of Custody |
|
|
146 | (1) |
|
7.6 Drill-Down Analysis of Chains of Custody |
|
|
147 | (1) |
|
|
147 | (1) |
8 Recurrent Substance-Blind Combinations of Evidence |
|
148 | (11) |
|
|
148 | (2) |
|
8.1.1 Basic Forms of Harmonious Evidence |
|
|
148 | (1) |
|
8.1.2 Patterns of Evidential Harmony |
|
|
149 | (1) |
|
|
150 | (2) |
|
8.2.1 Basic Forms of Dissonant Evidence |
|
|
150 | (2) |
|
8.2.2 Patterns of Evidential Dissonance |
|
|
152 | (1) |
|
|
152 | (2) |
|
8.3.1 Basic Forms of Redundant Evidence |
|
|
152 | (1) |
|
8.3.2 Patterns of Evidential Redundance |
|
|
153 | (1) |
|
8.4 Why Considering Evidence Combinations Is Important |
|
|
154 | (1) |
|
8.5 Basic Operations with Disciple-CD |
|
|
154 | (2) |
|
8.5.1 Hands On: Who Has Stolen the Cesium Canister? |
|
|
154 | (5) |
|
|
154 | (2) |
|
|
156 | (1) |
|
|
156 | (3) |
9 Major Sources of Uncertainty in Masses of Evidence |
|
159 | (14) |
|
|
159 | (3) |
|
9.1.1 What Is Incompleteness of Evidence? |
|
|
159 | (1) |
|
9.1.2 Examples of Incompleteness |
|
|
160 | (2) |
|
|
162 | (1) |
|
9.2.1 What Is Inconclusiveness of Evidence? |
|
|
162 | (1) |
|
9.2.2 Examples of Inconclusiveness |
|
|
163 | (1) |
|
|
163 | (2) |
|
9.3.1 What Is Ambiguity of Evidence? |
|
|
163 | (1) |
|
9.3.2 Examples of Ambiguity |
|
|
164 | (1) |
|
|
165 | (1) |
|
9.4.1 What Is the Dissonance of Evidence? |
|
|
165 | (1) |
|
9.4.2 Examples of Dissonance |
|
|
166 | (1) |
|
9.5 Imperfect Believability |
|
|
166 | (1) |
|
9.5.1 What Is Imperfect Believability of Evidence? |
|
|
166 | (1) |
|
9.5.2 Examples of Imperfect Believability |
|
|
166 | (1) |
|
9.6 Basic Operations with Disciple-CD |
|
|
167 | (4) |
|
9.6.1 Hands On: Does a Terrorist Organization Have the Cesium Canister? |
|
|
167 | (10) |
|
|
167 | (1) |
|
9.6.1.2 Extracting Evidence from Information |
|
|
168 | (2) |
|
|
170 | (1) |
|
|
171 | (2) |
10 Assessing and Reporting Uncertainty: Some Alternative Methods |
|
173 | (40) |
|
|
173 | (1) |
|
10.2 General Classes of Probability and Uncertainty |
|
|
174 | (1) |
|
10.3 Enumerative Probabilities: Obtained by Counting |
|
|
174 | (3) |
|
10.4 Nonenumerative Probabilities: Nothing to Count |
|
|
177 | (1) |
|
10.5 Epistemic Probability (1): The Subjective Bayesian View |
|
|
177 | (12) |
|
|
178 | (5) |
|
10.5.1.1 Analysis Using Likelihood Ratios |
|
|
178 | (1) |
|
|
179 | (4) |
|
|
183 | (6) |
|
10.5.2.1 Constructing the Argument Structure |
|
|
183 | (1) |
|
10.5.2.2 Forming the Key List |
|
|
183 | (1) |
|
10.5.2.3 Identifying the Likelihoods and Prior Probabilities |
|
|
184 | (3) |
|
10.5.2.4 Using the Bayesian Network |
|
|
187 | (2) |
|
10.5.2.5 Utility and Feasibility of Bayesian Network Analyses |
|
|
189 | (1) |
|
10.6 Epistemic Probability (2): Belief Functions |
|
|
189 | (8) |
|
10.6.1 Belief Functions and Evidential Support |
|
|
189 | (3) |
|
10.6.2 Examples of Assigning Evidential Support |
|
|
192 | (1) |
|
10.6.3 Dempster's Rule for Combining Partial Beliefs |
|
|
193 | (4) |
|
10.7 Baconian Probability and the Importance of Evidential Completeness |
|
|
197 | (5) |
|
10.7.1 Variative and Eliminative Inferences |
|
|
197 | (1) |
|
10.7.2 Importance of Evidential Completeness |
|
|
197 | (4) |
|
10.7.3 Baconian Probability of Boolean Expressions |
|
|
201 | (1) |
|
10.8 Imprecision and Fuzzy Probability |
|
|
202 | (3) |
|
10.8.1 Fuzzy Force of Evidence |
|
|
202 | (2) |
|
10.8.2 Fuzzy Probability of Boolean Expressions |
|
|
204 | (1) |
|
10.8.3 On Verbal Assessments of Probabilities |
|
|
204 | (1) |
|
10.9 A Summary of Uncertainty Methods and What They Best Capture |
|
|
205 | (3) |
|
10.10 Basic Operations with Disciple-CD |
|
|
208 | (3) |
|
10.10.1 Hands On: Will a Bomb Be Set Off in Washington, D.C.? |
|
|
208 | (6) |
|
|
208 | (1) |
|
|
209 | (2) |
|
|
211 | (2) |
11 Analytic Bias |
|
213 | (12) |
|
11.1 Basic Interpretations of the Term "Bias" |
|
|
213 | (1) |
|
11.2 Biases of the Analyst |
|
|
214 | (5) |
|
11.2.1 Biases in the Evaluation of Evidence |
|
|
215 | (1) |
|
11.2.2 Biases in the Perception of Cause and Effect |
|
|
216 | (1) |
|
11.2.3 Biases in Estimating Probabilities |
|
|
217 | (1) |
|
11.2.4 Hindsight Biases in Evaluating Intelligence Reporting |
|
|
218 | (1) |
|
11.3 Some Frequently Overlooked Origins of Bias |
|
|
219 | (3) |
|
|
219 | (1) |
|
11.3.2 Persons in Chains of Custody of Evidence |
|
|
220 | (1) |
|
11.3.3 Consumers of Intelligence Analyses |
|
|
221 | (1) |
|
11.4 Biases and the Evaluation of Analysts |
|
|
222 | (1) |
|
11.5 Recognizing and Countering Biases with Disciple-CD |
|
|
223 | (1) |
|
|
223 | (2) |
12 Learning and Reusing Analytic Expertise: Beyond Disciple-CD |
|
225 | (14) |
|
|
225 | (1) |
|
12.2 Learning Agent Shell |
|
|
225 | (3) |
|
12.3 Learning Agent Shell for Evidence-based Reasoning |
|
|
228 | (3) |
|
|
228 | (1) |
|
|
229 | (2) |
|
12.4 Development of a Cognitive Assistant |
|
|
231 | (5) |
|
12.5 Evidence-Based Reasoning Everywhere |
|
|
236 | (3) |
Glossary of Terms |
|
239 | (10) |
References |
|
249 | (6) |
Appendixes |
|
255 | (2) |
|
1 Methodological Guidelines |
|
|
255 | (1) |
|
|
255 | (1) |
|
3 Operations with Disciple-CD |
|
|
256 | (1) |
Index |
|
257 | |