Preface |
|
xix | |
1 What Is Forensic Systems Engineering? |
|
1 | (6) |
|
1.1 Systems and Systems Engineering |
|
|
1 | (1) |
|
1.2 Forensic Systems Engineering |
|
|
2 | (2) |
|
|
4 | (3) |
2 Contracts, Specifications, and Standards |
|
7 | (10) |
|
|
7 | (2) |
|
|
9 | (3) |
|
|
9 | (1) |
|
|
10 | (2) |
|
|
12 | (2) |
|
|
14 | (2) |
|
|
16 | (1) |
|
|
16 | (1) |
3 Management Systems |
|
17 | (8) |
|
|
18 | (1) |
|
3.1.1 Operations and Good Business Practices |
|
|
18 | (1) |
|
3.1.2 Attributes of Management Standards |
|
|
18 | (1) |
|
3.2 Effective Management Systems |
|
|
19 | (4) |
|
|
19 | (1) |
|
3.2.2 Total Quality Management |
|
|
20 | (1) |
|
|
20 | (1) |
|
|
21 | (1) |
|
3.2.5 Production Part Approval Process |
|
|
22 | (1) |
|
3.3 Performance and Performance |
|
|
23 | (1) |
|
|
23 | (1) |
|
|
24 | (1) |
|
|
24 | (1) |
4 Performance Management: ISO 9001 |
|
25 | (12) |
|
4.1 Background of ISO 9000 |
|
|
26 | (6) |
|
4.1.1 ISO 9001 in the United States |
|
|
27 | (1) |
|
4.1.2 Structure of ISO 9000:2005 |
|
|
27 | (1) |
|
4.1.3 The Process Approach |
|
|
28 | (4) |
|
|
32 | (3) |
|
4.2.1 Reference Performance Standards |
|
|
33 | (1) |
|
4.2.2 Forensics and the Paper Trail |
|
|
34 | (1) |
|
|
35 | (1) |
|
|
35 | (2) |
5 The Materiality of Operations |
|
37 | (10) |
|
5.1 Rationale for Financial Metrics |
|
|
38 | (3) |
|
|
38 | (1) |
|
5.1.1.1 Title III: Corporate Responsibility |
|
|
38 | (1) |
|
5.1.1.2 Title IV: Enhanced Financial Disclosures |
|
|
39 | (1) |
|
|
39 | (2) |
|
5.1.3 The Materiality of Quality |
|
|
41 | (1) |
|
5.2 Mapping Operations to Finance |
|
|
41 | (3) |
|
5.2.1 The Liability of Quality |
|
|
43 | (1) |
|
|
44 | (1) |
|
|
44 | (1) |
|
|
44 | (3) |
6 Process Liability |
|
47 | (8) |
|
6.1 Theory of Process Liability |
|
|
48 | (4) |
|
6.1.1 Operations and Process Liability |
|
|
50 | (1) |
|
6.1.2 Process Liability and Misfeasance |
|
|
51 | (1) |
|
6.2 Process Liability and the Law |
|
|
52 | (1) |
|
|
52 | (1) |
|
|
52 | (3) |
7 Forensic Analysis of Process Liability |
|
55 | (16) |
|
7.1 Improper Manufacturing Operations |
|
|
57 | (5) |
|
7.1.1 Verification and Validation |
|
|
57 | (2) |
|
7.1.1.1 Nonstandard Design Procedures |
|
|
57 | (1) |
|
7.1.1.2 Unverified or Unvalidated Design |
|
|
58 | (1) |
|
7.1.1.3 Tests Waived by Management |
|
|
58 | (1) |
|
7.1.1.4 Altered Test Procedures and Results |
|
|
58 | (1) |
|
7.1.2 Resource Management |
|
|
59 | (2) |
|
7.1.2.1 Unmonitored Outsourcing |
|
|
59 | (1) |
|
7.1.2.2 Substandard Purchased Parts |
|
|
60 | (1) |
|
|
60 | (1) |
|
7.1.2.4 Ineffective Flow Down |
|
|
61 | (1) |
|
|
61 | (1) |
|
7.1.3.1 Forced Production |
|
|
61 | (1) |
|
7.1.3.2 Abuse and Threats by Management |
|
|
62 | (1) |
|
7.2 Management Responsibility |
|
|
62 | (5) |
|
7.2.1 Effective Internal Controls |
|
|
62 | (1) |
|
7.2.2 Business Standards of Care |
|
|
63 | (1) |
|
7.2.3 Liability Risk Management |
|
|
64 | (1) |
|
7.2.4 Employee Empowerment |
|
|
65 | (1) |
|
7.2.5 Effective Management Review |
|
|
65 | (1) |
|
7.2.6 Closed-Loop Processes |
|
|
66 | (1) |
|
|
67 | (4) |
8 Legal Trends to Process Liability |
|
71 | (6) |
|
8.1 An Idea Whose Time Has Come |
|
|
71 | (1) |
|
8.2 Some Court Actions Thus Far |
|
|
72 | (3) |
|
8.2.1 QMS Certified Organizations |
|
|
73 | (1) |
|
8.2.2 QMS Noncertified Organizations |
|
|
74 | (1) |
|
|
75 | (2) |
9 Process Stability and Capability |
|
77 | (14) |
|
|
77 | (6) |
|
9.1.1 Stability and Stationarity |
|
|
78 | (1) |
|
9.1.2 Stability Conditions |
|
|
79 | (1) |
|
|
80 | (2) |
|
9.1.4 Measuring Process Stability |
|
|
82 | (1) |
|
|
83 | (2) |
|
9.2.1 Measuring Capability |
|
|
83 | (2) |
|
9.2.2 A Limit of Process Capability |
|
|
85 | (1) |
|
|
85 | (2) |
|
9.3.1 Instability and the Rare Event |
|
|
85 | (1) |
|
9.3.2 Identifying the Rare Event |
|
|
86 | (1) |
|
|
87 | (1) |
|
|
88 | (3) |
10 Forensic Issues in Product Reliability |
|
91 | (16) |
|
10.1 Background in Product Reliability |
|
|
91 | (3) |
|
10.2 Legal Issues in the Design of Reliability |
|
|
94 | (3) |
|
10.2.1 Good Design Practices |
|
|
95 | (1) |
|
10.2.2 Design Is Intrinsic to Manufacturing and Service |
|
|
95 | (1) |
|
|
95 | (1) |
|
10.2.4 Paper Trail of Evidence |
|
|
96 | (1) |
|
10.2.5 Reliability Is an Implied Design Requirement |
|
|
97 | (1) |
|
10.3 Legal Issues in Measuring Reliability |
|
|
97 | (3) |
|
|
97 | (1) |
|
10.3.2 Estimation of MTTF |
|
|
98 | (1) |
|
10.3.3 The More Failure Data the Better |
|
|
99 | (1) |
|
10.3.4 The Paper Trail of Reliability Measurement |
|
|
99 | (1) |
|
10.4 Legal Issues in Testing for Reliability |
|
|
100 | (2) |
|
10.4.1 Defined and Documented Life Test Procedures |
|
|
100 | (1) |
|
10.4.2 Life Test Records and Reports |
|
|
101 | (1) |
|
|
101 | (1) |
|
10.5 When Product Reliability Is not in the Contract |
|
|
102 | (2) |
|
|
102 | (1) |
|
|
103 | (1) |
|
10.6 Warranty and Reliability |
|
|
104 | (1) |
|
|
105 | (2) |
11 Forensic View of Internal Control |
|
107 | (18) |
|
|
108 | (2) |
|
11.1.1 Purpose of Control |
|
|
108 | (1) |
|
|
109 | (1) |
|
11.1.3 Control Elements in Operations |
|
|
109 | (1) |
|
|
110 | (5) |
|
11.2.1 Model of a Continuous System |
|
|
111 | (1) |
|
11.2.2 Transfer Functions |
|
|
112 | (3) |
|
11.3 Implementing Controls |
|
|
115 | (2) |
|
11.4 Control of Operations |
|
|
117 | (6) |
|
11.4.1 Proportional (Gain) Control |
|
|
118 | (1) |
|
11.4.2 Controlling the Effect of Change |
|
|
119 | (2) |
|
11.4.2.1 Integral Control |
|
|
120 | (1) |
|
11.4.2.2 Derivative (Rate) Control |
|
|
121 | (1) |
|
11.4.3 Responsibility, Authority, and Accountability |
|
|
121 | (2) |
|
|
123 | (2) |
12 Case Study: Madelena Airframes Corporation |
|
125 | (14) |
|
12.1 Background of the Case |
|
|
126 | (1) |
|
|
127 | (1) |
|
12.2.1 MAC Policies and Procedures (Missile Production) |
|
|
127 | (1) |
|
|
127 | (1) |
|
12.3 Examining the Evidence |
|
|
128 | (4) |
|
12.3.1 Evidence: The Players |
|
|
129 | (1) |
|
|
129 | (3) |
|
|
132 | (1) |
|
12.4.1 Deposition of the General Manager |
|
|
132 | (1) |
|
12.4.2 Deposition of the Senior Test Engineer |
|
|
132 | (1) |
|
12.4.3 Deposition of the Production Manager |
|
|
132 | (1) |
|
12.4.4 Deposition of the Chief Design Engineer |
|
|
133 | (1) |
|
12.4.5 Deposition of the Test Programs Manager |
|
|
133 | (1) |
|
|
133 | (3) |
|
12.5.1 Review of the Evidence |
|
|
133 | (1) |
|
|
134 | (3) |
|
12.5.2.1 Clause 7.3.1(b) Design and Development Planning |
|
|
134 | (1) |
|
12.5.2.2 Clause 7.3.5 Design and Development Verification |
|
|
135 | (1) |
|
12.5.2.3 Clause 7.3.6 Design and Development Validation |
|
|
135 | (1) |
|
12.5.2.4 Clause 8.1 General Test Requirements |
|
|
135 | (1) |
|
12.5.2.5 Clause 8.2.4 Monitoring and Measurement of Product |
|
|
135 | (1) |
|
12.5.2.6 Clause 4.1 General QMS Requirements |
|
|
135 | (1) |
|
12.5.2.7 Clause 5.6.1 General Management Review Requirements |
|
|
135 | (1) |
|
12.6 Arriving at the Truth |
|
|
136 | (1) |
|
|
137 | (1) |
|
12.7.1 Synthesis of Damages |
|
|
137 | (1) |
|
12.7.2 Costs of Correction |
|
|
137 | (1) |
|
|
138 | (1) |
13 Examining Serially Dependent Processes |
|
139 | (10) |
|
13.1 Serial Dependence: Causal Correlation |
|
|
140 | (2) |
|
13.2 Properties of Serial Dependence |
|
|
142 | (5) |
|
13.2.1 Work Station Definition |
|
|
142 | (1) |
|
|
142 | (2) |
|
|
143 | (1) |
|
|
143 | (1) |
|
|
143 | (1) |
|
13.2.3 Development of the Conditional Distribution |
|
|
144 | (1) |
|
|
145 | (2) |
|
13.3 Serial Dependence: Noncausal Correlation |
|
|
147 | (1) |
|
13.4 Forensic Systems Analysis |
|
|
147 | (1) |
|
|
148 | (1) |
|
|
148 | (1) |
14 Measuring Operations |
|
149 | (12) |
|
14.1 ISO 9000 as Internal Controls |
|
|
151 | (1) |
|
|
152 | (2) |
|
14.3 The QMS Forensic Model |
|
|
154 | (3) |
|
14.3.1 Estimating Control Risk |
|
|
155 | (1) |
|
|
156 | (1) |
|
14.4 The Forensic Lab and Operations |
|
|
157 | (1) |
|
|
158 | (1) |
|
|
159 | (1) |
|
|
159 | (2) |
15 Stability Analysis of Dysfunctional Processes |
|
161 | (18) |
|
|
162 | (3) |
|
|
162 | (1) |
|
15.1.2 Common and Special Causes |
|
|
163 | (1) |
|
15.1.3 Disturbances and Interventions |
|
|
163 | (1) |
|
|
163 | (2) |
|
|
165 | (3) |
|
15.3 Question Before the Law |
|
|
168 | (1) |
|
|
169 | (4) |
|
|
170 | (1) |
|
15.4.2 Mathematical Model for Correlation |
|
|
170 | (3) |
|
|
173 | (1) |
|
|
174 | (1) |
|
|
174 | (5) |
16 Verification and Validation |
|
179 | (18) |
|
|
180 | (5) |
|
16.1.1 An Historical View |
|
|
180 | (2) |
|
16.1.2 Productivity versus Quality |
|
|
182 | (3) |
|
|
185 | (5) |
|
16.2.1 Verification and Validation Defined |
|
|
186 | (1) |
|
16.2.2 Inspection and Test |
|
|
187 | (1) |
|
16.2.3 Monitor and Measure |
|
|
188 | (1) |
|
16.2.4 Subtle Transitions |
|
|
189 | (1) |
|
16.3 The Forensic View of Measurement |
|
|
190 | (4) |
|
16.3.1 Machine Tools and Tooling |
|
|
190 | (1) |
|
|
191 | (1) |
|
|
192 | (1) |
|
|
192 | (1) |
|
16.3.5 First Article Inspection |
|
|
193 | (1) |
|
|
194 | (1) |
|
|
194 | (3) |
17 Forensic Sampling of Internal Controls |
|
197 | (14) |
|
|
198 | (3) |
|
|
199 | (1) |
|
|
199 | (1) |
|
|
200 | (1) |
|
17.1.4 One Hundred Percent Inspection |
|
|
201 | (1) |
|
|
201 | (3) |
|
|
201 | (1) |
|
17.2.2 Statistical and Nonstatistical Sampling |
|
|
202 | (1) |
|
17.2.3 Fixed Size and Stop-or-Go |
|
|
203 | (1) |
|
17.2.4 Sample Selection and Size |
|
|
204 | (1) |
|
|
204 | (5) |
|
17.3.1 Internal Control Sampling |
|
|
204 | (2) |
|
|
206 | (1) |
|
17.3.2.1 Acceptable Deviation Rate |
|
|
206 | (1) |
|
17.3.2.2 System Deviation Rate |
|
|
207 | (1) |
|
|
207 | (1) |
|
|
207 | (1) |
|
17.3.3.2 Alpha and Beta Risks |
|
|
208 | (1) |
|
|
208 | (1) |
|
|
209 | (1) |
|
17.4 Forensic System Caveats |
|
|
209 | (1) |
|
|
210 | (1) |
18 Forensic Analysis of Supplier Control |
|
211 | (18) |
|
|
213 | (2) |
|
18.2 Supply Chain Management |
|
|
215 | (1) |
|
18.3 Forensic Analysis of Supply Systems |
|
|
216 | (7) |
|
18.3.1 Basic Principles of Supplier Control |
|
|
216 | (1) |
|
18.3.2 The Forensic Challenge |
|
|
216 | (7) |
|
18.3.2.1 Ensure that Purchased Units Conform to Contracted Specifications |
|
|
217 | (1) |
|
18.3.2.2 Assessment of the Supplier Process |
|
|
218 | (1) |
|
|
218 | (1) |
|
18.3.2.4 Customer Relations |
|
|
219 | (2) |
|
18.3.2.5 Verification and Storage of Supplies |
|
|
221 | (1) |
|
18.3.2.6 Identification and Traceability |
|
|
222 | (1) |
|
18.4 Supplier Verification: A Case Study |
|
|
223 | (3) |
|
|
224 | (1) |
|
|
224 | (2) |
|
|
226 | (1) |
|
18.5 Malfeasant Supply Systems |
|
|
226 | (1) |
|
|
227 | (2) |
19 Discovering System Nonconformity |
|
229 | (18) |
|
19.1 Identifying Nonconformities |
|
|
231 | (3) |
|
19.1.1 Reporting Nonconformities |
|
|
232 | (1) |
|
|
233 | (1) |
|
19.2 The Elements of Assessment |
|
|
234 | (2) |
|
19.2.1 Measures of Performance |
|
|
234 | (1) |
|
19.2.2 Considerations in Forensic Analysis of Systems |
|
|
235 | (1) |
|
|
236 | (2) |
|
19.4 Describing Nonconformities |
|
|
238 | (2) |
|
19.5 A Forensic View of Documented Information |
|
|
240 | (6) |
|
19.5.1 Requirements in Documented Information |
|
|
241 | (1) |
|
19.5.2 The Quality Manual |
|
|
241 | (2) |
|
19.5.3 Documented Information Control |
|
|
243 | (1) |
|
|
244 | (2) |
|
|
246 | (1) |
|
|
246 | (1) |
Appendix A The Engineering Design Process: A Descriptive View |
|
247 | (8) |
|
A.1 Design and Development |
|
|
248 | (4) |
|
|
248 | (1) |
|
A.1.2 Customer Requirements |
|
|
249 | (1) |
|
|
249 | (1) |
|
A.1.4 Intermediate Testing |
|
|
249 | (2) |
|
|
251 | (1) |
|
A.2 Forensic Analysis of the Design Process |
|
|
252 | (1) |
|
|
253 | (2) |
Appendix B Introduction to Product Reliability |
|
255 | (18) |
|
B.1 Reliability Characteristics |
|
|
256 | (3) |
|
B.1.1 Reliability Metrics |
|
|
256 | (1) |
|
|
257 | (2) |
|
|
259 | (4) |
|
|
259 | (1) |
|
|
260 | (2) |
|
|
260 | (2) |
|
|
262 | (1) |
|
|
262 | (1) |
|
B.3 Design for Reliability |
|
|
263 | (2) |
|
B.4 Measuring Reliability |
|
|
265 | (4) |
|
B.4.1 On Reliability Metrics |
|
|
265 | (1) |
|
B.4.2 Graphing Failure Data |
|
|
266 | (3) |
|
B.5 Testing for Reliability |
|
|
269 | (2) |
|
|
271 | (2) |
Appendix C Brief Review of Probability and Statistics |
|
273 | (14) |
|
|
274 | (2) |
|
C.1.1 Average: The Mean Value |
|
|
274 | (1) |
|
C.1.2 Average: The Median |
|
|
275 | (1) |
|
|
275 | (1) |
|
C.2 Measures of Dispersion |
|
|
276 | (1) |
|
|
276 | (1) |
|
|
276 | (1) |
|
|
277 | (4) |
|
C.3.1 Continuous Distributions |
|
|
277 | (2) |
|
C.3.2 Discrete Distributions |
|
|
279 | (2) |
|
|
281 | (3) |
|
C.4.1 Estimating Parametric Change |
|
|
281 | (3) |
|
|
284 | (1) |
|
|
284 | (1) |
|
|
285 | (2) |
Appendix D Sampling of Internal Control Systems |
|
287 | (18) |
|
|
288 | (3) |
|
|
289 | (1) |
|
|
290 | (1) |
|
|
290 | (1) |
|
|
291 | (3) |
|
D.2.1 Acceptable Deviation Rate |
|
|
292 | (1) |
|
D.2.2 System Deviation Rate |
|
|
293 | (1) |
|
|
293 | (1) |
|
|
294 | (3) |
|
|
294 | (1) |
|
D.3.2 Consumer and Producer Risks |
|
|
294 | (1) |
|
D.3.3 Alpha and Beta Errors |
|
|
295 | (2) |
|
|
297 | (5) |
|
D.4.1 Statistical Inference |
|
|
297 | (1) |
|
D.4.2 Sample Distributions |
|
|
298 | (1) |
|
|
299 | (1) |
|
|
299 | (1) |
|
D.4.5 Confidence Interval |
|
|
300 | (2) |
|
|
302 | (3) |
Appendix E Statistical Sampling Plans |
|
305 | (16) |
|
E.1 Fixed-Size Attribute Sampling Plan |
|
|
306 | (7) |
|
E.1.1 Determine the Objectives |
|
|
306 | (1) |
|
E.1.2 Define Attribute and Deviation Conditions |
|
|
306 | (1) |
|
E.1.2.1 Acceptable Deviation Rate |
|
|
306 | (1) |
|
E.1.2.2 System Deviation Rate |
|
|
307 | (1) |
|
E.1.3 Define the Population |
|
|
307 | (1) |
|
E.1.4 Determine the Method of Sample Selection |
|
|
307 | (1) |
|
E.1.5 Determine the Sample Size |
|
|
308 | (4) |
|
E.1.6 Perform the Sampling Plan |
|
|
312 | (1) |
|
E.1.7 Evaluate Sample Results |
|
|
312 | (1) |
|
|
313 | (3) |
|
E.2.1 Acceptable Deviation Rate |
|
|
313 | (1) |
|
|
314 | (2) |
|
|
316 | (1) |
|
E.3 One Hundred Percent Inspection |
|
|
316 | (1) |
|
E.4 Application: An Attribute Sampling Plan |
|
|
317 | (1) |
|
|
318 | (3) |
Appendix F Nonstatistical Sampling Plans |
|
321 | (8) |
|
|
322 | (5) |
|
F.1.1 Frame of the Sampling Plan |
|
|
322 | (1) |
|
F.1.2 Attribute and Deviation Conditions |
|
|
323 | (1) |
|
|
323 | (1) |
|
F.1.4 Nonstatistical Sample Selection |
|
|
324 | (1) |
|
|
325 | (1) |
|
F.1.6 The Effect of Sample Size on Beta Error |
|
|
326 | (1) |
|
F.1.7 Evaluating Sample Results |
|
|
327 | (1) |
|
F.2 Nonstatistical Estimations |
|
|
327 | (1) |
|
|
328 | (1) |
Index |
|
329 | |