Preface |
|
v | |
Acknowledgements |
|
vi | |
|
|
1 | (12) |
|
|
1 | (2) |
|
1.1.1 Erik van Veenendaal |
|
|
1 | (2) |
|
|
3 | (1) |
|
|
3 | (2) |
|
|
5 | (1) |
|
1.4 Expectations and Business Outcomes |
|
|
6 | (2) |
|
1.5 Career Paths for Testers |
|
|
8 | (1) |
|
|
9 | (1) |
|
1.7 The Certification Exam |
|
|
10 | (1) |
|
|
11 | (2) |
|
2 The Context of Improvement |
|
|
13 | (46) |
|
|
13 | (4) |
|
2.2 What Can Be Improved? |
|
|
17 | (3) |
|
|
20 | (5) |
|
2.4 The Generic Improvement Process |
|
|
25 | (12) |
|
|
25 | (3) |
|
2.4.2 The IDEAL Improvement Framework |
|
|
28 | (2) |
|
2.4.3 Fundamental Concepts of Excellence |
|
|
30 | (7) |
|
2.5 Overview of Improvement Approaches |
|
|
37 | (19) |
|
2.5.1 Overview of Model-Based Approaches |
|
|
37 | (2) |
|
2.5.2 Overview of Analytical Approaches |
|
|
39 | (1) |
|
|
39 | (1) |
|
2.5.4 Other Approaches to Improving the Test Process |
|
|
39 | (17) |
|
|
56 | (3) |
|
3 Model-Based Improvement |
|
|
59 | (86) |
|
3.1 Introduction to Test Process Improvement Models |
|
|
60 | (11) |
|
3.1.1 Desirable Characteristics of Test Process Improvement Models |
|
|
60 | (3) |
|
3.1.2 Using Models: Benefits and Risks |
|
|
63 | (4) |
|
3.1.3 Categories of Models |
|
|
67 | (4) |
|
3.2 Software Process Improvement (SPI) Models |
|
|
71 | (20) |
|
3.2.1 Capability Maturity Model Integration (CMMI) |
|
|
71 | (13) |
|
|
84 | (6) |
|
3.2.3 Comparing CMMI and ISO/IEC 15504 |
|
|
90 | (1) |
|
3.3 Test Process Improvement Models |
|
|
91 | (46) |
|
3.3.1 The Test Process Improvement Model (TPI NEXT) |
|
|
91 | (15) |
|
3.3.2 Test Maturity Model integration (TMMi) |
|
|
106 | (18) |
|
3.3.3 Comparing TPI NEXT to TMMi |
|
|
124 | (4) |
|
3.3.4 Systematic Test and Evaluation Process (STEP) |
|
|
128 | (4) |
|
3.3.5 Critical Testing Processes (CTP) |
|
|
132 | (5) |
|
3.4 Comparing Process Models and Content Models |
|
|
137 | (1) |
|
3.5 Suitability of SPI Models and Test Process Improvement Models |
|
|
138 | (3) |
|
|
141 | (4) |
|
4 Analytical-Based Improvement |
|
|
145 | (60) |
|
|
145 | (1) |
|
|
146 | (28) |
|
4.2.1 Selecting Items for Causal Analysis |
|
|
148 | (6) |
|
4.2.2 Gathering and Organizing the Information |
|
|
154 | (6) |
|
4.2.3 Identifying Root Causes |
|
|
160 | (9) |
|
4.2.4 Drawing Conclusions |
|
|
169 | (1) |
|
4.2.5 Causal Analysis with System Diagrams |
|
|
170 | (1) |
|
4.2.6 Causal Analysis during Formal Reviews |
|
|
171 | (2) |
|
4.2.7 Causal Analysis Lessons Learned |
|
|
173 | (1) |
|
|
174 | (17) |
|
|
174 | (1) |
|
|
175 | (3) |
|
|
178 | (7) |
|
4.3.4 Supporting Tools and Techniques |
|
|
185 | (5) |
|
4.3.5 Bottom-Up Improvement |
|
|
190 | (1) |
|
4.4 Analysis Using Measures, Metrics, and Indicators |
|
|
191 | (10) |
|
4.4.1 Test Effectiveness Metrics |
|
|
193 | (2) |
|
4.4.2 Test Efficiency / Cost Metrics |
|
|
195 | (1) |
|
|
196 | (1) |
|
4.4.4 Predictability Metrics |
|
|
197 | (1) |
|
4.4.5 Product Quality Metrics |
|
|
197 | (3) |
|
4.4.6 Test Maturity Metrics |
|
|
200 | (1) |
|
|
201 | (4) |
|
5 Selecting Improvement Approaches |
|
|
205 | (18) |
|
5.1 Selecting Test Process Improvement Approaches |
|
|
205 | (1) |
|
|
206 | (4) |
|
|
210 | (2) |
|
5.4 Analytical Approaches |
|
|
212 | (3) |
|
|
215 | (1) |
|
5.6 Analytical Approaches and Improvement Models |
|
|
216 | (5) |
|
5.6.1 Analytical-Based Improvement with CMMI |
|
|
216 | (1) |
|
5.6.2 Analytical-Based Improvement with TPI NEXT |
|
|
217 | (2) |
|
5.6.3 Analytical-Based Improvement with TMMi |
|
|
219 | (1) |
|
5.6.4 Analytical-Based Improvement with CTP and STEP |
|
|
220 | (1) |
|
|
221 | (2) |
|
6 Process for Improvement |
|
|
223 | (36) |
|
|
223 | (6) |
|
6.1.1 IDEAL Process Improvement Framework |
|
|
223 | (2) |
|
|
225 | (4) |
|
6.2 Initiating the Improvement Process |
|
|
229 | (9) |
|
6.2.1 Identify Stimulus for Change |
|
|
230 | (1) |
|
6.2.2 Set Objectives for Test Improvement |
|
|
231 | (6) |
|
|
237 | (1) |
|
|
237 | (1) |
|
6.2.5 Charter Infrastructure |
|
|
238 | (1) |
|
6.3 Diagnosing the Current Situation |
|
|
238 | (11) |
|
6.3.1 Planning the Assessment |
|
|
240 | (1) |
|
6.3.2 Assessment Preparation |
|
|
241 | (1) |
|
6.3.3 Performing Interviews |
|
|
242 | (1) |
|
|
243 | (1) |
|
|
243 | (1) |
|
6.3.6 Performing Solution Analysis |
|
|
244 | (3) |
|
6.3.7 Recommending Improvement Actions |
|
|
247 | (2) |
|
6.4 Establishing a Test Improvement Plan |
|
|
249 | (3) |
|
|
250 | (1) |
|
6.4.2 Develop an Implementation Approach |
|
|
250 | (1) |
|
6.4.3 Planning the Improvements |
|
|
251 | (1) |
|
6.5 Acting to Implement Improvement |
|
|
252 | (3) |
|
6.5.1 Selecting and Executing a Pilot |
|
|
254 | (1) |
|
6.5.2 Manage and Control the Implementation |
|
|
254 | (1) |
|
6.6 Learning from the Improvement Program |
|
|
255 | (1) |
|
|
256 | (3) |
|
7 Organization, Roles, and Skills |
|
|
259 | (42) |
|
|
259 | (9) |
|
7.1.1 The Test Process Group (TPG) |
|
|
260 | (6) |
|
7.1.2 Test Improvement with Remote, Offshore, and Outsourced Teams |
|
|
266 | (2) |
|
7.2 Individual Roles and Staffing |
|
|
268 | (4) |
|
7.2.1 The Test Process Improver |
|
|
268 | (2) |
|
|
270 | (1) |
|
|
271 | (1) |
|
7.3 Skills of the Test Process Improver/Assessor |
|
|
272 | (26) |
|
7.3.1 Interviewing Skills |
|
|
273 | (10) |
|
|
283 | (3) |
|
7.3.3 Presentation and Reporting Skills |
|
|
286 | (6) |
|
|
292 | (1) |
|
|
293 | (2) |
|
7.3.6 Skills of Persuasion |
|
|
295 | (2) |
|
|
297 | (1) |
|
|
298 | (1) |
|
|
298 | (3) |
|
|
301 | (30) |
|
|
302 | (1) |
|
|
303 | (5) |
|
8.2.1 The Fundamental Change Process |
|
|
303 | (1) |
|
|
304 | (2) |
|
8.2.3 Tipping Points and Change |
|
|
306 | (2) |
|
|
308 | (5) |
|
8.3.1 Establish the Need for Improvement |
|
|
308 | (3) |
|
8.3.2 Create a Sense of Urgency |
|
|
311 | (1) |
|
8.3.3 Establish the Improvement Team |
|
|
312 | (1) |
|
8.4 Decide What to Change |
|
|
313 | (3) |
|
8.4.1 Establish a Vision of the Future |
|
|
314 | (1) |
|
8.4.2 Set Specific Objectives and Align to Business Goals |
|
|
314 | (1) |
|
8.4.3 Decide on an Implementation Strategy |
|
|
314 | (1) |
|
8.4.4 Balance Short-Term and Longer-Term Benefits |
|
|
315 | (1) |
|
|
316 | (6) |
|
8.5.1 Communicating for Buy-In and Understanding |
|
|
316 | (1) |
|
|
317 | (1) |
|
|
318 | (1) |
|
8.5.4 Handling Resistance to Change |
|
|
319 | (2) |
|
8.5.5 Climbing Out of Chaos: Developing Transforming Ideas |
|
|
321 | (1) |
|
|
322 | (6) |
|
8.6.1 Rollout of New Ideas and Practices |
|
|
323 | (3) |
|
8.6.2 Provide Lasting Support |
|
|
326 | (1) |
|
8.6.3 Create a New Culture of Improvement |
|
|
327 | (1) |
|
8.6.4 Practice Continuous Improvement Principles |
|
|
327 | (1) |
|
|
328 | (1) |
|
|
328 | (3) |
|
9 Critical Success Factors |
|
|
331 | (20) |
|
9.1 Critical Success Factors |
|
|
331 | (9) |
|
|
332 | (2) |
|
9.1.2 Getting the Job Done |
|
|
334 | (3) |
|
9.1.3 Critical Success Factors: A Case Study |
|
|
337 | (3) |
|
9.2 Setting a Culture for Improvement |
|
|
340 | (7) |
|
9.2.1 Defining "Improvement Culture" |
|
|
340 | (1) |
|
9.2.2 Aspects of Improvement Culture |
|
|
341 | (4) |
|
9.2.3 Test Process Improvement Manifesto |
|
|
345 | (2) |
|
|
347 | (4) |
|
10 Adapting to Different Life Cycle Models |
|
|
351 | (8) |
|
10.1 Test Process Improvement with Different Life Cycles |
|
|
351 | (6) |
|
|
357 | (2) |
|
|
359 | (26) |
|
Appendix B Literature and References |
|
|
385 | (6) |
|
|
385 | (3) |
|
|
388 | (1) |
|
|
389 | (1) |
|
|
390 | (1) |
|
Appendix C The Syllabus Parts |
|
|
391 | (2) |
|
|
393 | (10) |
|
|
393 | (2) |
|
D.2 Part 1 Exam: "Assessing Test Processes" |
|
|
395 | (2) |
|
D.3 Part 2 Exam: "Implementing Test Process Improvement" |
|
|
397 | (2) |
|
|
399 | (1) |
|
|
400 | (3) |
|
Appendix E Summary of Cognitive Levels (K-Levels) |
|
|
403 | (2) |
|
|
405 | (6) |
Index |
|
411 | |