confidence band). This result suggests an extraordinary degree of certainty about the sum of our 1,000 cost items! The absurdity of this result should reinforce Example 11.2 and the importance of dependence. More generally, it illustrates the way i ndependence induces a square root rule reduction in variability as n increases. The standard deviation of the cost of one of our cost items, 1.4 (square root of 1.960), is about 14% of 10.2, but 44 is about 0.43% of 10,200, a reduction by a factor of about 32 (equal to the square root of 1,000). More generally, compared with a single item (n ¼ 1), n ¼ 4 increases total variability by a factor of 2, n ¼ 16 increases total variability by a factor of 4, n ¼ 64 increases total variability by a fact or of 8, and so on. Mean variance approac hes need not assume independence, and an expert system-driven probability evaluation package could usefully embrace moment- based approaches as well as CIM and Monte Carlo. Moment-based approaches offer speed and precision when appropriate assumptions hold and catastrophic systematic errors (bias) when appropriate assumptions do not hold. To summarize this section, three types of procedure can be considered, each with their advantages and disadvantages. Current industry standard software based on Monte Carlo simulation is effective if used by experts with adequate understanding of dependence in a manner reflecting the usage of CIM ap- proaches, but it is not a simple, efficient solution to all evaluate phase issues. It needs to be used with care and better software is feasible, even for the simple treatment of dependence discussed in this chapter. The discrete probability approach used in this book is useful at a conceptual level, even if it is replaced by a Monte Carlo simulation approach for operational purposes. Mean variance approaches are also conceptually useful, but need great care when used in practice, especially if they tempt people to assume independence that does not exist. Conclusion In a fully developed SHAMPU process, the evaluate phase is the pivotal phase that directs where and how successive iterations develop the analysis. Figure 4.1 indicates two key loops back to other phases, the estimate and define phases, but in practice selective loops back to other phases (like the structure phase) will provide effective short cuts. The effectiveness and the efficiency of the RMP as a whole depends on how well this iterative process works, in terms of the ability of the analysts to detect what is importan t and what is not, before spending too much time on the unimportant, without overlooking important threats or opportunities that do not stand out initially. Extensive probabilistic analysis based on carefully re- searched data can be very useful, but often such analysis is not appropriate. What is usually essential is an initial rough sizing of uncertainty from all the 228 Evaluate overall implications key sources that require management, followed by refinement in some areas where that refinement pays sufficient dividends. With notable exceptions, a focus on sizing uncertainty in order to evalu ate the best way to use later passes should be the goal of a first pass through the SHAMPU process. It provides the initial understanding of which areas need the most attention and which can receive less. This assessment is itself prone to risk, which must be managed. But treating all aspects of project uncertainty as equally important in a single-pass process is foolish. First-pass probabilities used to initiate the evaluate phase should be seen as simple statements of belief by those reasonably able to judge, brought together to provide a basis for discussing what matters and what does not. The numbers should be simple order-of-magnitude assessments, with a clear, overall health warning to the effect that no one will be held accountable for their accuracy. Only when and if it becomes clear where data analysis and objectively estimated probabilities might be useful should the accuracy of such estimates become a concern. The approach recommended here is based on a need for efficient and effec- tive decision taking, understanding that the probabilities used are a means to an end, not an end in themselves. What matters at the end of the day is the quality of the decisions taken, not the validity of the probability estimates. The results we get when we combine probability distributions are critically dependent on the dependence assumptions used. Assuming independence when this is not an appropriate assumption renders probabilistic risk analysis mislead- ing and dangerous, not just useless. Those who are not prepared to understand and reflect important dependencies should avoid probabilistic risk analysis. Those using risk analysis results provided by others should pay particular atten- tion to the understanding of dependence displayed by their analysts and totally reject any probabilistic analysis that suggests a failure to deal with dependence in an appropriate manner. If the range of values associated with an important variable is clearly misjudged by a factor of 10, the associated risk analysis is clearly suspect. If independence is assumed between half a dozen key variables when 50% dependence (or a coefficient of correlation of 0.5 or some equivalent level of dependence) is appropriate, the associated risk analysis is much more misleading. A factor of 10 error on a single variable may be a trivial error in comparison. Understanding dependence and understanding structure are related issues. The most effective way to deal with dependenc e in a statistical sense is to give it a causal structure that explains it. Statistical dependence is causal dependence we have failed to identify and structure. Sometimes a causal structure for dependence is not feasible, and other times it is not cost-effective . In such cases experienced analysts can effectively employ measures like percentage dependence or coefficients of correlation. However, to develop that experience, working with causal structures and conditional specifi- cations is an important part of the learning process. Conclusion 229 Harness the plans12 Plans are nothing, planning is everything.—Napoleon Bonaparte Introduction The plans produced during the harness phase of the SHAMPU (Shape, Harness, And Manage Project Uncertainty) process may be ‘nothing’ in Napoleon’s implied sense that the one thing we can be sure of is that things will not happen exactly as planned. However, in Napoleon’s implied sense the process of developing these plans is essential. In the SHAMPU process the earlier shaping phases (Table 4.2) are particularly important, but translating shaped, strategic plans into tactical plans ready for implementation is also important. Harnessing the plans involves working with several different types of plans, and careful distinction between them and associated terminology is useful. Project reference plans are the starting position for the SHAMPU process as captured in the define phase. They are defined in terms of all six Ws and over the whole of the project life cycle (PLC), at a level of detail appropriate to a strategic approach to uncertainty management. Project strategic plans are the reference plans with proactive responses to uncertainty and other changes suggested by the shaping phases embedded in them. By implication, reference plans are prototype strategic plans, the different terms allowing us to distinguish a starting point from an end point at this level. Project tactical plans are a more detailed version of strategic plans, with added detail to make them appropriate for implementation. They are defined over a tactical planning horizon chosen to facilitate the development of effective plans for implementation purposes. This detail usually includes a much- disaggregated activity structure (20 activities may become 200 or 2,000), more detailed designs, and new detail, like detailed resource allocations and milestones initiating payments and an associated expenditure profile. Producing this detail takes the project out of the plan stage and into the allocate stage of the PLC, as indicated in Figur e 4.2. Project contingency plans are the operat ional form of recommended reactive responses that include trigger points (decision rules) initiating the reactive re- sponses. They reflect anticipated potential departures from targets that deserve planning attention now, whether or not resource commitments are involved now. They may be defined at a tactical level or a strategic level. Both strategic and tactical plans as defined above are base plans, incorporating proactive responses but not reactive responses. Project action plans are defined over action horizons that accommodate appropriate lead times, but do not go beyond these lead times. Action plans are distinguished from other plans by a commitment to action. By implication other plans remain relatively fluid. This commitment to action may be at a strategic level or at a tactical level and may include commitments needed for contingency plans to remain feasible. The deliverables the harness phase should provide are in two basic forms: one is clearly stated project plan s of all the kinds noted above; the other is docu- mented uncertainty analysis associated with these plans. It is useful to identify the specific tasks of the harness phase under three headings: 1. Cons olidate and expla in the strategy—Document, verify, assess, and report project stra tegy and associated uncertainty analysis, completing a current update of the process that has been ongoing since the SHAMPU process began and providing a snapshot of the current state of play. 2. Formulate the tactics—Use the project strategic plans and the associated uncertainty analysis to select project management tactics, and develop these into tactical plans and contingency plans that are ready for commitment to implementation within the action horizons , using lower-level risk manag e- ment processes (RMPs) and intermediate levels of plans as appropriate. 3. Support and convince—Explain why the outputs associated with 1 and 2 above are effective and efficient, providing a case that is as convincing as the analysis to date will allow. Figure 12.1 portrays the way these three specific tasks and associated steps can be used, assuming separate support and convince tasks for strategy and tactics, with separate associated assess for approval tasks. The structure of this chapter follows that of Figure 12.1. Figure 12.1 shows no planned iterations apart from those within the harness phase involved in formulating the tactics. In practice, some unplanned iterations back to the SHAMPU define phase may be involved, but they are problems to be managed, not opportunities to be seized. Figure 12.1 shows unplanned iterations back to the define phase after assessing the strategy for approval, best avoided because of the negative aspects of a failed approval process. Figure 12.1 also shows unplanned iterations back to the define phase after assessing the tactics for approval, but unplanned iterations back to earlier SHAMPU phases this late in the PLC suggests earlier SHAMPU process failures that were serious. Stopping the project is a possibility, but stopping a project this late in the PLC suggests earlier process failures that were very serious. The defining difference between the end of the evaluate phase and the start of the harness phase is the desirability (or not) of iterations back to earlier phases, linked to the desirability of going public with the insights generated by the process in some cases. 232 Harness the plans Introduction 233 Figure 12.1—Specific tasks of the harness phase Consolidate and explain the strategy: reference plans Reference plans reflect the proj ect description at a strategic level captured in the SHAMPU define phase as noted at the end of Chapter 5. As indicated earlier, senior executives and directors often comment on the immense value to them of a care- fully crafted, simple explanation of the nature of the project that reference plans should provide. Ensuring an appropriate reference plan is available in an appro- priately polished form is the essence of this step of the harness phase, drawing to an effective conclusion a process that started at the outset of the SHAMPU process. It is important to recognize that to this point reference plans may have been dynamic, with relationships to initial reference plans that may not be worth elaboration. Reference plans can usefully capture some earlier misconceptions: to provide convenient ‘straw men’ to be knocked down by the uncertainty analysis in order to demonstrate the value of the RMP. Unlike strategic plans, reference plans need not be credible given the latest apprecia tion of project uncertainty, because they need not embed proactive responses. But if reference plans seriously embarrass any major players, they put the whole RMP at risk. Helping all major players to bury their embarrassing earlier misconceptions by revising reference plans is a key part of a successful RMP. Selective memory is often expedient. A key purpose of the RMP is to uncover aspects of project reference plans or associated project planning processes that need changes. However, provided all necessary changes are made, sometimes some changes are best kept confidential to the analyst and those responsible, to avoid any unnecessary embarrassment. Assuring everyone involved that this will be the case can be an important starting point at the beginning of the analysis. It is comparable with the confidentiality agreement any consultant signs before he or she starts work, except that it is informal and a separate arrangement with each individual player. The operative words here are avoiding unnecessary embar- rassment, provided necessary changes are made. The practical reason why it is important to adopt this position, and make it very public at the outset in the SHAMPU define phase, should be obvious. If this is not done, people will be inhibited and defensive in a way that is both natural and entirely reasonable given their perspective and to the detriment of the project as a whole from the project owner’s perspective. Our ultimate purpose is to help the project’s owners succeed. ‘Impartial science’ operating like a loose cannon will be recognized as a danger by all concerned and properly treated as such. Consolidate and explain the strategy: uncertainty analysis An uncertainty analysis report at the strategic level of interest here should include as a minimum a comprehensive list of threats and opportunities, assessed in 234 Harness the plans terms of implications given recommended proactive and reactive responses, along with an assessment of alternative potential proactive and reactive responses. Uncertainty analysis needs to be documented to back up associated recom- mendations and to provide an explanation of the need for both proactive and reactive responses. A bottom-up risk analysis process involves a bottom-up documentation process, but it needs to be presented top-down to explain the overall position first, then elaborate on what is driving the uncertainty that matters, using nested sets of diagrams like Figure 11.5. The process of interpreting uncertainty analysis in top-down terms can be regarded as the essence of this step in the harness phase. This top-down per- spective can produce new insights, and it is important to give it the time and space in the overall process that it deserves. Like writing the executive summary for a report, attempting to explain what we understand can be an important process in clarifying and developing that understanding. Communicating insight to allow decision takers to make choices reflecting all relevant issues is the goal. Analysis has serious limitations, and a failure to address these limitations when attempting to offer advice is a very serious mistake, possibly the most serious mistake an analyst can make. Consider a reasonably simple example to illustrate what is involved. Example 12.1 Highlighting the implications of different bid prices A major, international computer company wanted a formal system to address bidding for ‘systems integration’ projects, involving the supply of hardware, new software, revisions to existing software, revamped physical facilities, and retrained staff. The approach developed, often used by the authors as the basis for a case study for teaching purposes, employs a simplified version of the process explained in earlier chapters of this book to making some technical choices and choices between alternative subcontractors in order to derive an estimated expected cost for the project. Suppose the expected cost is estimated at £15 million. The process then involves assessing a ‘probability of winning’ curve like that of Figure 12.2 and using it together with the expected cost to define a table like Table 12.1. Figure 12.2 implies that attempting to ‘buy the work’ with a bid below about £13 million is counterproductive and winning the bid with certainty is not possible. Once the bid is above the expected cost of £15 milli on, the probability of winning drops rapidly, although the rate of decline of the probability of winning as the bid continues to increase has to drop off as the probability approaches 0. Table 12.1 implies that bidding at the expected cost involves zero ex- pected profit; each £1 million added to the bid increases the conditional Consolidate and explain the strategy: uncertainty analysis 235 expected profit (the expected profit given we win) by £1 million; each £1 million added to the bid increases the unconditional expected profit (profit  probability of winning) by an amount that peaks at a £17 million bid, thereafter declinin g because of the rate of decline of the probability of winning. The textbook solution that maximizes expected profit is to bid at £17 million. In practice, what is vitally important is not providing the decision takers with a recommendation ‘bid £17 million’, but instead giving them Table 12.1 and explaining: . If the expected cost is of the order of £15 million and if Figure 12.2 is roughly consistent with their beliefs, then bidding at £17 million will maximize short-term expected profit, but: (a) a bid of £16 million in- creases the chance of winning from 0.50 to 0.70, with an expected profit reduction of £0.3 million; (b) a bid of £15 million increases the chance of winning from 0.50 to 0.80, with an expected profit reduction of £1.0 million. 236 Harness the plans Figure 12.2—Probability of winning against amount bid. Table 12.1—Profitability of different bids in Example 12.1 bid (£m) probability of winning profit if win (£m) expected profit (£m) 13 0.90 À2.0 À1.80 14 0.88 À1.0 À0.88 15 0.80 0.0 0.00 16 0.70 1.0 0.70 17 0.50 2.0 1.00 18 0.20 3.0 0.60 19 0.10 4.0 0.40 20 0.05 5.0 0.25 . These trade-offs will not be significantly affected by minor changes to the expected cost or Figure 12.2, any decision takers’ ‘what ifs’ being amenable to modelling. . If such analysis is used regularly, recording the probability of winning as forecast by curves like Figure 12.2 will allow feedback to correct any bias in the estimation of such curves. . Such curves are implicit in any bidding process, as are the trade-offs that lead to departures from the short-run, profit-maximizing bid. . The use of a table like Table 12.1 allows quantification and data accu- mulation to test subjective estimates where this is feasible and useful, facilitating the use of this information in conjunction with management judgements about softer issues, such as long-term market advantages associated with winning a bid and the advantages of work to keep otherwise idle or redundant staff busy. Chapman and Ward (2002, chap. 3) elaborates this analysis. Consolidate and explain the strategy: strategic plans It is worth explaining the reference plans and the uncertainty analysis prior to developing an overview of the strategic plans—the focus of this step. The con- trast between the reference plans and the strategic plans that this provides can be interpreted as a selling opportunity for risk management, especially if an RMP is being used for the first time in an organization. However, the more substantive reason is a clear demonstration of the quality of thinking that took a plausible reference pla n and refined it, or reconstru cted it, to produce a risk efficient and robust strategic plan incorporating proactive responses developed by the uncer- tainty analysis. The strategic plans do not need a separate document. In a report that embodies both the reference plans and the uncertainty analysis, the strategic plans can take the form of a summary of recommended proactive responses and other changes to be embedded in the reference plans. However, in conceptual and operational terms it is useful to see the reference plans and strategic plans as separate entities. Users of RMPs at the senior executive and director level should see the pres- ence of a sound strategic plan associated with a sound process for its develop- ment as a key product of the RMP. A go decision for the project should be anticipated at this stage in the RMP, and no significant proj ect management errors of omission or commission should remain. Sometimes it is not feasible to produce such a plan, because of unresolved issues. This is usually a clear sign of impending problems that require immediate management, making a maybe decision for the project the prudent choice. Consolidate and explain the strategy: strategic plans 237 [...]... division, we are now leaving the realm of planning and moving into control, maintaining an ongoing interest in planning Even the very best of plans need adjusting in the heat of battle, but this chapter is also about the role of initiative and training to bridge the gap between what needs to be done and plans that work Planning in this general sense has been decomposed extensively in the earlier chapters... Chapter 17 takes a corporate perspective of project RMPs and considers what is involved in establishing and sustaining an organizational project risk management capability 14 Risk management initiated at different stages in the project life cycle Experience is not what happens to you, it is what you do with what happens to you.—Aldous Huxley Introduction The opportunities for undertaking risk management. .. within a particular PLC stage This can be unfortunate if it implies a limited, ad hoc, bolted-on, optional extra approach to risk management, rather than undertaking risk management as an integral, built -in part of project management Wherever it is carried out in a PLC, risk analysis needs to be regarded as a contribution to risk management of the whole project The opportunities for risk management include... initiated in later PLC stages: first in the allocate stage, then in the execute stage, and so on In each section the focus of the discussion is moving a first use of an RMP in any given project from the plan stage to another PLC stage, but, where appropriate, comments about building on risk management undertaken in earlier PLC stages are incorporated Initiating an RMP in a project s design stage Initiating... appropriate risk- sharing arrangements assessing the implications of contract conditions assessing and comparing competitive tenders determining appropriate target costs and bid prices for contracts estimating likely profit following project termination execute production identify remaining execution issues assessing implications of changes to design or plan revising estimates of cost on completion revising... concerned with providing an interface between analysis as reported in formal documents and a clear understanding of the issues in holistic terms It is also concerned with changing people’s perceptions in advance of formal reporting when culture change issues are involved, and it may involve important aspects of bargaining to achieve this Assess strategy to gain approval Assessing project strategy ought... Initiating an RMP in the design stage involves a significant change in emphasis, but not a totally new process Initiating a RMP such as the SHAMPU process in a project s plan stage can start with a define phase based on a reasonably complete project design Shifting initiation from the plan stage back to the design stage of Initiating an RMP in a project s design stage 259 the project necessarily involves more... management in a project before the plan stage is in general more difficult, because the project is more fluid and less well defined A more fluid 258 Risk management initiated at different stages in the project life cycle project means more degrees of freedom, more alternatives to consider, including alternatives that may be eliminated as the project matures, for reasons unrelated to the RMP A less well-defined project. .. about project objectives, in the limit decomposing project objectives and formally mapping their relationships with project activities, because pre-emptive responses to risks need to facilitate lateral thinking that addresses entirely new ways of achieving objectives Also there is scope for much more fundamental improvements in project plans, perhaps including uncertainty and risk- driven design or redesign... plans without such contingency plans are of limited use Routine project- planning meetings concerned with implementing planned actions should have some of the characteristics of a North American football huddle, including each member of the team being reminded what everyone else is planning to do, how and why they may fail, and reaffirming team bonding Project activity between meetings should have some . general Introduction If project management is decomposed into ‘planning’ and ‘control’, a classic binary division, we are now leaving the realm of planning and moving into control, maintaining. tactics Supporting and convincing with respect to project tactics should be relatively straightforward and quite different in nature from supporting and convincing with respect to project strategy maintaining an ongoing interest in planning. Even the very best of plans need adjusting in the heat of battle, but this chapter is also about the role of initiative and training to bridge the