does “prepare leadership for the truth” mean? Similarly, many generic items only appear in one section of the list. For example, the item “build trusting relationships” only appears in the marketing section but could easily appear in other stages of the change process. The current list may therefore be con- strained in its description of OD competencies. Third, Sullivan’s efforts have been qualitative and consensus building in purpose. He has been trying to get the field to develop and agree on the skills and knowledge necessary to practice OD. As a result, the list hasn’t been subjected to any sort of quanti- tative testing—a requirement of any good competency modeling effort (Roth- well & Lindholm, 1999). We don’t know, for example, whether contracting skills are any more or less important than implementation skills in success- ful OD practice. The time seems right, therefore, to apply a more quantitative approach to determine whether any structure exists within the list and to explore the rela- tive importance of various competencies. STUDY METHODOLOGY To approach the purpose of this study—to explore the structure and utility of the competency list generated by Sullivan and his colleagues—the following methodology was applied. First, the items were reviewed by the authors for use in a survey format. The items were screened for vocabulary (jargon), clarity, assumptions, and bias (Emory, 1980; Sudman & Bradburn, 1983). In particular, several items were “double barreled” and had to be split into two or more questions. For example, one of the original items in the list was “identify the formal and informal power in the client organization in order to gain further commitment and mobilize people in a common direction.” This item was broken up into several items, including “identify formal power” and “confirm commitment of resources.” On the other hand, many items were considered redundant and eliminated. The final number of items used in this study was 141. Second, the questionnaire was designed. The survey was organized into sec- tions similar to the most recent version of the OD competencies and roughly paralleled the action research process. The survey contained nine sections. The first two sections, marketing and start-up, roughly correspond to the entry and contracting phase of the general planned change model. The marketing section contained eighteen items and the start-up section contained thirteen items. The third section was titled diagnosis and feedback and con- tained twenty-three items. The action/intervention planning section contained sixteen items. The fifth section, intervention, contained seven items. The next two sections, evaluation and adoption, contained thirteen and nine items, COMPETENCIES OF OD PRACTITIONERS 141 10_962384 ch05.qxd 2/3/05 12:01 AM Page 141 respectively, and correspond to the evaluation and institutionalization processes of planned change. The eighth section was titled separation and contained five items. The final section presented thirty-seven items under the heading of “other competencies” and represented a list of new items suggested by experts and other contributors to Sullivan’s list of competencies. Respondents were asked whether the competency was essential to success in OD today and, if so, the importance of the item to successful OD practice. The response format for the first question was “yes/no,” while the importance scale ranged from 1 (not at all critical) to 5 (absolutely critical). The intent in the importance scale was to encourage the respondent to make discriminations between competency items that were “nice to have” versus “had to have” for success. The data analyzed here are the importance ratings of the 141 items. Third, the survey was placed on the web and invitations to complete the sur- vey were sent out to OD professionals. This occurred in a variety of formal and informal ways. For example, personal invitations were made during national and international presentations by the authors, the national OD Network invited people through its website home page, and an announcement and invitation were included in several issues of the OD Institute’s monthly newsletter. In addition, electronic invitations were sent to Internet listservs operated by the OD Network, the OD Institute, the Appreciative Inquiry Consortia, ASTD, Pepperdine University’s MSOD program, and the Association for Quality and Participation. Email invitations were sent to key OD leaders. Three hundred sixty-four people responded to the survey. The demographics of the respondents are shown in Table 5.1. The modal respondent was Ameri- can, came from the private sector, and had a master’s degree. The sample rep- resented a broad range of OD experience. Fourth, descriptive statistics were calculated for each item. Then the analy- sis proceeded along two parallel tracks. The first track was an analysis of the 142 PRACTICING ORGANIZATION DEVELOPMENT, 2ND EDITION Table 5.1. Sample Demographics Experience % Education % Sector % Nationality % 0–5 years 20 College 17 Private 73 U.S. 82 graduate 6–10 years 29 Master’s 62 Government 10 Canada 6 11–15years 20 Doctorate 18 Non-profit 12 Europe 4 >16 years 31 Other 3 Education 5 Other 8 *N = 364 10_962384 ch05.qxd 2/3/05 12:01 AM Page 142 items within a section of the survey, and the second track was an analysis of all the items together. In the first analysis, items within a section, such as market- ing or adoption, were factor analyzed to determine whether items possessed any underlying structure. Clusters of items measuring a similar concept were created and labeled. As part of this process, items that did not correlate with other items in the section were identified. That is, a factor analysis includes information about items that either do not correlate well with other items or that are too generic and therefore correlate with too many items to be useful. For example, the item “build trusting relationships” in the marketing section, while no doubt important, was consistently a part of almost every cluster and therefore provided little discriminating information. In the second analysis, all 141 items were submitted to a single factor analy- sis so that the items from one section could correlate with items in another sec- tion. As with the first analysis path, clusters of items measuring similar concepts were created and labeled, and any items that did not correlate with other items or correlated with too many items were identified. In the final step, the two sets of clusters were compared. Where there was similarity between the two sets, a cluster was retained using as many com- mon items as possible. Where unique clusters showed up in either set, the authors conferred and chose the clusters that seemed to best represent a broad scope of OD competencies. Based on this final set of clusters, the research questions concerning structure and utility of the list of competencies were addressed. The process of labeling a set of items to represent an underlying competency is neither scientific nor quantitative. It is, in fact, quite subjective, and the authors toyed with different labels in an attempt to convey the essence of the items within a cluster. This was easy in some cases and much more diffi- cult in others. In the first sectional analysis, we tried to use labels that acknowl- edged the phase of the process they represented. Thus, clusters from the diagnosis section were given labels with diagnosis in mind. In the pooled item analysis, we had more free reign to provide labels reflecting nuances in the mix of items representing the competency. In the end, we chose labels that we thought were fair, but recognize that we should be cautious in believing that our labels are the “right” ones. For the statistically minded reader, several assumptions were made about the data, including the ratio of sample size to the total number of items and whether generically worded items referred to a particular section of the survey. Any definitive OD competency study will have to substantially increase the sam- ple size relative to the number of items. However, the exploratory nature of this study supports a more relaxed set of assumptions. We hope the data presented here can improve the efficiency of any future study. A complete output of this analysis is available from the authors. COMPETENCIES OF OD PRACTITIONERS 143 10_962384 ch05.qxd 2/3/05 12:01 AM Page 143 RESULTS Descriptive Statistics The descriptive statistics from the survey displayed a very consistent pattern. Almost all of the items were rated very high (4) or absolutely critical (5). The range was from 3.5 to 4.9, with only 11 of 141 items having a mean below 4.0 and 51 items having means of 4.5 or higher. Despite the intent to steer respon- dents away from very high scores, nearly all of the distributions are skewed. This is not very helpful, and future studies need to take this into account. In other words, respondents were unable or unwilling to effectively discriminate between the items in terms of their importance. At some level, all of the OD practitioners in this sample are saying “all of these are really important.” It is Sullivan’s belief that only high rates have survived because the list has been scrubbed and revised by so many practitioners over the years. Section-by-Section Analysis In the first analysis, all items within a section were submitted together, but separate from other items and sections in the survey. The analysis produced thirty-two competency clusters using 116 of 141 items. The final results are shown in Table 5.2. Pooled-Item Analysis Table 5.3 describes the final competency clusters from the second analysis, where all items in the survey were submitted together. In this analysis, items from any section of the survey could correlate and form a cluster with items from any other section. The table identifies the name and the number of items in the cluster. In this analysis, thirty-three clusters were produced using 115 out of 141 items. The first fifteen clusters all contained multiple items and ranged in size from two to thirteen items. Three of the clusters contained two items, while the remaining clusters contained between three and thirteen items. The last eigh- teen clusters contained between one and four items, with thirteen of these clus- ters only containing one or two items. In comparison, only eight clusters in the first analysis had one or two items. Thus, the distribution of cluster sizes is more skewed in the second analysis. Many competency clusters in Table 5.3 closely resemble those from the section-by-section analysis. As a result, and in the interest of space, we proceed to a comparison of the two analyses. Comparison of the Two Analyses Table 5.4 presents a comparison of the clusters generated by each of the analy- ses. The table presents the two or more cluster name(s) and the number of items for each cluster where the cluster labels were similar. It then presents the 144 PRACTICING ORGANIZATION DEVELOPMENT, 2ND EDITION 10_962384 ch05.qxd 2/3/05 12:01 AM Page 144 COMPETENCIES OF OD PRACTITIONERS 145 Table 5.2. Section-by-Section Results Competency Label # of items Marketing Ability to describe OD processes 7 Quickly assess opportunities for change 4 Clarify outcomes and resources 3 Develop relationships 2 Make good client choices 1 Start-Up Set the conditions for change 4 Address power 3 Build cooperative relationships 3 Clarify roles 2 Diagnosis/Feedback Research methods 6 Keep the information flowing 5 Clarify data needs 4 Keeping an open mind re: data 3 Relevance 1 Action Planning Creating an implementation plan – I 4 Creating an implementation plan – II 3 Facilitate the action planning process 3 Obtain commitment from leadership 2 Intervention Adjust implementation 4 Transfer ownership of the change 3 Evaluation Ability to evaluate change 5 Use evaluation data to adjust change 4 Adoption Manage adoption and institutionalization 9 Separation Manage the separation 5 Other Competencies Master self 8 Be available to multiple stakeholders 7 Ability to work with large scale clients 4 Manage diversity 3 Be current in theory and technology 4 Maintain a flexible focus 2 Possess broad facilitation skills 2 Be comfortable with ambiguity 2 10_962384 ch05.qxd 2/3/05 12:01 AM Page 145 146 PRACTICING ORGANIZATION DEVELOPMENT, 2ND EDITION Table 5.3. Pooled-Item Analysis Results Competency Label # of items Self-mastery 13 Ability to evaluate change 6 Clarify data needs 4 Manage the transition and sustain momentum 8 Keep information flowing 7 Integrate theory and practice 6 Ability to work with large systems 6 Manage the separation 3 Participatively create a good action plan 6 Apply research methods appropriately 4 Manage diversity 4 Imagination skills 2 Focus on relevant issues 5 Clarify roles 2 Address power 2 Clarify outcomes 1 Keep an open mind regarding data 2 Stay current with technology 2 Apply effective interpersonal skills 3 Set appropriate expectations 4 Let data drive action 3 Manage ownership of change 3 Be mindful of process 2 Think systemically 3 Comfort with ambiguity 3 Action plan with results in mind 1 Involve leadership 2 Be credible 2 Be a quick study 2 Monitor the environment 1 Network your services 1 Make good client choices 1 Get leadership commitment 1 10_962384 ch05.qxd 2/3/05 12:01 AM Page 146 Table 5.4. Comparison of Competency Clusters Sectional Analysis Pooled Items Analysis Final Competence Name Competence Name # of items Competency Representative Items (# of items) (# of items) in common Label Self-mastery (8) Self-mastery (13) 8 Self-mastery Be aware of how one’s biases influence interaction Clarify personal values Clarify personal boundaries Manage personal biases Manage personal defensiveness Recognize when personal feelings have been aroused Remain physically healthy while under stress Resolve ethical issues with integrity Avoid getting personal needs met at the expense of the client Ability to evaluate Ability to evaluate 4 Ability to Choose appropriate evaluation methods change (5) change (6) evaluate Determine level of evaluation change Ensure evaluation method is valid Ensure evaluation method is reliable Ensure evaluation method is practical Clarify data Clarify data 2 Clarify data Determine an appropriate data collection process needs (4) needs (4) needs Determine the types of data needed Determine the amount of data needed 10_962384 ch05.qxd 2/3/05 12:01 AM Page 147 Table 5.4. Comparison of Competency Clusters Sectional Analysis Pooled Items Analysis Final Competence Name Competence Name # of items Competency Representative Items (# of items) (# of items) in common Label Manage adoption Manage the 4 Manage Help manage impact to related systems and institutionali- transition and transition and Use information to correct negative change zation (9) sustain institutionali- Transfer change skills to internal consultant so learning is momentum (8) zation continuous Maintain/increase change momentum Mobilize additional internal resources to support continued change Determine the parts of the organization that warrant a special focus of attention Ensure that learning will continue Stay current in Integrate theory 4 Integr ate theory Present the theoretical foundations of change theory and and practice (6) and practice Articulate an initial change process to use technology (4) Be current in Stay current in Integrate research with theory and pr actice technology (2) technology Communicate implications of systems theory Utilize a solid conceptual framework based on research Use the latest technology effectively Use the Internet effectively Ability to work Ability to work 4 Ability to w ork Facilitate large group (70–2,000) interventions with large-scale with large with large Apply the skills of international OD effectively clients (4) systems (6) systems Function effectively as an internal consultant 10_962384 ch05.qxd 2/3/05 12:01 AM Page 148 Sectional Analysis Pooled Items Analysis Final Competence Name Competence Name # of items Competency Representative Items (# of items) (# of items) in common Label Demonstrate ability to conduct transorganizational development Demonstrate ability to conduct community development Consider creative alternatives (?) Create an Participatively 5 Participatively Co-create an implementation plan that is (1) concrete, (2) implementation create a good create a good simple, (3) clear, (4) measurable, (5) r ewarded, and plan – I (4) implementation implementation (6) logically sequences activities plan (6) plan Create an implementation plan – II (3) Understand Apply research 4 Understand Utilize appropriate mix of methods to ensure (1) efficiency, research methods research (2) objectivity, and (3) validity methods (6) appropriately (4) methods Utilize appropriate mix of data collection technology Use statistical methods when appropriate Manage diversity Manage diversity 3 Manage Facilitate a participative decision-making process (3) (4) diversity Be aware of the influences of cultural dynamics on interactions with others Interpret cross-cultural influences in a helpful manner Handle diversity and diverse situations skillfully 10_962384 ch05.qxd 2/3/05 12:01 AM Page 149 Table 5.4. Comparison of Competency Clusters Sectional Analysis Pooled Items Analysis Final Competence Name Competence Name # of items Competency Representative Items (# of items) (# of items) in common Label Clarify roles (2) Clarify roles (2) 2 Clarify r oles Clarify the role of consultant Clarify the role of client Address power (3) Address power (2) 2 Addr ess power Identify formal power Identify informal power Deal effectively with resistance Keep an open Be patient (2) 2 Keep an open Suspend judgment while gathering data mind (3) mind Suppress hurtful comments during data gathering Transfer ownership Manage client 3 Manage client Reduce dependency on consultant of change (3) ownership of ownership of Instill responsibility for followthr ough change (3) change Involve participants so they begin to own the process Be comfortable Be comfortable 2 Be comfortable Perform effectively in an atmosphere of ambiguity with ambiguity (2) with ambiguity (3) with ambiguity Perform effectively in the midst of chaos Manage the Manage the 2 Manage the Be sure customers and stakeholders are satisfied with the separation (5) separation (3) separation intervention’s results Leave the client satisfied Plan for post-consultation contact Recognize when separation is desirable 10_962384 ch05.qxd 2/3/05 12:01 AM Page 150 . conduct transorganizational development Demonstrate ability to conduct community development Consider creative alternatives (?) Create an Participatively 5 Participatively Co-create an implementation. of Instill responsibility for followthr ough change (3) change Involve participants so they begin to own the process Be comfortable Be comfortable 2 Be comfortable Perform effectively in an atmosphere. analy- ses. The table presents the two or more cluster name(s) and the number of items for each cluster where the cluster labels were similar. It then presents the 144 PRACTICING ORGANIZATION DEVELOPMENT,