What Was Good? Positive results should be put on your Good list.These are processes that you might never have tried before during an assessment but that worked out well. These include such things as the change in interview style we mentioned earlier. Another example might be a new interviewer on the team who performed well and can added to the list of team leaders for your company. Our goal is to iden- tify the good attributes of our assessment process. N OTE Good lessons are just as important to understand as the negatives lessons. I’ve seen many organizations that approach the analysis of lessons learned as a pessimistic activity that generally only points out negative activities. This couldn’t be further from a healthy approach. The truth of the matter is that the process needs at least as much positive reinforcement as negative. Consider some of the activities that might have seemed “spur of the moment” when they were performed but eventually added value to the assessment process. This is important because it reassures team mem- bers that individual thought about the way our process assessment unfolds is a good attribute. Team members with positive attitudes will do much more to improve the process than those with negative atti- tudes. What Requires Improvement? Negative results should be put on your Poor list. Negative items might include processes that perform poorly in certain situations or the lack of a needed process altogether.These aren’t necessarily things you’ve known about for months, which is why we call them lessons learned.Your team will pick up these tidbits of infor- mation through experience and from assessing a variety of customers and indus- tries. In actuality, you should view these lessons as positive since they give the team an opportunity to improve immature processes. www.syngress.com Tying Up Loose Ends • Chapter 11 389 286_NSA_IAM_11.qxd 12/16/03 1:01 PM Page 389 Utilizing Lessons Learned Now you’ve created these lists of the lessons you’ve learned during the assess- ment process and you’re trying to figure out what to do with them. How do we integrate lessons learned into the IAM? The lessons we learn during assessments, if analyzed properly and taken advantage of, can lead to continuous process improvement. After all, continuous improvement of our processes will create a better product and hopefully generate more business for our company. Integrating Lessons Learned into the Business Process NSA does not provide information in the structure if the IATRP regarding how organizations should integrate lessons learned.This activity is normally a business process and should be considered unique to each organization. But there are a few things that appear to remain the same, regardless of the organization, when we try to integrate lessons learned.These things are: ■ Identifying lessons that provide value ■ Integrating the solution into normal procedures ■ Providing tracking of the process for future assessments The first step is deciding which lessons learned during the assessment actually provide value. As you analyze your lists, try to envision how each item can pro- vide value. What does the lesson give us that we don’t have covered in other pro- cesses? Is the lesson a result of not fully implementing or conducting processes that already exist, or is it a totally new process that needs to be considered? If our lesson is something that can be addressed in a process we already perform, it probably makes sense to adjust the existing process to address our lesson. If the process is something we haven’t previously utilized, we should consider inte- grating it into our normal assessment procedures. The integration of a new process can be difficult for an inexperienced assess- ment team. New team members don’t always adjust as well as we’d like and can forget new procedures. We know from experience that consultants in any field are liable to work on autopilot, allowing themselves to be carried through the process by their own habits. To counter this possibility, the organization conducting the assessments should consider creating methods for tracking the assessment process, including each individual process that occurs.The easiest method for doing something like this is to create a master checklist of activities that must be performed.As the www.syngress.com 390 Chapter 11 • Tying Up Loose Ends 286_NSA_IAM_11.qxd 12/16/03 1:01 PM Page 390 IAM assessment progresses, each team member will find themselves responsible for different pieces of the assessment process. If we include processes designed to address previous lessons learned, we ensure that each process is seen and addressed by the team members.A sample checklist is shown in Figure 11.3. NOTE Figure 11.3 is not a complete checklist but instead provides a sample of what can be done to track assessment activities. The actual document used should be customized to your organization’s own Figure 11.3 Sample Assessment Checklist Customer Mission and Data: 1. The assessment team and the customer have come to an understanding of: ■ The scope of the assessment ■ The way the assessment process works ■ The level of detail required in recommendations Document name/location:____________________________________ 2. The assessment team understands the customer mission, goals, and objectives. Document name/location:____________________________________ 3. The assessment team and the customer have defined the types of information the customer processes. Document name/location:____________________________________ 4. The assessment team and the customer have come to an understanding as to the perceived value of the customer data and information to the customer. Document name/location:____________________________________ Customer Criticality Matrices: 5. The assessment team and the customer have determined what information is critical to the customer mission and the systems containing that data. Document name/location:____________________________________ www.syngress.com Tying Up Loose Ends • Chapter 11 391 Continued 286_NSA_IAM_11.qxd 12/16/03 1:01 PM Page 391 Figure 11.3 Sample Assessment Checklist 6. The assessment team has worked with the customer to define the impact values associated with the OICM. Document name/location:____________________________________ 7. The defined impact values have been assigned to the customer data areas defined in Step 5 in relation to loss of CIA. Document name/location:____________________________________ 8. The sum of organizational criticality has been determined and documented. Document name/location:____________________________________ Making It Repeatable Another advantage of using a checklist similar to the example in Figure 11.3 is that it will also help keep the process repeatable.The repetition of each step through every assessment is key for maintaining a mature and reliable assessment process.Team members who have been doing assessments for a while don’t nor- mally have many questions about the process.They’ll be comfortable with the methods the organization uses to assess customers. New team members, however, need guidance to ensure a high-quality final product for the customer. Processes may seem foreign or new to these members. Consider the confusion new consultants may encounter when they first start using your organization’s methodology. One method for countering this learning curve is to add an easy-to-follow checklist that provides a foundation for assess- ment activities.This allows newer team members to gain a better understanding of the events that are supposed to occur within the assessment. In the end, the customer will have a higher-quality product and your team members will be more confident in their work. Hopefully this will also contribute to a more pro- ductive and cohesive team environment. Certainly there are other methods for ensuring that the processes are repeat- able. Creating standard processes and documenting these processes, in some form or another, will aid in creating an environment or repetition. Whether you use a checklist similar to the example in Figure 11.3 or you create something totally new, standardizing your activities will ensure that each assessment is conducted similarly, all team members are comfortable with the way the assessment is con- ducted, and the customer receives the same level of service regardless of what individuals are on the team. www.syngress.com 392 Chapter 11 • Tying Up Loose Ends 286_NSA_IAM_11.qxd 12/16/03 1:01 PM Page 392 WARNING The value of repeatable processes cannot be overstated. Organizations such as the ISO have created criteria and certification programs for orga- nizations that want to improve and demonstrate their ability to continue functioning. Repeatable processes and the documents that define these processes are key. Having the ability to maintain a higher level of perfor- mance over an extended period of time indicates maturity within the business and assures customers that they will receive the same level of product and service. Case Study: The University of Science The University of Science is a typical higher education institution focused on providing return value to the various industries the university supports through education, research, and development. Our organization was contracted several months ago to provide an IAM-based assessment of this educational institution. The assessment process went well and uncovered a large number of issues of which the customer was not previously aware. Understanding the Requirements According to our contract and statement of work with the customer, we did not have an obligation to provide document retention services.The customer had not expressed an interest in the service until the assessment was in full swing.The problem was that our company does not offer this service as a core competency. In order to help the customer in this area, we recommended a partner com- pany to the customer. Our partner has been providing these services for the past five years as part of its business continuity offering.The partner was appropriately equipped and able to offer this service to our client. But the piece of this recom- mendation that we could help the customer with concerns deciding what docu- mentation should be kept and what documentation should be destroyed. What Should We Keep? Initially, we concentrated on those documents that should be retained as part of the security trend for the customer organization. We recommended that the final report be retained as part of a good security program because it provides legacy www.syngress.com Tying Up Loose Ends • Chapter 11 393 286_NSA_IAM_11.qxd 12/16/03 1:01 PM Page 393 information on where the customer organization began addressing security and its progress thus far.The technical information and recommendations belong to the customer. Retaining this information depends heavily on the customer goals regarding the information. In this situation, the customer decided that once cur- rent findings have been resolved, the technical information will be retained as legacy documentation. All documents will be kept for three years to provide his- torical data for future assessment efforts. What Should We Destroy? The decision about what documents would be destroyed was relatively easy.The customer already had copies of all the standards and regulations used during the assessment process.Those documents could be destroyed since no new versions of those documents were released.The documents we created during the assess- ment in relation to our interviews were to be destroyed.The NSA IAM teaches that any notes taken during the assessment process should remain anonymous in order to keep the assessment process in a state of nonattribution.The customer was made aware of these issues during the development of the statement of work. According to our contract with the customer, the interview notes would be destroyed.The only exception is the information in the final report, which was combined from all the individual sources. Designating a Followup POC Since the delivery of the final report at the beginning of last week, we haven’t heard back from the customer. We’ve about reached the point when we need to consider following up with the customer.The team leader previously designated the POC for each area of the assessment and now gives the go-ahead to each team member to begin the followup process. The team POCs were selected based on their knowledge of the subject areas we dealt with during this assessment. Mike was selected to provide followup on the disaster recovery area because he has years of experience in this area and should be able to provide knowledgeable help to the customer. Sarah was chosen to follow up regarding the UNIX heavy environment at the customer location. The team leader will follow up with the customer POC concerning any issues or questions related to the final report, its format, or any other assessment-related questions. www.syngress.com 394 Chapter 11 • Tying Up Loose Ends 286_NSA_IAM_11.qxd 12/16/03 1:01 PM Page 394 What Have We Learned? Our last step is to analyze the lessons we learned during the assessment process. As in most assessments, some of our lessons learned are positive, others are of a more negative nature.The team leader lists the lessons learned in order to eval- uate their eventual value to our assessment process.All the team members have the freedom to submit issues as lessons learned. Each lesson is then analyzed one by one to determine its value and relevance to our assessment process. Our lessons learned include a new report format that seems a better fit for the assessment work being performed and a method of holding interviews in a group setting.The team sits down together to judge the value of these two lessons.The new suggested report format is actually just an expansion of what is already being done.The value provided is the customer’s clearer understanding of report findings.The team agrees to integrate this lesson into future assessments by including the new information in the template for our final reports. The second lesson deals with holding group interviews for the user commu- nity at large organizations.This allows us to get a better overall feel for the actual understanding of the customer security environment while making it clear to users that if there are concerns, they can contact the team offline to discuss the issues in private.The team discusses this second lesson and determines that this activity already occurs and does not require integration into the current assess- ment process. www.syngress.com Tying Up Loose Ends • Chapter 11 395 286_NSA_IAM_11.qxd 12/16/03 1:01 PM Page 395 Summary Document retention is not directly covered in the NSA IAM beyond simply stating that the information is customer proprietary and does not belong to the organization conducting the assessment. If you’re performing these assessments, consider all documentation sensitive. Documents should never be held by the assessing organization beyond a 90-day period.This time period enables you to answer any customer concerns or questions. Special conditions may exist where the customer has asked you to provide document retention services.There is a significant level of liability associated with maintaining sensitive documentation regarding customer security postures. Special storage requirements may exist, such as physical security concerns, storage space, and file system security. Other concerns include the backup and restora- tion of this information for business continuity purposes or understanding the ramifications of a compromise of customer data on your organization.The long- term retention of sensitive customer information is discouraged unless this is a core competency of your organization and is not covered by NSA in the IAM training course. Following up with the customer is a highly valuable activity. It can lead to answers to questions the customer might not have been capable of asking directly or questions they might not have asked for fear of sounding unintelligent.These activities are not covered directly by the NSA IAM beyond stating that followup is necessary. To ensure the highest quality of followup with the customer, the team member performing the activity needs to show appropriate concern for the cus- tomer’s situation. Remember to be tactful in all your dealings with the customer. Don’t make statements that can be misconstrued or misinterpreted.Try to remain friendly during the process. Assessments can be frustrating, but keep in mind that the customer is paying the bill and will likely talk to friends and colleagues con- cerning the assessment.Your ability to provide responsive and caring followup could provide opportunities for more work. Although not addressed in detail by NSA during the IAM training, the pro- cess of evaluating lessons learned is important for ensuring the continuing growth and evolution of your assessment services. Lessons can be negative or positive and should be integrated into your processes only if they provide ade- quate value. In some cases, lessons can be integrated into processes that already partially meet our requirements. www.syngress.com 396 Chapter 11 • Tying Up Loose Ends 286_NSA_IAM_11.qxd 12/16/03 1:01 PM Page 396 Best Practices Checklist Examining Document Retention  Understand the contract requirements for document retention.  Understand the liability for accepting responsibility for document retention.  Organize your documentation by areas: public domain, customer, and generated.  Consider the security requirements of retaining sensitive documentation.  Look into alternatives and partnerships if document retention is not a core competency of your business. Performing Customer Followup  Followup is a great method for eliminating customer confusion and ensuring customer satisfaction.  Express genuine concern for the issues the customer is facing.  Ask the right questions to obtain useful answers. Consider creating a baseline list of questions to begin the followup process.  Designate responsible team members for each portion of the fol- lowup process, and communicate the information to the team to ensure they’re prepared.  Track the followup process to ensure that no customer questions or concerns fall through the cracks. Evaluating Lessons Learned  Analyzing the lessons we learn during the assessment process helps create maturity within the process.  Lessons we learn during each assessment can be either positive or negative. www.syngress.com Tying Up Loose Ends • Chapter 11 397 286_NSA_IAM_11.qxd 12/16/03 1:01 PM Page 397  Negative lessons indicate areas that need improvement or enhance- ment.  Positive lessons promote team involvement and assessment process evolution.  Integrate the lessons that provide value into the overall assessment process so that they will continue to be used in future endeavors. Q: Does your company provide document retention services, and if not, why not? A: Document retention is simply not one of our core competencies, and until we decide to focus on that business area, we’re ill equipped to deal with the logistics or legalities of storing sensitive customer information. When cus- tomers inquire about this service, we refer them to partner entities that focus on this area and can provide better value. From a business perspective, this strategy allows us to remain strong in those areas we’re best at without get- ting sidetracked just to earn a dollar. Q: Does followup need to be performed on every size of customer organization, or should we really only concentrate on larger customers? A: As a business, we do follow up with every customer organization, regardless of size. In the end this is really a business decision, but we don’t feel that smaller customers are any less important than our larger customers. It’s takes a little extra time and it can be uncomfortable sometimes, but it’s worth it when the customer is satisfied with the results. Q: The example you give for an assessment checklist seems pretty generic. Do you know of any better examples? www.syngress.com 398 Chapter 11 • Tying Up Loose Ends Frequently Asked Questions The following Frequently Asked Questions, answered by the authors of this book, are designed to both measure your understanding of the concepts presented in this chapter and to assist you with real-life implementation of these concepts. To have your questions about this chapter answered by the author, browse to www.syngress.com/solutions and click on the “Ask the Author” form. You will also gain access to thousands of other FAQs at ITFAQnet.com. 286_NSA_IAM_11.qxd 12/16/03 1:01 PM Page 398 [...]... www.syngress.com 399 286 _NSA_ IAM_ 11.qxd 12/16/03 1:01 PM Page 400 286 _NSA_ IAM_ AppA.qxd 12/16/03 12:37 PM Page 401 Appendix A Forms, Worksheets, and Templates NOTE In addition to copying the documents available on the next few pages, you can download versions of the documents from the Syngress Solutions Web site for the title Security Assessment: Case Studies for Implementing the NSA IAM 401 286 _NSA_ IAM_ AppA.qxd... 27–28 protection measures, 242 NIST 800-26 Security Self Assessment Guide, 253 nonattribution, 54, 55 nonexistent documentation, 173 nonrepudiation, 97–98 notes, 252 NSA IAM See National Security Agency (NSA) Information Security (INFOSEC) Assessment Methodology (IAM) NSA (National Security Agency), 2, 3 NSA Security Recommendation Guides, 274 numbered scales, 103 104 O objectives, 69 Occupational Safety... example impact definitions, 100 104 impact to organization, 99 100 OICM, creating, 104 108 refining for SCM, 135–136 in SCM case study, 143 in TAP, 193, 207 of TOOT, 110 111 impact attributes in case study, 287 286 _NSA_ IAM_ Index.qxd 422 12/17/03 10: 31 AM Page 422 Index defined, 94 definitions, creating, 99 108 identifying, 93–99 prioritizing impact based on definitions, 105 107 review in closeout meeting,... the POC for multiple areas of the assessment process? A: Yes .The members of the assessment team should be able to cover multiple areas of knowledge within the assessment process For example, there is a good chance that the team member with Windows or UNIX experience will likely have network knowledge as well Assigning that team member the responsibility of following up with the customer in those two areas... criticality information exchange, 231–232 information request, 69–70 Information Security (INFOSEC) assessment, 6, 23 documentation, 234 NSA IAM for, 2 roles/responsibilities, 234–235 Information Security (INFOSEC) baseline classes/categories, 232–246 in general, 232–233 management aspects, 233–236 operational aspects, 243–246 technical aspects, 236–243 information technology (IT) framework, 48–49 information... pre -assessment meeting, 85–93 See also closeout meeting methodology of assessment, 7–8 mission associating information types with, 90–93 identification, 66–69 organizational, defining, 46–47 section of TAP, 192, 204–205 mission-oriented types, 93 mode of operation, 158 N National Security Agency (NSA) , 2, 3 National Security Agency (NSA) Information Security (INFOSEC) Assessment Methodology (IAM) assessment. .. I Information Security Policy I Information Systems Security Policy I Internet Usage Policy I IT Strategy I Mission Statement I Organization Chart I Organizational Description I Organizational Security Policy/Procedures I Personnel Security Policy I Physical Security Policy I Security Policy I Security Strategy I Strategy Document www.syngress.com 286 _NSA_ IAM_ AppA.qxd 12/16/03 12:37 PM Page 409 Forms,... information criticality in case study, 108 –112 critical information topics, identifying, 86–93 definition of, 83 impact attribute definitions, 99 108 impact attributes, identifying, 93–99 steps for process of, 82–86 in TAP case study, 205, 206–207 in technical assessment plan, 193–194 Organizational Information Criticality Matrix (OICM) in closeout meeting, 316, 325–326 creating, 104 108 , 111–112 security. .. 411 286 _NSA_ IAM_ AppA.qxd 412 12/16/03 12:37 PM Page 412 Appendix A • Forms, Worksheets, and Templates Elements of the Technical Assessment Plan The following are elements that you may find helpful when creating your TAP However, we encourage you to thoroughly read Chapter 6, “Understanding the Technical Assessment Plan,” to gain a further understanding and prescriptive advice regarding the format of... decisions for the customer during an IAM- based assessment. The customer knows best about their own business, but your team should provide guidance and expertise on procedures that should be considered When it comes to document retention or destruction, you can make recommendations to the customer, but the final decision should be the customer’s to make Q: Can a single team member be the POC for multiple . Solutions Web site for the title Security Assessment: Case Studies for Implementing the NSA IAM. Appendix A 401 286 _NSA_ IAM_ AppA.qxd 12/16/03 12:37 PM Page 401 IAM Pre -Assessment Site Visit Checklist Organization. covered in the NSA IAM beyond simply stating that the information is customer proprietary and does not belong to the organization conducting the assessment. If you’re performing these assessments, consider. make recommendations to the customer, but the final decision should be the cus- tomer’s to make. Q: Can a single team member be the POC for multiple areas of the assessment process? A: Yes .The members of the assessment