628 Hands-On Microsoft SQL Server 2008 Integration Services This wizard offers the following Package Management Options during upgrade process: Update connection strings to use new provider names c Updates the connection strings to use the SQL Server 2008 names for the OLE DB Provider for Analysis Services and the SQL Server Native Client provider. Note that the SSIS Package Upgrade Wizard updates only the connection strings that are stored in connection managers and not the ones that are constructed dynamically at run time. Validate upgrade packages c Validates the upgraded packages and saves only those upgrade packages that pass the validation. Create new package ID c Creates new package IDs for the upgraded packages. Continue upgrade process when a package upgrade fails c Continues to upgrade the remaining packages when a package cannot be upgraded. Figure 14-18 Upgrading multiple packages with SSIS Package Upgrade Wizard Chapter 14: Migrating to Integration Services 2008 629 Backup original packages c When you are upgrading packages that are saved in the file system, it allows you to create backups of original packages during the upgrade process (Figure 14-19) so that you can continue to use existing packages if the upgrade process doesn’t complete successfully. e original packages are backed up in a subfolder called SSISBackupFolder created by this upgrade wizard. After completing the upgrade wizard, the packages get migrated to new version and can no longer be converted back to SSIS 2005. If you need to use the older version, you will have to get a copy from the backup. Irrespective of the tool or the method you use, the upgrade process migrates the c existing VSA compatible scripts in any Script task and Script component to Microsoft Visual Studio Tools for Applications (VSTA). Figure 14-19 Package Management Option with SSIS Package Upgrade Wizard 630 Hands-On Microsoft SQL Server 2008 Integration Services When you execute an SSIS 2005 package using SSIS 2008 dtexec utility, the c utility temporarily migrates the package to SSIS 2008 format before execution. It does not save the upgraded package, however. If there is any issue with migration, the package will not be executed. When you open and load an SSIS 2005 package after adding it to a project in c BIDS 2008, it automatically gets converted to SSIS 2008 format. If you save this converted package, the upgrade becomes permanent and cannot be reverted back. Most of the SSIS 2005 components seamlessly migrate to SSIS 2008 format, c while some might have issues and could generate warning messages. For example, certain connection strings could generate warning messages while the Lookup transformation will migrate automatically, though you may prefer to re-configure the new Lookup transformation to take advantage of additional features. No custom components or third-party components will be migrated. You will c have to manually recompile these components to enable them to work with SQL Server 2008 Integration Services. Summary Integration Services gives you the option of running your DTS 2000 package without making any changes using the enhanced run-time support. However, you may find yourself at a crossroads in deciding when to upgrade your packages, as Data Transformation Services has been marked deprecated and will not be supported in the future versions. You’ve also learned about the provision of the SQL Server 2000 DTS Designer components, the method to include or run your DTS 2000 packages in Integration Services projects using the Execute DTS 2000 Package task, and you have used the Package Migration Wizard to migrate a DTS 2000 package to Integration Services. One important thing you learned about migrating your DTS 2000 packages is that there is no straightforward method, though the Package Migration Wizard makes it easy to do; still, you may have to mend some configurations yourself. In the second part of the chapter you learned about migration options from SSIS 2005 and found that it is much easier compared to migrating from DTS 2000. Troubleshooting and Performance Enhancements Chapter 15 In This Chapter c Troubleshooting Integration Services Packages c Performance Enhancements c Performance Monitoring Tools c Using Parallel Processing c Summary 632 Hands-On Microsoft SQL Server 2008 Integration Services Y ou’ve come a long way with Integration Services and have worked with its various aspects to configure and develop SSIS packages. By now, not only you can develop workflow and data transformations in your packages but can also deploy them to an enterprise infrastructure securely with complete administration and control over the storage locations. In this chapter, you will learn the skills necessary for debugging your packages, in general and at the component level. You will then learn about the facilities that you can use to determine what happens at package run time to fix these issues accordingly. You must have come across various alerts and warning messages raised by Integration Services throughout your experience with package development. In this chapter you will explore how Integration Services validates component configurations in your packages and handles errors. In the second half of the chapter, you will learn to configure your packages performing at peak levels. You’ll explore the performance counters provided by Integration Services, custom events of the Data Flow task, execution trees and execution plans, among other options to help you keep memory usage in control. This chapter also offers a high-level discussion on package design techniques to optimize the pipeline and exploit the parallelism features. Let’s start the last leg of our journey with Integration Services by learning to troubleshoot SSIS packages. Troubleshooting Integration Services Packages During package development you might have seen various alerts and warnings appear on the screen. For example, when you drop the Execute SQL task on the Control Flow surface, you see a crossed red glyph on the task indicating “No connection manager is specified.” These validation alerts are available by default in Business Intelligence Development Studio (BIDS). Some features are available in Integration Services by default, so you don’t need to set any configurations; other debugging features, such as breakpoints and logging, must be configured and you must interpret the results. You can broadly lump these features into two categories: Debugging features that are available by default c Debugging tools that are required to be configured c The following section covers the features that are available to you by default. Debugging Features Available by Default Integration Services provides several features that help you in debugging packages by alerting and providing useful information that you might have overlooked while configuring components in your package. Various windows display variable values, expression results, alerts, and other messages. Following are brief descriptions of some of these windows. Chapter 15: Troubleshooting and Performance Enhancements 633 Error List Window You can launch this window from the View menu in BIDS. It lists errors and warning messages during design time, such as validation errors. The messages are generally descriptive, stating the type of error, source of the error, and description of the error. And if you find it a bit difficult to link the error message to the source task in a complex package, you can double-click the error message to open the editor for that particular task. Locals Window This window is quite helpful in debugging by displaying local variables in the current scope. The Locals window is available only when you’ve suspended execution in your package using breakpoints. While execution is suspended at breakpoints, you can launch this window by choosing Debug | Windows to see the state or current values of local variables at that time. Watch Window This window is a great debugging facility that lets you watch for specific variables during package execution. As with the Locals window, you can launch this window by choosing Debug | Windows when your package has suspended execution after encountering a breakpoint. You can directly type in a variable to see its value or set a watch for that variable. You can also assign a new value to the variable manually and inject this value into the package execution to see the results or modify the execution process. You will use this feature to modify the execution of the package in the following exercise. Output Window This window is also available from the View menu. It shows results of execution, like the Progress tab described next, and hence includes errors that occur during run time. You will be using this window quite a lot to check status or error messages, more often than the Progress tab, which is a lot more verbose such that it becomes difficult to look for desired messages in it. Progress Tab The Designer includes a Progress tab during package execution. It displays lot of progress information about the package and its components. You can find start and stop times, validation errors or warnings, and other execution errors or warnings about the components of the package here. When package execution is finished and you switch back to design mode, this tab changes to the Execution Results tab and shows the messages of the last execution of the package. 634 Hands-On Microsoft SQL Server 2008 Integration Services Debugging Tools Requiring Configuration The debugging tools such as breakpoints, logging, precedence constraints, data flow paths, and data viewers must be configured in order to use them. Breakpoints As SSIS Designer is based in the Visual Studio environment, it can leverage some of the functionality provided by this programming environment. You can set a breakpoint in your Integration Services package, for example, to help you debug the control flow in your package at run time. Breakpoints suspend the execution of the package and let you review run-time environment conditions, such as variable values and the state of your package at that particular point in execution. You can set breakpoints at the package level, on containers, and on Control Flow tasks. You can also use breakpoints in your scripts while working with the Script task and break your code at execution time. The VSTA environment is used in Integration Services to develop and debug scripts. This enables you to apply breakpoints to your scripts to pause execution during debugging. You can apply breakpoints in one of the following ways: Right-click at the left of the row and choose Breakpoint | Insert Breakpoint c option from the context menu. Select the line where you want to insert the breakpoint and choose Debug | c Toggle Breakpoint, or press the 9 key. When you set a breakpoint on a line, it is highlighted in red and a red circle appears to the left of the line. The run-time handling of breakpoints in the scripts is the same as handling them while running the package in the Designer. You can step into, step over, or step out of the breakpoint using the provided options from Debug menu. VSTA provides a debugging window that you can use with breakpoints to get information about what’s happening with script execution. You can print variable values, execute procedures, or evaluate expressions using this window. This is equivalent to the Locals and Watch windows provided in the SSIS Designer. Let’s work through a Hands-On exercise to see how breakpoints can be set within your packages. Hands-On: Setting Breakpoints to See Variables Values You will be setting breakpoints at various levels in an Integration Services package to see how variables change values during execution. Chapter 15: Troubleshooting and Performance Enhancements 635 Method In this exercise, you will be using the Contacting Opportunities with Property Expressions Integration Services project you developed in Chapter 8. This project creates personalized e-mail messages. The package contains a Foreach Loop container that executes seven times, once for each segmented record; reads the October Prospects contact details such as first name, last name, and e-mail address; and passes this information on to the Send Mail task that creates personalized e-mail using this information. The Foreach Loop container passes this information to the Send Mail task using variables. You’ve seen the package executing but did not see how the variables change the values. In this exercise, you’ll be using breakpoints to see exactly how the variables are getting updated. Exercise (Set Breakpoints) Here, you will continue working with the Mailing Opportunities.dtsx package and set the breakpoints to see the runtime values. 1. Open the Contacting Opportunities with Property Expressions project in BIDS and then open the Mailing Opportunities.dtsx package in the Designer. 2. Right-click the Iterating October Opportunities Foreach Loop Container and choose Edit Breakpoints from the context menu. The Set Breakpoints dialog box will open. 3. In this dialog box, you can set breakpoints using various break conditions. Take a moment to read through the conditions that are available here. For this exercise, you will be using the Break at the beginning of every iteration of the loop break condition, the last in the list. Select the check box at the left of this break condition. You will see the Hit Count Type column displaying Always for this break condition. Click this and then click the down arrow to see the following four options in the Hit Count Type column: Always c Execution will always be suspended. Hit count equals c Specify a value in the Hit Count column when you want to skip some execution cycles of the component—such as when your container iterates over a record set that contains thousands of records and you want to see the variable values after a few thousand iterations. e breakpoint will suspend execution after the number of breakpoint occurrences has reached the value specified in the Hit Count column. Hit count greater than or equal to c Similar to Hit count equals option, except the execution is suspended when the number of times the breakpoint has occurred is equal to or greater than the value specified in the Hit Count column. e execution will be suspended each time the breakpoint is hit after 636 Hands-On Microsoft SQL Server 2008 Integration Services the Hit Count value is reached, whereas Hit count equals breaks the execution only once when the Hit Count value is equal to the number of times the breakpoint has occurred. Hit count multiple c You may be interested in seeing variable values at intervals to test that the package is working as expected. You can use this option and specify a value in the Hit Count column to break the execution only when the breakpoint occurrence is a multiple of this value. For example, you may set a value of 100 in the Hit Count column with this option and the execution will be suspended every 100th time the breakpoint occurs. For this exercise, leave Always selected, and you don’t have to specify any value in the Hit Count column (see Figure 15-1). Click OK to close the Set Breakpoints dialog box. 4. You will see a red circle on the Iterating October Opportunities Foreach Loop container, indicating that a breakpoint has been set on this component. Press 5 to execute the package. 5. You will see that the October Opportunities task has been successfully completed and the execution is stopped at Iterating October Opportunities with this container appearing in yellow, and the breakpoint red circle has a yellow arrow inside it. The execution has been suspended at the first hit of the breakpoint. Choose Debug | Windows | Locals to open the Locals window. Expand the Figure 15-1 Setting a breakpoint on a Foreach Loop container Chapter 15: Troubleshooting and Performance Enhancements 637 Variables node to see the list of system as well as user variables. Note the User::fname and User::lname variable values. 6. Click the Continue button (green triangle) or press 5 to continue execution. This time you will see that the Mailing Opportunities task has been executed successfully and the package execution is stopped again at the Iterating October Opportunities as it hits the breakpoint a second time. Note the User::fname and User::lname variable values, which have been updated with the values of second row in the data set. 7. Go to the Progress tab in the SSIS Designer, and locate the Task Mailing Opportunities section. Note the two rows showing initialization and completion of the task. Continue the package execution by clicking Continue or pressing  5. 8. The package executes one more iteration of the Iterating October Opportunities Foreach Loop Container and stops as it hits the breakpoint a third time. Note the User::fname and User::lname variable values, which have been updated with the values of the third row in the data set. Go to the Progress tab and check out the Task Mailing Opportunities section. Note that this time it shows ⇒ Start (2) at the beginning and ⇐Stop (2) at the end of the section, indicating that this task has completed two execution cycles. 9. Go to the Control Flow tab, right-click Iterating October Opportunities, and select Edit Breakpoints from the context menu. For the selected Break Condition, change the Hit Counter Type from Always to Hit Count Equals and specify 2 in the Hit Count column. Click OK to close the dialog box and press  5 to continue the execution of the package. 10. This time you will notice that the Mailing Opportunities task has executed a number of cycles—exactly four—and the execution stops again. You can verify this by going to the Progress tab and checking out the Task Mailing Opportunities section, which shows that the task has been executed six times. 11. Choose Debug | Windows | Watch | Watch 1 to add a watch window. In the Watch 1 window, click in the Name column and type User::email and then press the  key. You will see that this variable is added in the Watch 1 window and displays its current value. Expand the User::email variable and type in an alternative e-mail address in the Value field; then press the  key on the keyboard. You will see the value of User::email is changed to the one you’ve specified. Figure 15-2 shows the various values in the Progress tab, Watch 1 window, and Locals window. 12. Press 5 to continue execution. This time, the package completes execution successfully and stops. Press - 5 to switch to design mode. Check your inbox and the alternate e-mail address inbox. You will receive six e-mails in your inbox and one e-mail at your alternative e-mail address. 13. Remove the breakpoint from Iterating October Opportunities, save all the files, and close the project. . Troubleshooting Integration Services Packages c Performance Enhancements c Performance Monitoring Tools c Using Parallel Processing c Summary 632 Hands-On Microsoft SQL Server 2008 Integration Services Y ou’ve. third-party components will be migrated. You will c have to manually recompile these components to enable them to work with SQL Server 2008 Integration Services. Summary Integration Services. component to Microsoft Visual Studio Tools for Applications (VSTA). Figure 14-19 Package Management Option with SSIS Package Upgrade Wizard 630 Hands-On Microsoft SQL Server 2008 Integration Services When