248 Hands-On Microsoft SQL Server 2008 Integration Services 7. Move on to the Reporting page. This page has Console events and Console logging boxes, which can be configured with the numerous given options. You can use the Reporting page to set the options for the package by selecting the events and the information about the running package to report to the console. Console events specify the events and types of messages to report, whereas Console logging specifies the information that you want written to the log when the selected event occurs. 8. In the Logging page, you can tell the Package Execution utility which log providers to use at run time by specifying the log providers and the connection string for connecting to a log. The options for log providers are: SSIS Log Provider For Windows Event Log c SSIS Log Provider For XML Files c SSIS Log Provider For Text Files c SSIS Log Provider For SQL Server c SSIS Log Provider For SQL Server Profiler c Click in the field under the Log Provider column to choose from one of the available log providers. Then you can specify the connection string in the Connection String field. 9. The Set Values page is probably the one you will be using most often when you want to update property values of packages, executables, connections, variables, and log providers at run time. This useful feature makes running and testing packages easier. You can modify the property values by typing in the modified value and the path to the property. The path can be specified using a backslash (\) before the container, the period (.) before the property, and including a collection member within the brackets. 10. In the Verification page, you can set the attributes that must be met before allowing a package to run. For example, you can set the attributes to run only signed packages. Other options are to verify the package for the build number, package ID number, or version ID number. 11. Finally, the Command Line page allows you to review and optionally edit the command (see Figure 6-4) that has been generated by the options you selected in the various pages. Select the Edit The Command Line Manually radio button to make changes in the command line. If you accidentally make mistakes here and want to undo your changes, you can do so by selecting the Restore The Original Options radio button. Please note that when you select options in the other pages of the Package Execution utility, the command line gets updated; however, this is not the case if you make changes to the command line—i.e., these changes don’t get reflected in the options of relevant pages. Also, these changes are not validated for the same reason. 12. Now click Execute to run the package, and you will see the progress of the running package in Package Execution Progress window. Chapter 6: Administering Integration Services 249 Review In this exercise, you learned about the Package Execution utility that you can run from within the SQL Server Management Studio or from the command prompt. You also learned various options and how to set values for the properties at run time. You saw that this utility generates a command line that can be copied to a file and used with the CommandFile option in DTExec, or pasted directly to the Command Prompt window when running a package using DTExec. The DTExecUI user interface provides the facility to select several options that you find difficult to write syntactically correct and create command lines that can be passed on to the DTExec command prompt tool. The DTExecUI and DTExec utilities have a couple of differences worth noting. First, DTExecUI is a GUI-based tool that helps you create command lines easily, but lacks the ability to automate the execution of packages. On the other hand, DTExec is a handy and powerful command prompt tool that is a bit difficult to program, especially Figure 6-4 Command generated by the Package Execution utility 250 Hands-On Microsoft SQL Server 2008 Integration Services if you are not deploying packages very often, but once the command line is created, it can automate running of packages with the help of the SQL Server Agent service. So you will use DTExecUI when you want to run a package one off or manually and also when you want to write a complex command line for running a package automatically using DTExec and the SQL Server Agent. The second difference that can cause much grief on a 64-bit server is that the DTExecUI is a 32-bit only utility, whereas DTExec comes in both 64-bit and 32-bit flavors. You might get conflicting results if you’re running your packages using different versions of the utilities. DTExec Utility SSIS provides the command prompt utility DTExec to configure and execute SSIS packages; it replaces the DTSRun utility provided in Data Transformation Services of SQL Server 2000. DTExec is a powerful utility that lets you access packages saved in the SQL Server, on the file system or in the SSIS Package Store. It allows you to configure connections, checkpoints, variables, logging, and reporting for the package you want to run. With DTExec, you can run packages on the local server unless you use the SQL Server Agent or custom-build application to run DTExec commands from a remote computer. The SQL Server Agent allows you to automate the execution of your SSIS packages using the DTExec utility. You can select the Operating System (CmdExec) Agent subsystem, while adding a step to the job and use the DTExec command directly in the command area. As mentioned earlier, you can use the DTExecUI utility to create a command line with all the options you require and use it directly with DTExec. By default, the DTExec utility is located in the <drive>:\Program Files\Microsoft SQL Server\100\DTS\Binn folder. DTExec comes in both 32-bit and 64-bit versions, so on a 64-bit server if you install 32-bit utilities, a 32-bit version of DTExec may also exist in the <drive>:\Program Files(x86)\Microsoft SQL Server\100\DTS\Binn folder. In such a case, the environment variable PATH contains the 32-bit path before the 64-bit path, making 32-bit DTExec run by default from a command prompt. However, the SQL Server Agent doesn’t use this path statement; rather, it uses a registry setting to identify the DTExec location and will run the 64-bit version of DTExec. Make sure you test your packages against the correct version of DTExec before creating SQL Server Agent jobs with this utility. You can, though, avoid this situation by modifying the order of paths in the PATH environment variable. The basic syntax for the command is shown next; the parameters are specified as option-value pairs. The options are specified beginning either with a minus sign (–) or a slash (/) and are not case-sensitive, barring passwords. DTExec /option [value] [/option [value]] Chapter 6: Administering Integration Services 251 When the DTExec utility is executed, it passes through different phases, depending upon the parameters specified in the command line, and in the end, it can return an exit code as well. Let’s study these phases and the parameters that decide the behavior of DTExec. Command Sourcing Phase This is the initialization phase in which the DTExec utility reads the parameters—i.e., the list of options and arguments specified. If you want to see only the help on the commands using a /? or /H option, for example, DTExec will run for this phase only and all the subsequent phases will be skipped. If you specify an argument along with help options, DTExec will display help for that argument. DTExec /H DTS /Rem is another option that is parsed in this phase. Using /Rem, you can include remarks on the command prompt or more appropriately in command files. These remarks are discarded during the command sourcing phase. In the command sourcing phase, if the /CommandFile (/com) option is used, DTExec opens up the file specified with the /CommandFile option and reads all the options from the file until EOF is reached. You can either use the complete option name or use the short form of the option shown in parentheses with each option. The file specified with /CommandFile option is a text file that contains additional DTExec command options. Here’s an example: Open a blank text file in Notepad and type /DTS “\msdb\ mailing opportunities” at the beginning of the file. Save this as C:\dtscmds.txt. Then go to a command prompt and type this command: DTExec /com "C:\dtscmds.txt" You will see that the Mailing Opportunities package has executed successfully. DTExec doesn’t put any restriction on how you write command options—i.e., on the same line or on different lines in the command file. Package Load Phase After passing through the first phase, DTExec goes on to load the package from the storage specified by the /SQL, /FILE, or /DTS option. To load a package stored in the MSDB database of the SQL Server, type the following command using the /SQL (/SQ) option: DTExec /SQL "Mailing Opportunities" 252 Hands-On Microsoft SQL Server 2008 Integration Services The /File (/F) option loads a package saved in the file system. The file path can be specified in either Universal Naming Convention (UNC) format or as a local path. DTExec /F "C:\SSIS\Packages\Mailing Opportunities.dtsx" Don’t forget to include the file extension when using the /File option. Finally, to load a package saved in the SSIS Package Store, you will use the /DTS (/DT) option. DTExec /DTS "\MSDB\Mailing Opportunities" You can use only one of these three options, as they are mutually exclusive. Using these options together will cause the package to fail. You can also use the following supportive options: /User c can be used with the /SQL option when the package is protected by SQL Server Authentication. If you omit the /User option, Windows Authentication is used. It is recommended that you use Windows Authentication whenever possible. /Password c (/P) option is used with the /User option. If you omit the /Password option, a blank password will be used. /Decrypt c (/De) is used to load the package that has been encrypted with a password encryption; this option is used to set the decryption password. /Server c (/Ser) is used to specify the server from which to retrieve the package when the /SQL or /DTS option is specified. If you omit the /Server option when using /SQL or /DTS, the local server default instance will be used to retrieve the package. The following command will run the package stored at the RemoteServer: DTExec /SQL "Mailing Opportunities" /Ser RemoteServer Configuration Phase After the package is loaded, DTExec goes on to process rest of the options that set flags, variables, or properties; verify the package versioning; and specify the logging and reporting. These options are as follows: e c /CheckPointing (/CheckP) option is used when you want to include checkpoints during package execution. e valid values are on or off. e default is on when the option is specified without a value. e checkpoints enable a failed Chapter 6: Administering Integration Services 253 package to rerun from the point of failure. For example, if you are loading multiple tables in a data warehouse and your package fails halfway through, then using checkpoints will enable this failed package to rerun from the failed task position and save the processing for already imported tables. To achieve this, the DTExec utility records the current position in a file called CheckFile. e c /CheckFile (/CheckF) option is used by DTExec to specify the checkpoint file so that a failed package can be restarted from the point of failure. e filename specified with the /CheckFile option is used to update the value of the CheckPointFileName property of the package. e c /ConfigFile (/Conf) option is used at run time when you want to set different configuration values than what were specified at design time. e run- time values can be stored in an XML file and loaded before executing the package using the /ConfigFile option. e c /Connection (/Conn) option can be used to change the connection managers at run time. You have to specify the connection manager name or ID used in the package along with the connection string to which you want to change. e c /ConsoleLog (/Cons) option can be used to display the specified log entries to the console (see Figure 6-5). You can specify the columns you want to see by using the displayoptions parameter, for which the values include N for Name, C for Computer, O for Operator, S for Source Name, G for Source GUID, X for Execution GUID, M for Message, and T for time. If you do not specify Figure 6-5 Use of the /ConsoleLog option displays the execution log on the screen 254 Hands-On Microsoft SQL Server 2008 Integration Services the option parameters, no column is displayed, and if you specify the option parameters, all the values for the selected columns will be displayed that you may want to limit. For limiting the display of log entries on the console, you use extended options that are called list_options. e list_options allow you to log only the sources that are specified with parameter ( I) for inclusion or only the sources that are not explicitly excluded with parameter ( E) using a list. e c /Logger (/L)option lets you choose one or more log providers to be used at run time by specifying an ID and a connection string to establish a connection to the destination store for log entries. e options of the log providers are as follows: SSIS Log Provider for Text Files c SSIS Log Provider for SQL Profiler c SSIS Log Provider for SQL Server c SSIS Log Provider for Windows Event Log c SSIS Log Provider for XML Files c e c /Reporting (/Rep) option works as follows: By default, DTExec will report errors, warnings, and progress of the events when the package is executed. However, you can specify the types of messages to report using the /Reporting option. You can choose from the options listed in the following table: Option Symbol and Name Description N – No Reporting A mutually exclusive option with all other options and should be specified alone. E – Errors Errors are reported. W – Warnings Warnings are reported. I – Information Informational messages are reported. C – Custom Events Custom events are reported. D – Data Flow Task Events Events related to Data Flow task are reported. P – Progress Progress of the execution of package is reported. V – Verbose Reporting A mutually exclusive option with all other options and should be specified alone. e c /Restart (/Res) option specifies a new value for the CheckpointUsage property on the package. e possible parameters are shown next: Chapter 6: Administering Integration Services 255 Deny Sets CheckpointUsage property to DTSCU_NEVER. Force (Default) Sets CheckpointUsage property to DTSCU_ALWAYS. ifPossible Sets CheckpointUsage property to DTSCU_IFEXISTS. e c /Set option is used to modify the configuration values of variables, properties, containers, log providers, Foreach enumerators, or connections at run time. Both the property path and the value are to be specified according to specific syntax rules. e syntax uses a backslash ( \) as a container separator, a period (.) to delimit properties, square brackets ( []) to specify collection members, and a semicolon (;) to separate the property path and its value. e c /MaxConcurrent (/M) option limits the number of executables running in parallel on your machine. You can specify a positive non-zero integer value or –1. A value of –1 means the maximum concurrently running executables (total number of processors on the computer plus two) are allowed. e c /Sum (/Su) option is an optional switch showing a counter that contains the number of rows that will be passed on to the next component. e c /VerifyBuild (/VerifyB) option works like this: During the development stage when you will be making lots of changes to your package, you may want to make sure that the package is executed with a stable version of the build while you can carry on the development work without jeopardizing the normal running of the package. You can specify the build numbers you want to run and verify your package against those numbers. In case of a mismatch, the package will fail to run. e /VerifyBuild option has three arguments, major, minor, and build, which can be specified in three forms: major or major; minor or major; minor; build. e c /VerifyPackageID (/VerifyP) option is similar to the /VerifyBuild option; you can specify the package ID with this option to let it verify against the package GUID. e c /VerifySigned (/VerifyS) option is a security option to check that the package is signed. If the package is not signed or the signature is not valid, the execution of the package will fail. e c /VerifyVersionID (VerifyV) option verifies the version ID of a package to be executed against the version ID you specify with this option. 256 Hands-On Microsoft SQL Server 2008 Integration Services Validation and Execution Phase In this phase, the package is validated and then finally run. However, you can also choose only to validate the package without running it. e c /Validate (/Va) option lets you validate only without running the package. e c /WarnAsError (/W) option, specified along with the /Validate option, can go one step further by letting the package treat the warnings as errors and will fail if a warning occurs during validation phase. e c /Dump errorcode option can create the debug dump files when one or more specified events—error, warning, or information—occur while the package is running. You can specify the events by specifying the error codes, separated by a semicolon (;), that you want to use to invoke the system to create the debug dump files. Integration Services creates a binary debug dump file with an extension .mdmp and a text-formatted debug dump file with an extension .tmp; by default, it stores these files in the <drive>:\Program Files\Microsoft SQL Server\100\ Shared\ErrorDumps folder. e c /DumpOnError option works like this: If your package has been failing and you want to debug the failures, you can create debug dump files. You can use this option if you want to create the debug dump files, .mdmp and .tmp, only when an error occurs while the package is running. Finally, the DTExec utility can return an exit code depending upon the state of the package loading and execution. The exit codes are values from 0 through 6 that can be interpreted as follows: Exit Code Description 0 The package was executed successfully. 1 The package failed to execute. 3 The user canceled execution of the package. 4 The package could not be found at the specified location. 5 The package could not be loaded. 6 The command line could not be interpreted due to syntax or semantic errors. SQL Server Agent The SQL Server Agent can automate the running of SSIS packages. SQL Server Agent jobs can have one or more steps, each running a different package and a bit of work that the agent can do for you. The jobs can also be scheduled to run at specific times using extensive SQL Server Agent scheduling capabilities. Chapter 6: Administering Integration Services 257 One thing worth mentioning here is that Integration Services is a server component and not a redistributable component, as was its predecessor DTS 2000. This means that you can’t install SSIS (as redistributable software) on a client machine and execute SSIS packages as you could have done in the past with DTS 2000. Also, SSIS packages will always run locally even when they load a package from a remote server; hence, you can’t run a package (outside the design environment BIDS) on a computer where Integration Services is not installed. However, there will be instances where users or applications need to run packages while Integration Services is not installed on the local computer. In those instances the SQL Server Agent (or any other scheduling agent) comes in handy, as it allows you to schedule and automate the execution of SSIS packages not only on local servers but also on the remote servers. You can start the jobs on remote servers using system stored procedure sp_start_job or SMO. The only drawback of this method is that after you’ve created jobs on the SQL Server Agent, you can’t configure the jobs at run time; that is, you can’t provide parameters or other options at run time that you can do with DTExec utility. Hands-On: Automating Running an SSIS Package with SQL Server Agent In this exercise, you will learn how to use the SQL Server Agent service to create and automate jobs for running SSIS packages. Method We will use the SQL Server Agent in SQL Server Management Studio to Create a new job and add a package (step) to the job c Create a schedule to execute the package c View the job history after running c For this exercise, we will again use the Mailing Opportunities package as a test package. Exercise (Create a New Job and add a Package to the Job) You will be creating an SQL Server Agent job and then will add an SSIS package to a step in the job here. 1. Start SQL Server Management Studio and connect to the database engine. Once connected, verify that the SQL Server Agent service is started. If not, start the SQL Server Agent service. . 248 Hands-On Microsoft SQL Server 2008 Integration Services 7. Move on to the Reporting page. This page has Console events. especially Figure 6-4 Command generated by the Package Execution utility 250 Hands-On Microsoft SQL Server 2008 Integration Services if you are not deploying packages very often, but once the command. the /SQL (/SQ) option: DTExec /SQL "Mailing Opportunities" 252 Hands-On Microsoft SQL Server 2008 Integration Services The /File (/F) option loads a package saved in the file system.