ptg 1284 CHAPTER 35 Understanding Query Optimization . FAST n—This hint instructs SQL Server to optimize the query to return the first n rows as quickly as possible, even if the overall throughput is reduced. In other words, it improves response time at the expense of total query execution time. This option generally influences the Query Optimizer to retrieve data using a nonclus- tered index that matches the ORDER BY clause of a query instead of using a different access method that would require a sort operation first to return rows in the speci- fied order. After n number of rows have been returned, the query continues execu- tion normally to produce its full result set. . ROBUST PLAN—This option forces the Query Optimizer to attempt a plan that works for the maximum potential row size, even if it means degrading performance. If you have very wide VARCHAR columns, some types of query plans might create intermediate tables, and if any of the internal operations need to store and process rows in these intermediate tables, some rows might exceed SQL Server’s row size limit. If this happens, SQL Server generates an error during query execution. When the ROBUST PLAN hint is specified, the Query Optimizer does not consider any plans that might encounter this problem. . MAXDOP number—This hint overrides the server-level configuration setting for max degree of parallelism for the current query in which the hint is specified. . KEEP PLAN—When this hint is specified, it forces the Query Optimizer to relax the estimated recompile threshold for a query. The estimated recompile threshold is the point at which a query is automatically recompiled when the estimated number of indexed column changes have been made to a table by updates, inserts, or deletes. Specifying KEEP PLAN ensures that the query is not recompiled as frequently when there are multiple updates to a table. This option is useful primarily for queries whose execution plan stays in memory, such as for stored procedures. You might want to specify this option for a stored procedure that does a lot of work with temporary tables, which can lead to frequent recompilations of the execution plan for the stored procedure. . KEEPFIXED PLAN—This query hint tells the Query Optimizer not to recompile the query plan when there are changes in statistics or modifications to indexed columns used by the query via updates, deletes, or inserts. When this option is specified, the query is recompiled only if the schema of the underlying tables is changed or sp_recompile is executed against those tables. . EXPAND VIEWS—This hint tells the Query Optimizer not to consider any indexed view as a substitute for any part of the query and to force the view to be expanded into its underlying query. This hint essentially prevents direct use of indexed views in the query plan. . MAXRECURSION number—This hint specifies the maximum number of recursions allowed for the common table expression query, where number is an integer between 0 and 32767. When 0 is specified, no limit is applied. If this option is not specified, the default limit for the server is 100. For more information on common table Download from www.wowebook.com ptg 1285 Managing the Optimizer 35 expressions and recursive queries, see Chapter 43, “Transact-SQL Programming Guidelines, Tips, and Tricks.” . RECOMPILE—This hint forces SQL Server not to keep the execution plan generated for the query in the plan cache after it executes. This forces a new plan to be gener- ated the next time the same or a similar query plan is executed. RECOMPILE is useful for queries with variable values that vary widely each time they are compiled and executed. This hint can be used for individual statements within a stored procedure in place of the global WITH RECOMPILE option when you want only a subset of queries inside the stored procedure to be recompiled rather than all of them. . OPTIMIZE FOR ( @variable_name = literal_constant [ , n ] )— This hint instructs SQL Server to use a specified value to optimize the SARGs for a local variable that is otherwise unknown when the query is compiled and optimized. The value is used only during query optimization and not during query execution. OPTIMIZE FOR can help improve optimization by allowing the Query Optimizer to use the statistics histogram rather than index densities to estimate the rows that match the local variable, or can be used when you create plan guides. . OPTIMIZE FOR UNKNOWN—This hint instructs the query optimizer to use statistical data instead of the initial values for local variables when the query is compiled and optimized, including parameters created with forced parameterization. . TABLE HINT (object_name [ , table_hint [ [, ] n ] ] )—New in SQL Server 2008, you can now specify table hints in the Query Hint OPTION clause. It is recommended that the TABLE HINT clause be used only in the context of a plan guide. For all other ad hoc queries, it is recommend that normal table hints be used. . USE PLAN N’xml_plan’—This hint instructs SQL Server to use an existing query plan for a query as specified by the designated xml_plan. The USE PLAN query hint can be used for queries whose plans result in slow execution times but for which you know better plans exist. NOTE Optimizer hints are not always executed. For example, the Query Optimizer is likely to ignore a HASH UNION hint for a query using the UNION ALL statement. Because UNION ALL means to return all rows whether or not there are duplicates, you don’t need to hash these values to determine uniqueness and remove duplicates, so the normal con- catenation is likely to still take place. Forced Parameterization In SQL Server 2008, if a SQL statement is executed without parameters, the Query Optimizer parameterizes the statement internally to increase the possibility of matching it against an existing execution plan. This process is called simple parameterization, some- times referred to as auto-parameterization. Simple parameterization is somewhat limited in that it can parameterize only a relatively small number of queries that match a small Download from www.wowebook.com ptg 1286 CHAPTER 35 Understanding Query Optimization number of very simple and strictly defined query templates. For example, simple parame- terization is not possible for queries that contain any of the following query elements: . References to more than one table . IN clauses or OR expressions . UNION . Any query hints . DISTINCT . TOP . Subqueries . GROUP BY . Not equal (<> or !=) comparisons . References to functions SQL Server 2008 enables you to override the default simple parameterization behavior of SQL Server and provide parameterization for more complex queries by specifying that all SELECT, INSERT, UPDATE, and DELETE statements in a database be implicitly parameterized when they are compiled by the Query Optimizer. You enable this by setting the PARAMETERIZATION option to FORCED in the ALTER DATABASE statement: ALTER DATABASE dbname SET PARAMETERIZATION {FORCED | SIMPLE} Setting the PARAMETERIZATION option is an online operation that can be issued at any time and requires no database-level exclusive locks. Forced parameterization may improve the performance of queries for certain databases by reducing the frequency of query compilations and recompilations. Essentially, forced para- meterization provides the query plan reuse benefits of parameterized queries without requiring you to rewrite a single line of application code. The databases that may benefit from forced parameterization generally support OLTP-type applications that experience high volumes of concurrent queries, such as point-of-sale applications. When the PARAMETERIZATION FORCED option is enabled, any literal value that appears in a SELECT, INSERT, UPDATE, or DELETE statement, submitted in any form, is converted to a parameter during query compilation. The exceptions are literals that appear in the follow- ing query constructs: . INSERT EXECUTE statements . Statements inside the bodies of stored procedures, triggers, or user-defined functions. SQL Server already reuses query plans for these routines. . Prepared statements that have already been parameterized by the client-side application. . Statements inside a T-SQL cursor. Download from www.wowebook.com ptg 1287 Managing the Optimizer 35 . Any statement run in a context where ANSI_PADDING or ANSI_NULLS is set to OFF. . Statements that contain more than 2,097 literals eligible for parameterization. . Statements that reference variables, such as WHERE st.state = @state. . Statements that contain the RECOMPILE or OPTIMIZE FOR query hints. . Statements that contain a COMPUTE clause. . Statements that contain a WHERE CURRENT OF clause. If an execution plan for a query is cached, you can determine whether the query is para- meterized by referencing the sql column of the sys.syscacheobjects DMV. If a query is parameterized, the names and data types of parameters are listed in this column before the text of the submitted SQL (for example, @1 tinyint). Guidelines for Using Forced Parameterization Consider the following guidelines when determining whether to enable forced parameteri- zation for a database: . Forced parameterization, in effect, changes the literal constants in a query to para- meters when the query is compiled, and thus, the Query Optimizer might choose suboptimal plans for queries. For example, the Query Optimizer may be less likely to match the query to an indexed view or an index on a computed column. It may also choose suboptimal plans for queries posed on partitioned tables and distributed partitioned views. Forced parameterization should not be used for environments that rely heavily on indexed views and indexes on computed columns. . Enabling the PARAMETERIZATION FORCED option causes all query plans for the data- base to be flushed from the plan cache. . Generally, the PARAMETERIZATION FORCED option should be used only by experienced database administrators after determining that doing this does not adversely affect performance. If forced parameterization is enabled and you want to override this behavior and have simple parameterization used for a single query and any others that are syntactically equivalent but differ only in their parameter values, you can use plan guides and specify PARAMETERIZATION SIMPLE when creating the plan guides. Conversely, rather than enabling PARAMETERIZATION FORCED for an entire database, you can use plan guides and specify the PARAMETERIZATION FORCED query option only for a specific set of syntactically equivalent queries that you have determined would benefit from forced parameterization. Using the USE PLAN Query Hint The USE PLAN query hint in SQL Server 2008 can be used to encourage the Query Optimizer to use the specified XML query plan for processing the query. This option provides more control over influencing the execution of a query than is possible with the other available query hints, such as FORCE ORDER, LOOP JOIN, and KEEP PLAN. None of these options individually are powerful enough to influence the Query Optimizer to Download from www.wowebook.com ptg 1288 CHAPTER 35 Understanding Query Optimization consistently choose a particular query plan, especially when the referenced table row counts, statistics, indexes, and other attributes of the environment change. The USE PLAN query hint is specified in the OPTION clause, and you provide it with a query plan in XML format. Listing 35.7 provides an example of the USE PLAN hint being speci- fied for a merge join for a simple query that consists of a join between two tables. (Note: For the sake of space, the full XML plan has been truncated.) LISTING 35.7 Specifying the USE PLAN Query Option select st.stor_name, s.ord_date from sales s join stores st on s.stor_id = st.stor_id WHERE st.state = ‘NY’ OPTION (USE PLAN N’ <?xml version=”1.0” encoding=”utf-16”?> <ShowPlanXML xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xmlns:xsd=”http://www.w3.org/2001/XMLSchema” Version=”1.1” Build=”10.50.1352.12” xmlns=”http://schemas.microsoft.com/ sqlserver/2004/07/showplan”> <BatchSequence> <Batch> <Statements> <StmtSimple StatementCompId=”1” StatementEstRows=”10710.8” StatementId=”1” StatementOptmLevel=”FULL” StatementSubTreeCost=”1.71032” StatementText=”select st.stor_name, s.ord_date&#xD;&#xA; from sales s join stores st on s.stor_id = st.stor_id&#xD;&#xA; WHERE st.state = ‘’NY’’&#xD;&#xA;” StatementType=”SELECT” QueryHash=”0x35DE42B697A8BAAF “ QueryPlanHash=”0x9F4AE50605763B05”> <StatementSetOptions ANSI_NULLS=”true” ANSI_PADDING=”true” ANSI_WARNINGS=”true” ARITHABORT=”true” CONCAT_NULL_YIELDS_NULL=”true” NUMERIC_ROUNDABORT=”false” QUOTED_IDENTIFIER=”true” /> <QueryPlan DegreeOfParallelism=”1” CachedPlanSize=”16” CompileTime=”6” CompileCPU=”6” CompileMemory=”232”> <MissingIndexes> <MissingIndexGroup Impact=”13.6636”> <MissingIndex Database=”[bigpubs2008]” Schema=”[dbo]” Table=”[stores]”> <ColumnGroup Usage=”EQUALITY”> <Column Name=”[state]” ColumnId=”5” /> </ColumnGroup> <ColumnGroup Usage=”INCLUDE”> <Column Name=”[stor_id]” ColumnId=”1” /> <Column Name=”[stor_name]” ColumnId=”2” /> </ColumnGroup> </MissingIndex> </MissingIndexGroup> Download from www.wowebook.com ptg 1289 Managing the Optimizer 35 </MissingIndexes> <RelOp AvgRowSize=”39” EstimateCPU=”0.363144” EstimateIO=”0” EstimateRebinds=”0” EstimateRewinds=”0” EstimateRows=”10710.8” LogicalOp=”Inner Join” NodeId=”0” Parallel=”false” PhysicalOp=”Merge Join” EstimatedTotalSubtreeCost=”1.71032”> <Object Database=”[bigpubs2008]” Schema=”[dbo]” Table=”[sales]” Index=”[UPKCL_sales]” Alias=”[s]” IndexKind=”Clustered” /> </IndexScan> </RelOp> </Merge> </RelOp> </QueryPlan> </StmtSimple> </Statements> </Batch> </BatchSequence> </ShowPlanXML> To obtain an XML-formatted query plan, which you can provide to the USE PLAN query hint, SQL Server 2008 provides the following methods: . Using the SET SHOWPLAN_XML and SET STATISTICS XML session options . Querying the plan column of the sys.dm_exec_query_plan dynamic management view for a cached query plan . Using SQL Server Profiler and capturing either the Showplan XML, Showplan XML Statistics Profile, or Showplan XML For Query Compile event classes NOTE When the XML query plan contains a character string in single quotation marks (‘), the quotation marks must be escaped by a second quotation mark before using the plan with the USE PLAN query hint. For example, a plan that contains WHERE A.varchar = ‘This is a string’ must be escaped by modifying the code to WHERE A.varchar = ‘’This is a string’’; otherwise, it will generate a syntax error when submitted for execution. You may choose to use the USE PLAN hint for queries where the execution plan chosen leads to slow execution times but for which you know a better plan exists. This scenario may commonly occur for queries that might have executed well in an earlier version of SQL Server but that perform poorly under an upgraded version. Another scenario could be a complex query that involves multiple tables where the compiled or recompiled query plan generated is occasionally not optimal, possibly as a result of out-of-date or missing statistics in any of the underlying tables or because of complex constructs in the query Download from www.wowebook.com ptg 1290 CHAPTER 35 Understanding Query Optimization that cause the Query Optimizer to inaccurately estimate the size of the intermediate query results. The USE PLAN query hint can be specified only for SELECT and SELECT INTO statements. Also, you can force only query plans that can be produced by the Query Optimizer’s normal optimization strategy. Because the USE PLAN option requires that the XML execution plan be hard-coded in the SQL statement itself, it is not a viable solution for deployed or third-party applications where it may not be possible or feasible to modify the queries directly. It’s really useful only as a tool for troubleshooting poorly running queries. To force query plans to apply query hints to queries when you cannot or do not want to directly change the application or SQL code, you might consider using plan guides. Using Plan Guides At times, you might find it necessary to use query hints to improve the performance of queries for a particular query or a small set of queries. Although this may be easy to do when you have access to the application code, often the particular queries to be modified are embedded within a third-party application, and alteration of the queries themselves is virtually impossible. Also, if you start hard-coding query hints in your application code, changing them as necessary when data volumes change or when upgrading to a new version of SQL Server can be a difficult undertaking. The plan guides feature in SQL Server 2008 provides an ideal solution for such scenarios by offering another mechanism for injecting query hints into the original query without having to modify the query itself. The plan guides mechanism uses an internal lookup system table, based on information in the sys.plan_guides catalog view, to map the origi- nal query to a substitute query or query template. As described earlier in this chapter, when a SQL statement is submitted, it is first compared against the cached plans to check for a match. If a match exists, the cached query plan is used to execute the query. If no cached plan exists for the query, the Query Optimizer next looks for a match against the set of existing plan guides, if any, stored in the current database for a match. If an active plan guide that matches the SQL statement is found, the original matching statement is substituted with the one from the plan guide, the query plan is compiled and cached, and the query is executed using the plan gener- ated from the plan guide. Queries that can benefit from plan guides are generally those that are parameter based and those that are likely performing poorly because they use cached query plans whose para- meter values do not represent a more representative scenario. The plan guides feature essentially consists of two stored procedures to create, drop, enable, and disable plan guides and the sys.plan_guides metadata view that describes the stored plan guides. Plan guides are created and administered by using the two system stored procedures: . sp_create_plan_guide Download from www.wowebook.com ptg 1291 Managing the Optimizer 35 . sp_control_plan_guide The syntax for these procedures is as follows: sp_create_plan_guide [ @name = ] N’plan_guide_name’ , [ @stmt = ] N’statement_text’ , [ @type = ] N’{ OBJECT | SQL | TEMPLATE }’ , [ @module_or_batch = ] { N’[ schema_name. ] object_name’ | N’batch_text’ | NULL } , [ @params = ] { N’@parameter_name data_type [ , n ]’ | NULL } , [ @hints = ] { N’OPTION ( query_hint [ , n ] )’ | N’XML_execution plan’ | NULL } sp_control_plan_guide [ @operation = ] N’<control_option>’ [ , [ @name = ] N’plan_guide_name’ ] <control_option>::= { DROP | DROP ALL | DISABLE | DISABLE ALL | ENABLE | ENABLE ALL } Note that in SQL Server 2008, the sp_create_plan_guide stored procedure enables you to pass an XML execution plan directly in the @hints parameter instead of embedding the output in a USE PLAN hint. This capability simplifies the process of applying a fixed query plan as a plan guide hint. In addition, a new stored procedure, sp_create_plan_guide_from_handle, allows you to create one or more plan guides from a query plan in the plan cache. The syntax for sp_create_plan_guide_from_handle is as follows: sp_create_plan_guide_from_handle [ @name = ] N’plan_guide_name’ , [ @plan_handle = ] plan_handle , [ [ @statement_start_offset = ] { statement_start_offset | NULL } ] Instead of specifying an actual XML execution plan, you pass the handle for a query plan currently in the plan cache to the @plan_handle parameter. As shown previously in this chapter, a plan_handle can be obtained from the sys.dm_exec_query_stats DMV. If the Download from www.wowebook.com ptg 1292 CHAPTER 35 Understanding Query Optimization cached plan contains multiple queries in a SQL batch, you can specify the starting posi- tion of the statement within the batch via the @statement_start_offset parameter. The statement offset corresponds to the statement_start_offset column in the sys.dm_exec_query_stats dynamic management view. If no statement offset is specified, a plan guide is created for each statement in the batch using the query plan for the speci- fied plan handle. The resulting plan guides are equivalent to plan guides that use the USE PLAN query hint to force the use of a specific plan. Creating Plan Guides Plan guides can be created to match queries executed in the following contexts: . An OBJECT plan guide matches queries that execute in the context of T-SQL stored procedures, scalar functions, or multistatement table-valued functions. . A SQL plan guide matches queries that execute in the context of ad hoc T-SQL state- ments and batches that are not part of a stored procedure or other compiled data- base object. . A TEMPLATE plan guide matches ad hoc queries that parameterize to a specified form. These plan guides are used to override the current SET PARAMETERIZATION database option. In the sp_create_plan_guide statement, you specify the query that you want optimized and provide the OPTION clause with the query hints necessary to optimize the query in the manner desired, or an XML execution plan for the query plan you want the query to use. When the query executes, SQL Server matches the query to the plan guide and applies the forced query plan to the query at runtime. The plan guide can specify any of the following query hints individually or combined with others, when applicable: . {HASH | ORDER} GROUP . {CONCAT | HASH | MERGE} UNION . {LOOP | MERGE | HASH} JOIN . FAST n . FORCE ORDER . MAXDOP number_of_processors . OPTIMIZE FOR ( @variable_name = literal_constant ) [ , n ] . OPTIMIZE FOR UNKNOWN . RECOMPILE . ROBUST PLAN . KEEP PLAN . KEEPFIXED PLAN Download from www.wowebook.com ptg 1293 Managing the Optimizer 35 . EXPAND VIEWS . MAXRECURSION number . TABLE HINT (object_name [ , table_hint [ [, ] n ] ] ) . USE PLAN <xmlplan> . PARAMETERIZATION { SIMPLE | FORCED } The PARAMETERIZATION { SIMPLE | FORCED } query hint can be used only within a plan guide, and it specifies whether a query is parameterized as part of compiling a query plan. This option overrides the current setting of the PARAMETERIZATION option set at the data- base level. Listing 35.8 provides a sample plan guide created for a simple SQL statement. LISTING 35.8 Creating a Plan Guide for a Simple SQL Statement sp_create_plan_guide @name = N’PlanGuide1’, @stmt = N’SELECT COUNT(*) AS Total FROM dbo.sales s, dbo.titles t WHERE s.title_id = t.title_id and t.pubdate BETWEEN ‘’1/1/2004’’ AND ‘’1/1/2006’’ ‘, @type = N’SQL’, @module_or_batch = NULL, @params = NULL, @hints = N’OPTION (HASH JOIN)’ For plan guides of type ’SQL’ or ’TEMPLATE’ to match a query successfully, the values for batch_text and @parameter_name data_type [, n ] must be provided in exactly the same format as their counterparts submitted by the application. Specifically, they must match character for character, including comments and whitespaces. TIP When you are creating plan guides, be careful to specify the query in the @stmt para- meter and any parameter names and values in the @params parameter exactly as they are received from the application. The best way to ensure this is to capture the batch or statement text from SQL Server Profiler. (See Chapter 6 for more information on using SQL Server Profiler to capture SQL queries.) Also, as with the XML query plans passed to the USE PLAN query hint, single-quoted literal values, such as ’1/1/2000’, need to be delimited with single quotation marks escaped by additional single quota- tion marks, as shown in Listing 35.6. Managing Plan Guides You use the sp_control_plan_guide stored procedure to enable, disable, or drop a plan guide. The following example drops the plan guide created in Listing 35.8: Download from www.wowebook.com . (<> or !=) comparisons . References to functions SQL Server 2008 enables you to override the default simple parameterization behavior of SQL Server and provide parameterization for more complex. data volumes change or when upgrading to a new version of SQL Server can be a difficult undertaking. The plan guides feature in SQL Server 2008 provides an ideal solution for such scenarios by. is to capture the batch or statement text from SQL Server Profiler. (See Chapter 6 for more information on using SQL Server Profiler to capture SQL queries.) Also, as with the XML query plans