Nielsen c11.tex V4 - 07/23/2009 1:54pm Page 262 Part II Manipulating Data With Select What’s New with Subqueries? S ubqueries are fundamental to SQL and there’s been a steady evolution of their capabilities. Significant recent improvements include the following: ■ SQL Server 2005 saw the introduction of the Apply structure for user-defined functions and subqueries. ■ With SQL Server 2008, Microsoft adds row constructors that can be used in the subquery to provide hard-coded values to the query. ■ Also new with SQL server 2008 is composable SQL — a new way to plug together multiple DML statements. Anytime there’s a new way to connect together different parts of the SQL query, it opens new doors for experimentation and building new queries. It’s good to see Microsoft continue to evolve and progress in critical areas such as subqueries. Simple Subqueries Simple subqueries are executed in the following order: 1. The simple subquery is executed once. 2. The results are passed to the outer query. 3. The outer query is executed once. The most basic simple subquery returns a single (scalar) value, which is then used as an expression in the outer query, as follows: SELECT (SELECT 3) AS SubqueryValue; Result: SubqueryValue 3 The subquery (SELECT 3) returns a single value of 3, which is passed to the outer SELECT statement. The outer SELECT statement is then executed as if it were the following: SELECT 3 AS SubqueryValue; Of course, a subquery with only hard-coded values is of little use. A useful subquery fetches data from a table, for example: USE OBXKites; SELECT ProductName 262 www.getcoolebook.com Nielsen c11.tex V4 - 07/23/2009 1:54pm Page 263 Including Data with Subqueries and CTEs 11 FROM dbo.Product WHERE ProductCategoryID =(Select ProductCategoryID FROM dbo.ProductCategory Where ProductCategoryName = ‘Kite’); To execute this query, SQL Server first evaluates the subquery and returns a value to the outer query (your unique identifier will be different from the one in this query): Select ProductCategoryID FROM dbo.ProductCategory Where ProductCategoryName = ‘Kite’; Result: ProductCategoryID c38D8113-2BED-4E2B-9ABF-A589E0818069 The outer query then executes as if it were the following: SELECT ProductName FROM dbo.Product WHERE ProductCategoryID =‘c38D8113-2BED-4E2B-9ABF-A589E0818069’; Result: ProductName Basic Box Kite 21 inch Dragon Flight Sky Dancer Rocket Kite If you think subqueries seem similar to joins, you’re right. Both are a means of referencing multiple data sources within a single query, and many queries that use joins may be rewritten as queries using sub- queries. Best Practice U se a join to pull data from two data sources that can be filtered or manipulated as a whole after the join. If the data must be manipulated prior to the join, then use a derived table subquery. 263 www.getcoolebook.com Nielsen c11.tex V4 - 07/23/2009 1:54pm Page 264 Part II Manipulating Data With Select Common table expressions The common table expression (CTE) defines what could be considered a temporary view, which can be referenced just like a view in the same query. Because CTEs may be used in the same ways that simple subqueries are used and they compile exactly like a simple subquery, I’ve included them in the simple subquery heading and will show example code CTEs alongside simple subqueries. The CTE uses the WITH clause, which defines the CTE. Inside the WITH clause is the name, column aliases, and SQL code for the CTE subquery. The main query can then reference the CTE as a data source: WITH CTEName (Column aliases) AS (Simple Subquery) SELECT FROM CTEName; The WITH keyword not only begins a CTE, it also adds a hint to a table reference. This is why the statement before a CTE must be terminated with a semicolon — just one more reason to always terminate every statement with a semicolon. The following example is the exact same query as the preceding subquery, only in CTE format. The name of the CTE is CTEQuery. It returns the ProductionCategoryID column and uses the exact same SQL Select statement as the preceding simple subquery: WITH CTEQuery (ProductCategoryID) AS (Select ProductCategoryID from dbo.ProductCategory Where ProductCategoryName = ‘Kite’) (Note that a CTE by itself is an incomplete SQL statement. If you try to run the preceding code, you will get a syntax error.) Once the CTE has been defined in the WITH clause, the main portion of the query can reference the CTE using its name as if the CTE were any other table source, such as a table or a view. Here’s the complete example, including the CTE and the main query: WITH CTEQuery (ProductCategoryID) AS (Select ProductCategoryID from dbo.ProductCategory Where ProductCategoryName = ‘Kite’) SELECT ProductName FROM dbo.Product WHERE ProductCategoryID = (SELECT ProductCategoryID FROM CTEQuery); To include multiple CTEs within the same query, define the CTEs in sequence prior to the main query: WITH CTE1Name (column names) AS (Simple Subquery), 264 www.getcoolebook.com Nielsen c11.tex V4 - 07/23/2009 1:54pm Page 265 Including Data with Subqueries and CTEs 11 CTE2Name (column names) AS (Simple Subquery) SELECT FROM CTE1Name INNER JOIN CTE2Name ON Although CTEs may include complex queries, they come with two key restrictions: ■ Unlike subqueries, CTEs may not be nested. A CTE may not include another CTE. ■ CTEs may not reference the main query. Like simple subqueries, they must be self-contained. However, a CTE may reference any of the CTEs defined before it, or even itself (see below). Best Practice A lthough the CTE syntax may initially appear alien, for very complex queries that reference the same subquery in multiple locations, using a CTE may reduce the amount of code and improve readability. A CTE is really just a different syntax for a simple subquery used as a derived table, with one key exception: CTEs can recursively refer to the same table using a union, and this works great for searching an adjacency pairs pattern hierarchy. For more details on using CTEs for hierarchies, turn to Chapter 17, ‘‘Traversing Hierarchies.’’ Using scalar subqueries If the subquery returns a single value it may then be used anywhere inside the SQL SELECT statement where an expression might be used, including column expressions, JOIN conditions, WHERE conditions, or HAVING conditions. Normal operators (+, =, between, and so on) will work with single values returned from a subquery; data-type conversion using the CAST() or CONVERT() functions may be required, however. The example in the last section used a subquery within a WHERE condition. The following sample query uses a subquery within a column expression to calculate the total sales so each row can calculate the percentage of sales: SELECT ProductCategoryName, SUM(Quantity * UnitPrice) AS Sales, Cast(SUM(Quantity * UnitPrice) / (SELECT SUM(Quantity * UnitPrice) FROM dbo.OrderDetail) *100) AS PercentOfSales FROM dbo.OrderDetail AS OD INNER JOIN dbo.Product AS P ON OD.ProductID = P.ProductID INNER JOIN dbo.ProductCategory AS PC 265 www.getcoolebook.com Nielsen c11.tex V4 - 07/23/2009 1:54pm Page 266 Part II Manipulating Data With Select ON P.ProductCategoryID = PC.ProductCategoryID GROUP BY ProductCategoryName ORDER BY Count(*) DESC; The subquery, SELECT SUM(Quantity * UnitPrice) from OrderDetail, returns a value of 1729.895, which is then passed to the outer query’s PercentageOfSales column. The result lists the product categories, sales amount, and percentage of sales: ProductCategoryName Sales PercentOfSales Kite 1694.452500 87.891300 OBX 64.687500 3.355300 Clothing 117.050000 6.071300 Accessory 10.530000 0.546100 Material 5.265000 0.273000 Video 35.910000 1.862600 The following SELECT statement is extracted from the fsGetPrice() user-defined function in the OBXKites sample database. The OBXKites database has a Price table that allows each product to have a list of prices, each with an effective date. The OBX Kite store can predefine several price changes for a future date, rather than enter all the price changes the night before the new prices go into effect. As an additional benefit, this data model maintains a price history. The fsGetPrice() function returns the correct price for any product, any date, and any customer- discount type. To accomplish this, the function must determine the effective date for the date submitted. For example, if a user needs a price for July 16, 2002, and the current price was made effective on July 1, 2002, then in order to look up the price the query needs to know the most recent price date using max(effectivedate),whereeffectivedate is = @orderdate. Once the subquery determines the effective date, the outer query can look up the price. Some of the function’s variables are replaced with static values for the purpose of this example: SELECT @CurrPrice = Price * (1-@DiscountPercent) FROM dbo.Price INNER JOIN dbo.Product ON Price.ProductID = Product.ProductID WHERE ProductCode = ‘1001’ AND EffectiveDate = (SELECT MAX(EffectiveDate) FROM dbo.Price INNER JOIN dbo.Product ON Price.ProductID = Product.ProductID WHERE ProductCode = ‘1001’ AND EffectiveDate <= ‘2001/6/1’); Calling the function, Select dbo.fGetPrice(’1001’,’5/1/2001’,NULL); 266 www.getcoolebook.com Nielsen c11.tex V4 - 07/23/2009 1:54pm Page 267 Including Data with Subqueries and CTEs 11 the subquery determines that the effective price date is January 5, 2001. The outer query can then find the correct price based on the ProductID and effective date. Once the fGetPrice() function calcu- lates the discount, it can return @CurrPrice to the calling SELECT statement: 14.95 Using subqueries as lists Subqueries begin to shine when used as lists. A single value, commonly a column, in the outer query is compared with the subquery’s list by means of the IN operator. The subquery must return only a single column; multiple columns will fail. The IN operator returns a value of true if the column value is found anywhere in the list supplied by the subquery, in the same way that WHERE IN returns a value of true when used with a hard-coded list: SELECT FirstName, LastName FROM dbo.Contact WHERE HomeRegion IN (’NC’, ‘SC’, ‘GA’, ‘AL’, ‘VA’); A list subquery serves as a dynamic means of generating the WHERE IN condition list: SELECT FirstName, LastName FROM dbo.Contact WHERE Region IN (Subquery that returns a list of states); The following query answers the question ‘‘When OBX Kites sells a kite, what else does it sell with the kite?’’ To demonstrate the use of subqueries, this query uses only subqueries — no joins. All of these subqueries are simple queries, meaning that each can run as a stand-alone query. The subquery will find all orders with kites and pass those OrderIDs to the outer query. Four tables are involved in providing the answer to this question: ProductCategory, Product, OrderDetail, and Order. The nested subqueries are executed from the inside out, so they read in the following order (explained in more detail after the query): 1. The subquery finds the one ProductCategoryID for the kites. 2. The subquery finds the list of products that are kites. 3. The subquery finds the list of orders with kites. 4. The subquery finds the list of all the products on orders with kites. 5. The outer query finds the product names. SELECT ProductName FROM dbo.Product WHERE ProductID IN 4. Find all the products sold in orders with kites (SELECT ProductID 267 www.getcoolebook.com Nielsen c11.tex V4 - 07/23/2009 1:54pm Page 268 Part II Manipulating Data With Select FROM dbo.OrderDetail WHERE OrderID IN 3. Find the Kite Orders (SELECT OrderID Find the Orders with Kites FROM dbo.OrderDetail WHERE ProductID IN 2. Find the Kite Products (SELECT ProductID FROM dbo.Product WHERE ProductCategoryID = 1. Find the Kite category (Select ProductCategoryID FROM dbo.ProductCategory Where ProductCategoryName = ‘Kite’ ) ) ) ); You can highlight any of these subqueries and run it as a stand-alone query in a query win- dow by selecting just the subquery and pressing F5. Be sure to include the correct number of closing parentheses. Subquery 1 finds the ProductCategoryID for the kite category and returns a single value. Subquery 2 uses subquery 1 as a WHERE clause expression subquery that returns the kite ProductCategoryID.UsingthisWHERE clause restriction, subquery 2 finds all products for which the ProductCategoryID is equal to the value returned from subquery 2. Subquery 3 uses subquery 2 as a WHERE clause list subquery by searching for all OrderDetail rows that include any one of the productIDs returned by subquery 2. Subquery 4 uses subquery 3 as a WHERE clause list subquery that includes all orders that include kites. The subquery then locates all OrderDetail rows for which the orderID is in the list returned by subquery 3. The outer query uses subquery 4 as a WHERE clause list condition and finds all products for which the ProductID is in the list returned by subquery 4, as follows: ProductName Falcon F-16 Dragon Flight OBX Car Bumper Sticker Short Streamer Cape Hatteras T-Shirt Sky Dancer Go Fly a Kite T-Shirt Long Streamer Rocket Kite OBX T-Shirt 268 www.getcoolebook.com Nielsen c11.tex V4 - 07/23/2009 1:54pm Page 269 Including Data with Subqueries and CTEs 11 Drat! There are kites in the list. They’ll have to be eliminated from the query. To fix the error, the outer query needs to find all the products WHERE: ■ The ProductID is IN an order that included a kite and ■ The ProductID is NOT IN the list of kites Fortunately, subquery 2 returns all the kite products. Adding a copy of subquery 2 with the NOT IN operator to the outer query will remove the kites from the list, as follows: SELECT ProductName FROM dbo.Product WHERE ProductID IN 4. Find all the products sold in orders with kites (SELECT ProductID FROM dbo.OrderDetail WHERE OrderID IN 3. Find the Kite Orders (SELECT OrderID Find the Orders with Kites FROM dbo.OrderDetail WHERE ProductID IN 2. Find the Kite Products (SELECT ProductID FROM dbo.Product WHERE ProductCategoryID = 1. Find the Kite category (Select ProductCategoryID FROM dbo.ProductCategory Where ProductCategoryName = ‘Kite’)))) outer query continued AND ProductID NOT IN (SELECT ProductID FROM dbo.Product WHERE ProductCategoryID = (Select ProductCategoryID FROM dbo.ProductCategory Where ProductCategoryName = ‘Kite’)); Result: ProductName OBX Car Bumper Sticker Short Streamer 269 www.getcoolebook.com Nielsen c11.tex V4 - 07/23/2009 1:54pm Page 270 Part II Manipulating Data With Select Cape Hatteras T-Shirt Go Fly a Kite T-Shirt Long Streamer OBX T-Shirt For comparison purposes, the following queries answer the exact same question but are written with joins. The Product table is referenced twice, so the second reference that represents only the kites has an alias Kite. As with the previous subqueries, the first version of the query locates all products and the second version eliminates the kites: SELECT Distinct P.ProductName FROM dbo.Product AS P JOIN dbo.OrderDetail AS OrderRow ON P.ProductID = OrderRow.ProductID JOIN dbo.OrderDetail AS KiteRow ON OrderRow.OrderID = KiteRow.OrderID JOIN dbo.Product AS Kite ON KiteRow.ProductID = Kite.ProductID JOIN dbo.ProductCategory AS PC ON Kite.ProductCategoryID = PC.ProductCategoryID WHERE PC.ProductCategoryName = ‘Kite’; The only change necessary to eliminate the kites is the addition of another condition to the ProductCategory join. Previously, the join was an equi-join between Product and ProductCategory. Adding a -join condition of <> between the Product table and the ProductCategory table removes any products that are kites, as shown here: SELECT Distinct P.ProductName FROM dbo.Product AS P JOIN dbo.OrderDetail AS OrderRow ON P.ProductID = OrderRow.ProductID JOIN dbo.OrderDetail AS KiteRow ON OrderRow.OrderID = KiteRow.OrderID JOIN dbo.Product AS Kite ON KiteRow.ProductID = Kite.ProductID JOIN dbo.ProductCategory AS PC ON Kite.ProductCategoryID = PC.ProductCategoryID AND P.ProductCategoryID <> Kite.ProductCategoryID Where PC.ProductCategoryName = ‘Kite’; 270 www.getcoolebook.com Nielsen c11.tex V4 - 07/23/2009 1:54pm Page 271 Including Data with Subqueries and CTEs 11 Best Practice S QL is very flexible — there are often a dozen ways to express the same question. Your choice of SQL method should be made first according to your style and to which method enables you to be readable and logically correct, and then according to performance considerations. Test the actual queries for performance but keep in mind that slow and correct beats fast and wrong every time. Using subqueries as tables In the same way that a view may be used in the place of a table within the FROM clause of a SELECT statement, a subquery in the form of a derived table can replace any table, provided the subquery has an alias. This technique is very powerful and is often used to break a difficult query problem down into smaller bite-size chunks. Using a subquery as a derived table is an excellent solution to the aggregate-function problem. When you are building an aggregate query, every column must participate in the aggregate function in some way, either as a GROUP BY column or as an aggregate function (sum(), avg(), count(), max(),or min()). This stipulation makes returning additional descriptive information difficult. However, perform- ing the aggregate functions in a subquery and passing the rows found to the outer query as a derived table enables the outer query to then return any columns desired. For more information about aggregate functions and the group by keyword, see Chapter 12, ‘‘Aggregating Data.’’ The question ‘‘How many of each product have been sold?’’ is easy to answer if only one column from the Product table is included in the result: SELECT P.Code, SUM(Quantity) AS QuantitySold FROM dbo.OrderDetail AS OD JOIN dbo.Product AS P ON OD.ProductID = P.ProductID GROUP BY P.Code ORDER BY P.Code; Result: Code QuantitySold 1002 47.00 1003 5.00 1004 2.00 1012 5.00 271 www.getcoolebook.com . improvements include the following: ■ SQL Server 2005 saw the introduction of the Apply structure for user-defined functions and subqueries. ■ With SQL Server 2008, Microsoft adds row constructors that. query. ■ Also new with SQL server 2008 is composable SQL — a new way to plug together multiple DML statements. Anytime there’s a new way to connect together different parts of the SQL query, it opens. Nielsen c11.tex V4 - 07/23/2009 1:54pm Page 262 Part II Manipulating Data With Select What’s New with Subqueries? S ubqueries are fundamental to SQL and there’s been a steady evolution of their