Module 8: Using XQuery to Link and Query XML Documents Contents Overview Lesson: What Is XQuery? Lesson: Using XQuery 11 Lab 8: Using Simple and Advanced XQuery Expressions 17 Review 22 Course Evaluation 23 Information in this document, including URL and other Internet Web site references, is subject to change without notice Unless otherwise noted, the example companies, organizations, products, domain names, e-mail addresses, logos, people, places, and events depicted herein are fictitious, and no association with any real company, organization, product, domain name, e-mail address, logo, person, places or events is intended or should be inferred Complying with all applicable copyright laws is the responsibility of the user Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property  2001 Microsoft Corporation All rights reserved Microsoft, MS-DOS, Windows, Windows NT, ActiveX, BackOffice, bCentral, BizTalk, FrontPage, MSDN, MSN, Netshow, PowerPoint, SharePoint, Visio, Visual Basic, Visual C++, Visual C#, Visual InterDev, Visual Studio, Windows Media, and Xbox are either registered trademarks or trademarks of Microsoft Corporation in the U.S.A and/or other countries The names of actual companies and products mentioned herein may be the trademarks of their respective owners Module 8: Using XQuery to Link and Query XML Documents iii Instructor Notes Presentation: 50 Minutes Lab: 20 Minutes This module is designed to provide participants with an introduction to the core concepts of XQuery and to familiarize them with the structure of XQuery expressions XQuery can be a challenging topic to teach There is only enough time to cover what it is, what it can do, and its basic syntax, but not in any depth This module leaves the detailed coverage of XQuery for later courses Just as Course 1913A, Exchanging and Transforming Data Using XML and XSLT, is a fiveday course on Extensible Stylesheet Language Transformations (XSLT), it is possible to devote a full five-day course to XQuery after it is fully specified and achieves W3C Recommendation status Stress that XQuery is still in development by the World Wide Web Consortium (W3C) and is not yet finalized Participants may ask why the module is included, considering the state of development of XQuery It is included, because XQuery is expected to be a W3C Recommendation by the end of 2001 and heavily used in the Internet and environments like Microsoft® NET when it is a stable Recommendation XQuery is important, because to effectively data-centric work on the Internet, you need to have XQuery or its equivalent Without such a language and set of capabilities, the use of XML is severely limited Because XML is fundamentally about representing data and metadata, XQuery is essential After completing this module, participants will be able to: ! Understand the key concepts behind XQuery ! Design simple and advanced XQuery expressions ! Identify solutions where XQuery is the suitable technology ! Describe how you might apply XQuery expressions to query an XML document ! Relate XQuery to the broader Microsoft NET Framework and strategy iv Module 8: Using XQuery to Link and Query XML Documents Materials and Preparation Required Materials To teach this module, you need the following materials: ! Microsoft PowerPoint® file 2500A_08.ppt ! Access to the Microsoft MSDN® XQuery demonstration site at http://131.107.228.20/xquerydemo/ ! Access to the W3C XQuery site at http://www.w3.org/XML/Query Download and read the most recent W3C and MSDN Online documentation on XQuery Preparation Tasks To prepare for this module: ! Read all of the materials for this module ! Complete the lab ! Deliver the demonstration that uses the XQuery demonstration site This site is likely to be modified from time to time to reflect W3C progress and developments, so check it regularly so that you are not surprised by changes It is best to review it before each class ! Review as many XQuery use cases as you can to consolidate your understanding of XQuery ! Work out additional examples of where XQuery can be used in your location and with your clients Module 8: Using XQuery to Link and Query XML Documents v Instructor Setup for the Lab For this lab, the instructor computer and all student computers must be connected to the Internet The lab enables participants to use the MSDN XQuery demonstration site to gain more familiarity with the capabilities of XQuery Become thoroughly familiar with the demonstration site and be prepared to answer questions Lab 8: Using Simple and Advanced XQuery Expressions ! Prepare for the lab Download and review the data files from the XQuery demonstration site These are for ease of reference only Review the lab steps and be prepared to explain the XQuery syntax if required Participants are likely to find some of the queries, particularly the advanced queries, somewhat confusing Demonstration There are only two demonstrations in this module The first demonstration gives a quick example of how easy it is to an SQL query with a good graphical tool The discussion point of the demonstration is to contrast this with the tedious coding that would be required for XQuery without such tools You may want to keep this demonstration short or you may want to discuss it in some detail, depending on the backgrounds of the participants Be sure to point out that at present there is no comparable graphical tool for use with XQuery Graphical Queries in SQL and Access ! Prepare for the demonstration Be sure that you are familiar with building queries with both the Microsoft SQL Server™ and Microsoft Access graphical interfaces For the demonstration of the graphical interface in SQL Server, complete the following steps: a Open Microsoft SQL Server Enterprise Manager b Expand the pubs database under Databases c Under the pubs database, click Tables d Right-click the publishers table, point to Open Table, and then click Query e Click the Add table toolbar button, and then add the titles, titleauthor, and authors tables f Choose the following columns: pub_name, title, price, au_lname, and au_fname Remove the * from the SELECT statement so that you only have the indicated columns listed in the SELECT statement g Run the query by clicking the Run icon h In the criteria cell for pub_name, type New Moon Books and then rerun the query vi Module 8: Using XQuery to Link and Query XML Documents (Optional) If you have time, repeat the demonstration, this time using Access a Using Access, open the pubs.mdb file located in the install_folder\ Democode\Mod08 folder b Maximize the Access window and the database container window c In the database objects list, click Tables and then click the publishers table d On the toolbar, click New Object, and then click Query e Choose Design View from the list f On the toolbar, click Show Table to add the titles, titleauthor, and authors tables The tables show the relationship links that have been defined between the primary and foreign keys g Drag the pub_name, title, price, au_lname, and au_fname columns from the tables into the first row of the query grid, and then click the Run icon h Show that you get the same results as you did when you used SQL Server i Switch back to Design view, and in the criteria cell for pub_name, type New Moon Books j Rerun the query k Now select the SQL View of the query, and then compare it with the SQL query The second demonstration introduces the participants to the MSDN XQuery language demonstration site so that they will not have to spend much time becoming familiar with it during the lab Using a Variety of XQuery Methods ! Prepare for the demonstration On the MSDN XQuery demonstration site, open each of the links in the left pane and be sure that you are familiar with the topics presented there Become familiar with the behavior of the Web page You will need to refresh the page or use the Back button to return to the XQuery page after you select links for background topics (for example, Readme, Known issues) The page also refreshes when you select a sample query Show the class the sample files that have been copied to the install_folder\Democode\Mod08 folder Open bib.xml to show the structure of the document Under Default Cases, click Example 2, and then run the query You may have to scroll to see all parts of the query page Discuss the results Repeat this sequence for Example and for Example and walk through each query and the results Module 8: Using XQuery to Link and Query XML Documents vii Module Strategy The module should not take longer than the allotted time to Because this is likely to be new material to most participants, emphasize that this is an introduction to an aspect of XML technologies that is going to be very important to XML and to Microsoft NET Use the following strategy to present this module: ! What Is XQuery? This lesson is built around a number of questions that almost anyone who approaches XQuery will have Move through the topics at a comfortable pace and stop to discuss questions that the participants may have Be prepared to work through the examples and the concepts as needed Stress that XQuery is incomplete, as long as that remains true When XQuery reaches W3C Recommendation status, you will need to adjust the presentation of the module if the course is not revised Be sure to explain FLWR expressions These will be seen in the examples in the second lesson and in the lab Work through each of the topic slides and discuss the central concepts and associated ideas The class may want to discuss some topics in more depth Keep an eye on the time Be sure that you are comfortable with the complexities of the XQuery examples The best way to prepare for this is to read the W3C papers carefully and to spend adequate time with the XQuery demonstration site to get a good idea of how these queries look and behave ! Using XQuery This lesson is designed to focus attention on examples of XQuery syntax, which you can work through in detail with the class Be ready to expand upon these examples with what you learned from the W3C documents and the XQuery demonstration site Focus on the basics and try not to discuss the very advanced features of XQuery, because they will likely be too difficult for newcomers to the topic Discuss the similarities and differences between SQL queries and XML queries Emphasize the differences between SQL and XML in their ability to handle redefinition of output Also emphasize that XML is self-defining and self-modifying in this regard Module 8: Using XQuery to Link and Query XML Documents Overview ! What Is XQuery? ! Using XQuery *****************************ILLEGAL FOR NON-TRAINER USE****************************** Introduction In this module, you will see why Extensible Markup Language (XML) requires a specific query language and how XQuery fulfills this requirement This module is not designed to give you an exhaustive introduction to XQuery, because there is neither time nor space for that in this course It is designed to provide you with an introduction to XQuery and the background of significant developments in XQuery This module shows the difference between XQuery and XML Path Language (XPath) It then shows examples of XQuery expressions and syntax with sample XML documents to show how XQuery enables you to retrieve data in a manner analogous to an SQL query in a relational database Objectives After completing this module, you will be able to: ! Understand the key concepts behind XQuery ! Design simple and advanced XQuery expressions ! Identify solutions where XQuery is the suitable technology ! Describe how you might apply XQuery expressions to query an XML document ! Relate XQuery to the broader Microsoft® NET Framework and strategy Module 8: Using XQuery to Link and Query XML Documents Lesson: What Is XQuery? ! Why Do You Need XQuery? ! XQuery Expression Basics ! XQuery and SQL *****************************ILLEGAL FOR NON-TRAINER USE****************************** In this lesson the rationale behind the development of XQuery is explained You will learn how XQuery and XPath are related and how XQuery and SQL are related You will gain a first exposure to the query syntax of XQuery Lesson Objectives After completing this lesson you will be able to: ! Explain the relationship between XQuery and XPath ! Interpret the syntax of an XQuery expression ! Describe the uses of XQuery in relation to XPath and SQL 10 Module 8: Using XQuery to Link and Query XML Documents Demonstration: Graphical Queries in SQL and Access ! In this demonstration you will see a query being built in SQL Server 2000 using the graphical query interface ! (Optional) If time permits you will see a similar query built in Access 2002 *****************************ILLEGAL FOR NON-TRAINER USE****************************** In the demonstration, you will see a comparison of the graphical creation of an SQL query and a current prototype from Microsoft of a basic query interface Module 8: Using XQuery to Link and Query XML Documents 11 Lesson: Using XQuery ! Basic XQuery ! Advanced XQuery ! The Future of XQuery *****************************ILLEGAL FOR NON-TRAINER USE****************************** Introduction To use XQuery, you need specific skills beyond what is required by basic XML In this lesson, you will learn those skills and how to write an XQuery expression Lesson objectives After completing this lesson, you will be able to: ! Create basic queries using the XPath features of XQuery ! Explain and use the advanced XQuery operators ! Identify situations where you can use XQuery ! Plan and design how and where XQuery might be used in a Microsoft NET solution or application 12 Module 8: Using XQuery to Link and Query XML Documents Basic XQuery ! Basic XPath type queries document("animals.xml")//section document("animals.xml")//section [4]//illustrations [4]//illustrations [label = “Cat family”] [label = “Cat family”] document("animals.xml")//section document("animals.xml")//section [3 TO 7]//illustration [3 TO 7]//illustration document("animals.xml")//section document("animals.xml")//section [title = [title = "Cats"]//illusref/@illusid=>illus/label "Cats"]//illusref/@illusid=>illus/label *****************************ILLEGAL FOR NON-TRAINER USE****************************** Introduction In this topic, you will examine the structure of basic XQuery expressions You will need to take your time to examine these examples As a newcomer to XQuery, you might think its syntax is rather awkward Just as it usually takes time to get used to SQL queries for those new to relational databases, it will take time to get used to XQuery, because it is even less familiar Knowledge of SQL is only a partial advantage in learning XQuery Some may find that SQL knowledge hinders learning XQuery Examples The following are examples of types of XQuery expressions: ! Given a document called animals.xml, find, in the fourth section, the illustrations with the label “Cat family.” document ("animals.xml")//section[4]//illustration[label = "Cat family"] ! Given the same document, find all the illustrations in sections through document("animals.xml")//section[3 TO 7]//illustration ! You can use a special XQuery operator called a dereference (represented by =>) with documents that have an attached DTD or XSD and therefore allow the use of the ID, IDREF, and IDREFS special node types This enables you to select items based on the attributes of an element Given the same document, use the dereference operator to find the label of the element that is referenced by the illusid attribute of an element document("animals.xml")//section[title = "Cats"]//illusref/@illusid=>illus/label Module 8: Using XQuery to Link and Query XML Documents 13 Advanced XQuery FOR $p IN FOR $p IN distinct(document("books.xml")//publisher distinct(document("books.xml")//publisher LET $a := avg(document("books.xml") LET $a := avg(document("books.xml") //book/[publisher = $p]/price) //book/[publisher = $p]/price) RETURN RETURN {$p/text () } {$p/text () } {$a} {$a} Transforming a Document Alphabetical List *****************************ILLEGAL FOR NON-TRAINER USE****************************** Introduction Most of the interest in XQuery is due to its advanced capabilities These include the use of FLWR expressions to handle complex queries and manipulate query results You can manipulate query results by restructuring the XML so that there are different types of nodes or nodes that are in a different order from the source You can also join multiple data sources in joins, sort, and perform calculations Examples In this example, a LET clause binds the variable $a to the average price of books that are published by the publishers that are bound in the FOR clause: FOR $p IN distinct(document("books.xml")//publisher LET $a := avg(document("books.xml")//book/[publisher = $p]/price) RETURN {$p/text () ] {$a} 14 Module 8: Using XQuery to Link and Query XML Documents In this example, a document is transformed so that it has a different output form than the source document: 80] SORTBY (author[1], title) This example produces a list of publishers in alphabetical order For each publisher, book titles and prices are listed, with prices in descending order {FOR $p IN distinct(document("books.xml")//publisher) RETURN {$p/text()} {FOR $b IN document ("books.xml")//book[publisher = $p] RETURN SORTBY (price DESCENDING) } SORTBY(name) } Module 8: Using XQuery to Link and Query XML Documents 15 The Future of XQuery ! The universal query language for XML and Web-based data work ! Opens new possibilities to query combinations of data stores " XML and XML fragments in all kinds of documents " XML bound in relational database management systems *****************************ILLEGAL FOR NON-TRAINER USE****************************** Introduction The future of XQuery depends on many factors, but you will probably see the following patterns XQuery and the Internet The overall flexibility and power of XQuery and its associated technologies (XPath 2.0 and XSD) make XQuery the query language endorsed by W3C for use in XML documents and environments In the foreseeable future, you will see XQuery as the preferred query language for Internet-based and data-centric tasks Tools will also be developed that make XQuery easier to use when vendors, developers, and users work with XQuery and discover its strengths and weaknesses If the full power of XQuery as conceived by W3C can be adequately and conveniently achieved in production environments, XQuery is destined to become the common query language for all Internet-based commerce Because it is self-defining and self-modifying like XML, it is ideally suited to efficient querying over the Internet for virtually any purpose and with virtually any data store or source Such flexibility is highly attractive to Web and application developers if XQuery can also query data in a robust and secure manner 16 Module 8: Using XQuery to Link and Query XML Documents Demonstration: Using XQuery ! In this demonstration you will see one implementation of XQuery ideas as developed by Microsoft team members ! This demonstration is illustrative only as the specifications from the W3C are not finalized for XQuery ! This demonstration is web based and is found at: http://131.107.228.20/xquerydemo/demo.aspx *****************************ILLEGAL FOR NON-TRAINER USE****************************** Module 8: Using XQuery to Link and Query XML Documents 17 Lab 8: Using Simple and Advanced XQuery Expressions ! Exercise 1: Using Basic XQuery ! If Time Permits: Exploring XQuery *****************************ILLEGAL FOR NON-TRAINER USE****************************** Objectives After completing this lab, you will be able to: ! Use the Microsoft MSDN® XQuery language demonstration site at http://131.107.228.20/xquerydemo/ ! Run basic and advanced XQuery expressions by using the demonstration site ! Relate the XQuery examples on the demonstration site to patterns of data output that will result from any XQuery expression Note The files located in the install_folder\Labfiles\Mod08\Solution are the sample XML files provided at http://131.107.228.20/xquerydemo/demo.aspx Only bib.xml is required to complete the lab The rest are provided to allow you to conveniently view the XML source files used by the demo queries Prerequisites Before working on this lab, you must have a reasonable understanding of XQuery syntax Note You can download the assemblies used to run the Web-based demonstration so that you can try XQuery on your own sample files Go to http://131.107.228.20/xquerydemo/ and click Downloads for more information Estimated time to complete this lab: 20 minutes 18 Module 8: Using XQuery to Link and Query XML Documents Exercise Using Basic XQuery In this exercise, you will use the MSDN demonstration site to run several simple XQuery expressions against the demonstration data documents The XQuery demonstration Web site allows you to run sample XQuery statements against sample files, modify these sample queries, or create your own queries Some queries will return no results, either because the query violates the syntax rules of XQuery or because the query returns no records The XQuery demonstration allows you to switch the mode of the query from Execute to Compile Both modes return an error message if the query violates XQuery syntax The Compile mode confirms that the query entered actually compiles ! Use the XQuery demonstration site Open the XQuery demonstration site at http://131.107.228.20/xquerydemo/ In the left pane, click Readme This demonstration uses files provided by W3C Open the links to the files listed and view their contents In the left pane, click Default Cases, and then click Example 1: for statement What is this query going to do? Click Execute Query Does it what you expected? In the Query Expression box, change the year value from “1994” to “1991,” and then run the query Now what happens? Change the equal sign (=) to a right angle bracket (>), and then run the query again What happens this time? Use Internet Explorer to open bib.xml, located in the install_folder\ Labfiles\Lab08\Solution folder Compare its contents with your results Module 8: Using XQuery to Link and Query XML Documents 19 ! Run an advanced query On the demonstration site, in the left pane, under Default Cases, click Example 6: predicate What is this query designed to do? In the Query Expression box, change the bib.xml reference to books.xml The query does not return any records Why? ! Run an advanced query that shows calculations On the demonstration site, in the left pane, under Default Cases, click Example 5: min(), max(), avg(), sum() and count() functions What is this query designed to do? Run the query Does it what you expected? What values does your query return? 20 Module 8: Using XQuery to Link and Query XML Documents ! Run an advanced query that checks for text contained in an XML file On the demonstration site, in the left pane, click Use Case "TEXT", and then click Example What is this query designed to do? Run the query Which items are returned? Delete Foobar from the query, and then run it again What happens? How many results are returned this time? Close all open windows Module 8: Using XQuery to Link and Query XML Documents 21 If Time Permits Exploring XQuery If you have extra time, use the XQuery demonstration site and view as many of the other queries as you can For some queries, you will get error messages, because not all functionality that was planned for XQuery is implemented in the demonstration site What is your overall impression of XQuery from this set of exercises? _ _ _ _ 22 Module 8: Using XQuery to Link and Query XML Documents Review ! What Is XQuery? ! Working with XQuery *****************************ILLEGAL FOR NON-TRAINER USE****************************** Why is XQuery being developed? There is a need to extend queries across XML and relational data sources: XML documents, XML fragments, and conventional data storage in relational databases Existing XML technologies not this adequately XQuery has been developed to address this need What does the acronym FLWR refer to? The letters of the FLWR acronym stand for these query clauses: FOR, LET, WHERE, and RETURN What other XML language can you use to a limited degree to express a query? Why is this approach not adequate for real data purposes? You can use XPath to query XML Many data queries require sorting, projection, joining, and advanced functions XPath does not provide support for this Compare SQL as a query language and XQuery How are they similar, and how are they different? Both query languages are designed to retrieve data and to manipulate it while it is being returned SQL can be used only with relational databases, but XQuery can be used with XML and other data stores Module 8: Using XQuery to Link and Query XML Documents 23 Course Evaluation *****************************ILLEGAL FOR NON-TRAINER USE****************************** Your evaluation of this course will help Microsoft understand the quality of your learning experience At a convenient time between now and the end of the course, please complete a course evaluation, which is available at http://www.metricsthatmatter.com/survey Microsoft will keep your evaluation strictly confidential and will use your responses to improve your future learning experience THIS PAGE INTENTIONALLY LEFT BLANK ... an SQL query and a current prototype from Microsoft of a basic query interface Module 8: Using XQuery to Link and Query XML Documents 11 Lesson: Using XQuery ! Basic XQuery ! Advanced XQuery. .. data is stored in both XML data stores and relational databases 4 Module 8: Using XQuery to Link and Query XML Documents XQuery and XPath 2.0 The ideas incorporated in XQuery for powerful querying... apply XQuery expressions to query an XML document ! Relate XQuery to the broader Microsoft NET Framework and strategy iv Module 8: Using XQuery to Link and Query XML Documents Materials and Preparation