Đang tải... (xem toàn văn)
Bài giảng SQL
Pro Full-Text Search in SQL Server 2008■■■Michael Coles with Hilary Cotter Pro Full-Text Search in SQL Server 2008 Copyright © 2009 by Michael Coles and Hilary CotterAll rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher.ISBN-13 (pbk): 978-1-4302-1594-3ISBN-13 (electronic): 978-1-4302-1595-0Printed and bound in the United States of America 9 8 7 6 5 4 3 2 1Trademarked names may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, we use the names only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark.Lead Editor: Jonathan GennickTechnical Reviewer: Steve JonesEditorial Board: Clay Andres, Steve Anglin, Mark Beckner, Ewan Buckingham, Tony Campbell, Gary Cornell, Jonathan Gennick, Michelle Lowman, Matthew Moodie, Jeffrey Pepper, Frank Pohlmann, Ben Renow-Clarke, Dominic Shakeshaft, Matt Wade, Tom WelshProject Manager: Denise Santoro LincolnCopy Editor: Benjamin BergAssociate Production Director: Kari Brooks-CoponyProduction Editor: Laura EstermanCompositor/Artist: Octal Publishing, Inc.Proofreader: Patrick VincentIndexer: Broccoli Information ManagementCover Designer: Kurt KramesManufacturing Director: Tom DebolskiDistributed to the book trade worldwide by Springer-Verlag New York, Inc., 233 Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax 201-348-4505, e-mail orders-ny@springer-sbm.com, or visit http://www.springeronline.com. For information on translations, please contact Apress directly at 2855 Telegraph Avenue, Suite 600, Berkeley, CA 94705. Phone 510-549-5930, fax 510-549-5939, e-mail info@apress.com, or visit http://www.apress.com. Apress and friends of ED books may be purchased in bulk for academic, corporate, or promotional use. eBook versions and licenses are also available for most titles. For more information, reference our Special Bulk Sales–eBook Licensing web page at http://www.apress.com/info/bulksales.The information in this book is distributed on an “as is” basis, without warranty. Although every precaution has been taken in the preparation of this work, neither the author(s) nor Apress shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the information contained in this work. The source code for this book is available to readers at http://www.apress.com. For Devoné and Rebecca—Michael vContents at a GlanceAbout the Authors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiiAbout the Technical Reviewer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvAcknowledgments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviiIntroduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix■CHAPTER 1 SQL Server Full-Text Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1■CHAPTER 2 Administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19■CHAPTER 3 Basic and Advanced Queries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45■CHAPTER 4 Client Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75■CHAPTER 5 Multilingual Searching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99■CHAPTER 6 Indexing BLOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119■CHAPTER 7 Stoplists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145■CHAPTER 8 Thesauruses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165■CHAPTER 9 iFTS Dynamic Management Views and Functions . . . . . . . . . . . . 185■CHAPTER 10 Filters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207■CHAPTER 11 Advanced Search Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239■APPENDIX A Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257■APPENDIX B iFTS_Books Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265■APPENDIX C Vector-Space Searches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269■INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275 viiContentsAbout the Authors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiiAbout the Technical Reviewer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvAcknowledgments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviiIntroduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix■CHAPTER 1 SQL Server Full-Text Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1Welcome to Full-Text Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1History of SQL Server FTS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4Goals of Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6Mechanics of Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8iFTS Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9Indexing Process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Query Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Search Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12Measuring Quality. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13Synonymy and Polysemy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16■CHAPTER 2 Administration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19Initial Setup and Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19Enabling Database Full-Text Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21Creating Full-Text Catalogs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21The New Full-Text Catalog Wizard . . . . . . . . . . . . . . . . . . . . . . . . . . . 21The CREATE FULLTEXT CATALOG Statement . . . . . . . . . . . . . . . . . . 23Upgrading Full-Text Catalogs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24Creating Full-Text Indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25The Full-Text Indexing Wizard. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25The DocId Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33The CREATE FULLTEXT INDEX Statement . . . . . . . . . . . . . . . . . . . . . 33 viii■CONTENTS Full-Text Index Population . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35Full Population. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35Incremental Population . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36Update Population. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37Additional Index Population Options . . . . . . . . . . . . . . . . . . . . . . . . . . 37Catalog Rebuild and Reorganization. . . . . . . . . . . . . . . . . . . . . . . . . . 37Scheduling Populations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39Backups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40SQL Profiler Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41System Procedures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43■CHAPTER 3 Basic and Advanced Queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45iFTS Predicates and Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45FREETEXT and FREETEXTTABLE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47Adding a Language Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51Returning the Top N by RANK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56CONTAINS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58Phrase Searches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59Boolean Searches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60Prefix Searches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63Generational Searches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64Proximity Searches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65Weighted Searches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67CONTAINSTABLE Searches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69Advanced Search Topics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71Using XQuery contains() Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71Column Rank-Multiplier Searches . . . . . . . . . . . . . . . . . . . . . . . . . . . 71Taxonomy Search and Text Mining . . . . . . . . . . . . . . . . . . . . . . . . . . 73Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74■CHAPTER 4 Client Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75Hit Highlighting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75The Procedure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75Calling the Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82Search Engine–Style Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84Defining a Grammar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 ■CONTENTSixExtended Backus-Naur Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87Implementing the Grammar with Irony. . . . . . . . . . . . . . . . . . . . . . . . 88Generating the iFTS Query. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91Converting a Google-Style Query . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94Querying with the New Grammar . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96■CHAPTER 5 Multilingual Searching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99A Brief History of Written Language. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100iFTS and Language Complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101Writing Symbols and Alphabets . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102Bidirectional Writing and Capitalization . . . . . . . . . . . . . . . . . . . . . . 103Hyphenation and Compound Words . . . . . . . . . . . . . . . . . . . . . . . . . 104Nonalphanumeric Characters and Accent Marks . . . . . . . . . . . . . . 105Token Position Context. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105Generational Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106Gender . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106Storing Multilingual Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107Storing Plain Text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108Storing XML. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108Storing HTML Documents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111Storing Microsoft Office Documents. . . . . . . . . . . . . . . . . . . . . . . . . 112Storing Other Document Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112Detecting Content Language . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112Designing Tables to Store Multilingual Content . . . . . . . . . . . . . . . . . . . . 112Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118■CHAPTER 6 Indexing BLOBs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119LOB Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120Character LOB Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120XML LOB Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122Binary LOB Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127FILESTREAM BLOB Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130Efficiency Advantages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130FILESTREAM Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132T-SQL Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135Storage Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137OpenSqlFilestream API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 x■CONTENTS ■CHAPTER 7 Stoplists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145System Stoplists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145Creating Custom Stoplists. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147Managing Stoplists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150Upgrading Noise Word Lists to Stoplists . . . . . . . . . . . . . . . . . . . . . . . . . . 157Stoplist Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161Stoplists and Indexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161Stoplists and Queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164■CHAPTER 8 Thesauruses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165Thesaurus Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165Editing and Loading Thesaurus Files . . . . . . . . . . . . . . . . . . . . . . . . 167Expansion Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173Replacement Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175Global and Local Thesauruses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176A Practical Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177Translation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179Word Bags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180Additional Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180Accent and Case Sensitivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180Nonrecursion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181Overlapping Rules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182Stoplists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182General Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183■CHAPTER 9 iFTS Dynamic Management Views and Functions . . . . . . . 185iFTS and Transparency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185DMVs and DMFs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186Looking Inside the Full-Text Index . . . . . . . . . . . . . . . . . . . . . . . . . . 186Parsing Text. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188Accessing Full-Text Index Entries . . . . . . . . . . . . . . . . . . . . . . . . . . . 189Retrieving Population Information. . . . . . . . . . . . . . . . . . . . . . . . . . . 191Services and Memory Usage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 ■CONTENTSxiCatalog Views. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197Listing Full-Text Catalogs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197Retrieving Full-Text Index Metadata. . . . . . . . . . . . . . . . . . . . . . . . . 198Revealing Stoplists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202Viewing Supported Languages and Document Types . . . . . . . . . . 204Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205■CHAPTER 10 Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207Introducing Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207Standard Filters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207Third-Party Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208Custom Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209Custom Filter Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210Filter Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211Custom Filter Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214Filter Class Factory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215Filter Class. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220Compiling and Installing the Filter . . . . . . . . . . . . . . . . . . . . . . . . . . 229Testing the Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232Gatherer and Protocol Handler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235Word Breakers and Stemmers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237■CHAPTER 11 Advanced Search Techniques. . . . . . . . . . . . . . . . . . . . . . . . . . . . 239Spelling Suggestion and Correction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239Hamming Distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240Spelling Suggestion Implementation . . . . . . . . . . . . . . . . . . . . . . . . 241Name Searching. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243Phonetic Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244Soundex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244NYSIIS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245String Similarity Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247Longest Common Subsequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247Edit Distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249N-Grams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256 xii■CONTENTS ■APPENDIX A Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257■APPENDIX B iFTS_Books Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265Installing the Sample Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267Installing the Phonetic Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268Sample Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268■APPENDIX C Vector-Space Searches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269Documents As Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269■INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275 [...]... including SQL Server Central, ASPToday, and SQL Server Standard Michael is the author of the books Pro SQL Server 2008 XML (Apress, 2008) and Pro T -SQL 2008 Programmer’s Guide (Apress, 2008), and he is a contributor to Accelerated SQL Server 2008 (Apress, 2008) His current projects include speaking engagements and researching new SQL Server 2008 encryption and security functionality ■ HILARY COTTER is a SQL. .. SQL Server process: The SQL Server process contains both the SQL Server query processor, which compiles and executes SQL queries, and the full-text engine, which compiles and executes full-text queries This tight integration of the SQL Server and full-text query processors in SQL Server 2008 is a significant improvement over prior versions of SQL Server full-text search, allowing SQL Server to generate... SQL Server provides this advanced linguistic search capability out-of-the-box FullText Search (FTS) has been included with SQL Server since the SQL Server 7 release FTS allows you to perform linguistic searches of documents and text content stored in SQL Server databases using standard T -SQL queries FTS is a powerful tool that can be used to implement enterprise-class linguistic database searches SQL. .. T -SQL, as most of the sample code in the book is written in SQL Server 2008 T -SQL Sample code is also provided in C# and C++, where appropriate Although knowledge of these programming languages is not required, basic knowledge of procedural programming will help in understanding the code samples How This Book Is Structured This book is designed to address the needs of T -SQL developers who develop SQL. .. while (i < 10) T -SQL source code is also shown in code font Though T -SQL is not case sensitive, we’ve consistently capitalized keywords for readability Also note that, for readability purposes, we’ve lowercased data type names in T -SQL code Finally, following Microsoft’s best practices, we consistently use the semicolon T -SQL statement terminator The following demonstrates a line of T -SQL code: DECLARE... These custombuilt search routines achieved varying degrees of success SQL Server full-text search was designed to meet developer demand for a standard toolset to search documents and textual data stored in any SQL Server database SQL Server iFTS represents the next generation of SQL Server-based full-text search The iFTS functionality in SQL Server provides significant advantages over other alternatives,... cooperation with the SQL Server query engine to perform efficient linguistic searches We’ve highlighted some of the more important components involved in iFTS in the simplified diagram shown in Figure 1-3 Client Application Query SQL Server Process SQL Server Query Processor Full-Text Engine SQL Query Compilation and Execution Full-Text Query Compilation and Execution Indexer SQL Database X Lorem ipsum... applied science and engineering He is the author of a book on SQL Server replication and has written numerous white papers and articles on SQL Server and databases xiii About the Technical Reviewer ■ STEVE JONES, a Microsoft MVP, is the founder and editor of SQLServerCentral, the largest SQL Server community on the Internet He has been working with SQL Server since 1991 and has published numerous books and... works with the SQL Server query processor to return relevant results back to the user The new tighter integration between the full-text query processor and the SQL Server query processor (both are now hosted together within the SQL Server process) provides the ability to perform full-text searches that are more highly optimized than in previous versions of SQL Server As an example, in SQL Server 2005... administer, troubleshoot, and generally use SQL Server’s built-in linguistic search functionality in your own applications In this book, we’ll provide an in-depth tour of SQL Server 2008’s FTS features and functionality, from both the server and client perspective Who This Book Is For This book is intended for SQL Server developers and DBAs who want to get the most out of SQL Server 2008 Integrated Full-Text . including SQL Server Central, ASPToday, and SQL Server Standard. Michael is the author of the books Pro SQL Server 2008 XML (Apress, 2008) and Pro T -SQL 2008. is the founder and editor of SQLServer-Central, the largest SQL Server community on the Internet. He has been working with SQL Server since 1991 and has