Tài liệu HBase in Action docx

362 4,864 3
  • Loading ...
1/362 trang
Tải xuống

Thông tin tài liệu

Ngày đăng: 18/02/2014, 06:20

MANNINGNick Dimiduk Amandeep KhuranaFOREWORD BY Michael Stackwww.it-ebooks.infoHBase in ActionNICK DIMIDUK AMANDEEP KHURANATECHNICAL EDITORMARK HENRY RYANMANNINGShelter Islandwww.it-ebooks.infoFor online information and ordering of this and other Manning books, please visitwww.manning.com. The publisher offers discounts on this book when ordered in quantity. For more information, please contactSpecial Sales DepartmentManning Publications Co.20 Baldwin RoadPO Box 261Shelter Island, NY 11964Email: orders@manning.com©2013 by Manning Publications Co. All rights reserved.No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher.Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps.Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end.Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine.Manning Publications Co. Development editors: Renae Gregoire, Susanna Kline20 Baldwin Road Technical editor: Mark Henry RyanPO Box 261 Technical proofreaders: Jerry Kuch, Kristine KuchShelter Island, NY 11964 Copyeditor: Tiffany TaylorProofreaders: Elizabeth Martin, Alyson BrenerTypesetter: Gordan SalinovicCover designer: Marija TudorISBN 9781617290527Printed in the United States of America1 2 3 4 5 6 7 8 9 10 – MAL – 17 16 15 14 13 12www.it-ebooks.infoiiibrief contentsPART 1 HBASE FUNDAMENTALS. 11 ■ Introducing HBase 32 ■ Getting started 213 ■ Distributed HBase, HDFS, and MapReduce 51PART 2 ADVANCED CONCEPTS 834 ■ HBase table design 855 ■ Extending HBase with coprocessors 1266 ■ Alternative HBase clients 143PART 3 EXAMPLE APPLICATIONS 1797 ■ HBase by example: OpenTSDB 1818 ■ Scaling GIS on HBase 203PART 4 OPERATIONALIZING HBASE . 2379 ■ Deploying HBase 23910 ■ Operations 264www.it-ebooks.infowww.it-ebooks.infovcontentsforeword xiiiletter to the HBase community xvpreface xviiacknowledgments xixabout this book xxiabout the authors xxvabout the cover illustration xxviPART 1 HBASE FUNDAMENTALS . 11 Introducing HBase 31.1 Data-management systems: a crash course 5Hello, Big Data 6■Data innovation 7■The rise of HBase 81.2 HBase use cases and success stories 8The canonical web-search problem: the reason for Bigtable’s invention 9■Capturing incremental data 10■Content serving 13■Information exchange 141.3 Hello HBase 15Quick install 16■Interacting with the HBase shell 18Storing data 181.4 Summary 20www.it-ebooks.infoCONTENTSvi2 Getting started 212.1 Starting from scratch 22Create a table 22■Examine table schema 23■Establish a connection 24■Connection management 242.2 Data manipulation 25Storing data 25■Modifying data 26■Under the hood: the HBase write path 26■Reading data 28■Under the hood: the HBase read path 29■Deleting data 30■Compactions: HBase housekeeping 30■Versioned data 31■Data model recap 322.3 Data coordinates 332.4 Putting it all together 352.5 Data models 39Logical model: sorted map of maps 39■Physical model: column family oriented 412.6 Table scans 42Designing tables for scans 43■Executing a scan 45■Scanner caching 45■Applying filters 462.7 Atomic operations 472.8 ACID semantics 482.9 Summary 483 Distributed HBase, HDFS, and MapReduce 513.1 A case for MapReduce 52Latency vs. throughput 52■Serial execution has limited throughput 53■Improved throughput with parallel execution 53■MapReduce: maximum throughput with distributed parallelism 553.2 An overview of Hadoop MapReduce 56MapReduce data flow explained 57■MapReduce under the hood 613.3 HBase in distributed mode 62Splitting and distributing big tables 62■How do I find my region? 64■How do I find the -ROOT- table? 653.4 HBase and MapReduce 68HBase as a source 68■HBase as a sink 70■HBase as a shared resource 71www.it-ebooks.infoCONTENTSvii3.5 Putting it all together 75Writing a MapReduce application 76■Running a MapReduce application 773.6 Availability and reliability at scale 78HDFS as the underlying storage 793.7 Summary 81PART 2 ADVANCED CONCEPTS 834 HBase table design 854.1 How to approach schema design 86Modeling for the questions 86■Defining requirements: more work up front always pays 89■Modeling for even distribution of data and load 92■Targeted data access 984.2 De-normalization is the word in HBase land 1004.3 Heterogeneous data in the same table 1024.4 Rowkey design strategies 1034.5 I/O considerations 104Optimized for writes 104■Optimized for reads 106Cardinality and rowkey structure 1074.6 From relational to non-relational 108Some basic concepts 109■Nested entities 110■Some things don’t map 1124.7 Advanced column family configurations 113Configurable block size 113■Block cache 114■Aggressive caching 114■Bloom filters 114■TTL 115Compression 115■Cell versioning 1164.8 Filtering data 117Implementing a filter 119■Prebundled filters 1214.9 Summary 1245 Extending HBase with coprocessors 1265.1 The two kinds of coprocessors 127Observer coprocessors 127■Endpoint Coprocessors 1305.2 Implementing an observer 131Modifying the schema 131■Starting with the Base 132Installing your observer 135■Other installation options 137www.it-ebooks.infoCONTENTSviii5.3 Implementing an endpoint 137Defining an interface for the endpoint 138■Implementing the endpoint server 138■Implement the endpoint client 140Deploying the endpoint server 142■Try it! 1425.4 Summary 1426 Alternative HBase clients 1436.1 Scripting the HBase shell from UNIX 144Preparing the HBase shell 145■Script table schema from the UNIX shell 1456.2 Programming the HBase shell using JRuby 147Preparing the HBase shell 147■Interacting with the TwitBase users table 1486.3 HBase over REST 150Launching the HBase REST service 151■Interacting with the TwitBase users table 1536.4 Using the HBase Thrift gateway from Python 156Generating the HBase Thrift client library for Python 157Launching the HBase Thrift service 159■Scanning the TwitBase users table 1596.5 Asynchbase: an alternative Java HBase client 162Creating an asynchbase project 163■Changing TwitBase passwords 165■Try it out 1766.6 Summary 177PART 3 EXAMPLE APPLICATIONS 1797 HBase by example: OpenTSDB 1817.1 An overview of OpenTSDB 182Challenge: infrastructure monitoring 183■Data: time series 184Storage: HBase 1857.2 Designing an HBase application 186Schema design 187■Application architecture 1907.3 Implementing an HBase application 194Storing data 194■Querying data 1997.4 Summary 202www.it-ebooks.infoCONTENTSix8 Scaling GIS on HBase 2038.1 Working with geographic data 2038.2 Designing a spatial index 206Starting with a compound rowkey 208■Introducing the geohash 209■Understand the geohash 211■Using the geohash as a spatially aware rowkey 2128.3 Implementing the nearest-neighbors query 2168.4 Pushing work server-side 222Creating a geohash scan from a query polygon 224■Within query take 1: client side 228■Within query take 2: WithinFilter 2318.5 Summary 235PART 4 OPERATIONALIZING HBASE 2379 Deploying HBase 2399.1 Planning your cluster 240Prototype cluster 241■Small production cluster (10–20 servers) 242Medium production cluster (up to ~50 servers) 243■Large production cluster (>~50 servers) 243■Hadoop Master nodes 243■HBase Master 244■Hadoop DataNodes and HBase RegionServers 245ZooKeeper(s) 246■What about the cloud? 2469.2 Deploying software 248Whirr: deploying in the cloud 2499.3 Distributions 250Using the stock Apache distribution 251■Using Cloudera’s CDH distribution 2529.4 Configuration 253HBase configurations 253■Hadoop configuration parameters relevant to HBase 260■Operating system configurations 2619.5 Managing the daemons 2619.6 Summary 26310 Operations 26410.1 Monitoring your cluster 265How HBase exposes metrics 266■Collecting and graphing the metrics 266■The metrics HBase exposes 268■Application-side monitoring 272www.it-ebooks.info[...]... of your HBase cluster 273 Performance testing 273 What impacts HBase s performance? Tuning dependency systems 277 Tuning HBase 278 ■ ■ 10.3 Cluster management 283 Starting and stopping HBase 283 Graceful stop and decommissioning nodes 284 Adding nodes 285 Rolling restarts and upgrading 285 bin /hbase and the HBase shell 286 Maintaining consistency—hbck 293 Viewing HFiles and HLogs 296 Presplitting tables... next generation of HBase users The single strongest component of HBase is its thriving community—we hope you’ll join us in that community and help it continue to innovate in this new era of data systems NICK DIMIDUK www.it-ebooks.info PREFACE xvii If you’re reading this, you’re presumably interested in knowing how I got involved with HBase Let me start by saying thank you for choosing this book as your... HBase behaves the way it does, and you’ll be able to ask intelligent questions This book won’t turn you into an HBase committer It will give you a practical introduction to HBase xxi www.it-ebooks.info xxii ABOUT THIS BOOK Roadmap HBase in Action is organized into four parts The first two are about using HBase In these six chapters, you’ll go from HBase novice to fluent in writing applications on HBase. .. that doesn’t constrain the kind of data you store HBase isn’t a relational database like the ones to which you’re likely accustomed It doesn’t speak SQL or enforce relationships within your data It doesn’t 3 www.it-ebooks.info 4 CHAPTER 1 Introducing HBase allow interrow transactions, and it doesn’t mind storing an integer in one row and a string in another for the same column HBase is designed to... us, you’ll want to play with HBase before going much further We’ll wrap up by walking through installing HBase on your laptop, tossing in some data, and pulling it out Context is important, so let’s start at the beginning 1 2 1 2 HBase project mailing lists: http:/ /hbase. apache.org/mail-lists.html HBase JIRA site: https://issues.apache.org/jira/browse /HBASE www.it-ebooks.info Data-management systems:... nothing less It doesn’t venture into the bowels of the internal HBase implementation It doesn’t cover the broad range of topics necessary for understanding the Hadoop ecosystem HBase in Action maintains a singular focus on using HBase It aims to educate you enough that you can build an application on top of HBase and launch that application into production Along the way, you’ll learn some of those HBase. .. of HBase Most important, you’ll learn how to think in HBase The two chapters in part 3 move beyond sample applications and give you a taste of HBase in real applications Part 4 is aimed at taking your HBase application from a development prototype to a full-fledged production system Chapter 1 introduces the origins of Hadoop, HBase, and NoSQL in general We explain what HBase is and isn’t, contrast HBase. .. becoming more reliable and performant, due in large part to the engineering effort invested by the various companies backing and using it As more commercial vendors provide support, users are increasingly confident in using the system for critical applications A technology designed to store a continuously updated copy of the internet turns out to be pretty good at other things internet-related HBase. .. MapReduce team, building the first versions of their hosted HBase offering Nick also lived in Seattle, and we met often and talked about the projects we were working on Toward the end of 2010, the idea of writing HBase in Action for Manning came up We initially scoffed at the thought of writing a book on HBase, and I remember saying to Nick, “It’s gets, puts, and scans—there’s not a lot more to HBase from the... www.manning.com/HBaseinAction In the spirit of open source, we hope you’ll find our example code useful in your applications We encourage you to play with it, modify it, fork it, and share it with others If you find bugs, please let us know in the form of issues, or, better still, pull requests As they often say in the open source community: patches welcome Author Online Purchase of HBase in Action includes . Implementing an endpoint 137Defining an interface for the endpoint 138■Implementing the endpoint server 138■Implement the endpoint client 140Deploying. Bigtable’s invention 9■Capturing incremental data 10■Content serving 13■Information exchange 141.3 Hello HBase 15Quick install 16■Interacting with
- Xem thêm -

Xem thêm: Tài liệu HBase in Action docx, Tài liệu HBase in Action docx, Tài liệu HBase in Action docx, 5 Asynchbase: an alternative Java HBase client

Mục lục

Xem thêm

Gợi ý tài liệu liên quan cho bạn