Programming in Python A Complete Introduction to the Python Language Second Edition Mark Summerfield Upper Saddle River, NJ · Boston · Indianapolis · San Francisco New York · Toronto · Montreal · London · Munich · Paris · Madrid Capetown · Sydney · Tokyo · Singapore · Mexico City www.allitebooks.com Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks Where those designations appear in this book, and the publisher was aware of a trademark claim, the designations have been printed with initial capital letters or in all capitals The author and publisher have taken care in the preparation of this book, but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions No liability is assumed for incidental or consequential damages in connection with or arising out of the use of the information or programs contained herein The publisher offers excellent discounts on this book when ordered in quantity for bulk purchases or special sales, which may include electronic versions and/or custom covers and content particular to your business, training goals, marketing focus, and branding interests For more information, please contact: U.S Corporate and Government Sales (800) 382-3419 corpsales@pearsontechgroup.com For sales outside the United States, please contact: International Sales international@pearsoned.com Visit us on the Web: informit.com/aw Library of Congress Cataloging-in-Publication Data Summerfield, Mark Programming in Python : a complete introduction to the Python language / Mark Summerfield.—2nd ed p cm Includes bibliographical references and index ISBN 978-0-321-68056-3 (pbk : alk paper) Python (Computer program language) Object-oriented programming (Computer science) I Title QA76.73.P98S86 2010 005.13’3—dc22 2009035430 Copyright © 2010 Pearson Education, Inc All rights reserved Printed in the United States of America This publication is protected by copyright, and permission must be obtained from the publisher prior to any prohibited reproduction, storage in a retrieval system, or transmission in any form or by any means, electronic, mechanical, photocopying, recording, or likewise For information regarding permissions, write to: Pearson Education, Inc Rights and Contracts Department 501 Boylston Street, Suite 900 Boston, MA 02116 Fax: (617) 671-3447 ISBN-13: 978-0-321-68056-3 ISBN-10: 0-321-68056-1 Text printed in the United States on recycled paper at RR Donnelley in Crawfordsville, Indiana First printing, November 2009 www.allitebooks.com ContentsataGlance Listof Tables xv Introduction Chapter1 RapidIntroductiontoProceduralProgramming Chapter2 DataTypes 51 Chapter3 CollectionDataTypes 107 159 Chapter4 ControlStructuresandFunctions Chapter5 Modules 195 Chapter6 Object-OrientedProgramming Chapter7 FileHandling 233 287 Chapter8 AdvancedProgrammingTechniques Chapter9 Debugging,Testing,andProfiling Chapter10 ProcessesandThreading Chapter11 Networking 339 413 439 457 Chapter12 DatabaseProgramming Chapter13 RegularExpressions Chapter14 IntroductiontoParsing 475 489 513 Chapter15 IntroductiontoGUIProgramming 569 Epilogue 595 SelectedBibliography 597 Index 599 www.allitebooks.com Contents List of Tables xv Introduction Chapter Rapid Introduction to Procedural Programming Creating and Running Python Programs Python’s “Beautiful Heart” Piece #1: Data Types Piece #2: Object References Piece #3: Collection Data Types Piece #4: Logical Operations Piece #5: Control Flow Statements Piece #6: Arithmetic Operators Piece #7: Input/Output Piece #8: Creating and Calling Functions Examples bigdigits.py generate_grid.py Summary Exercises 9 14 14 16 18 21 26 30 33 36 39 39 42 44 47 Chapter Data Types Identifiers and Keywords Integral Types Integers Booleans Floating-Point Types Floating-Point Numbers Complex Numbers Decimal Numbers Strings Comparing Strings Slicing and Striding Strings String Operators and Methods 51 51 54 54 58 58 59 62 63 65 68 69 71 ix www.allitebooks.com String Formatting with the str.format() Method Character Encodings Examples quadratic.py csv2html.py Summary Exercises 78 91 94 94 97 102 104 Chapter Collection Data Types Sequence Types Tuples Named Tuples Lists Set Types Sets Frozen Sets Mapping Types Dictionaries Default Dictionaries Ordered Dictionaries Iterating and Copying Collections Iterators and Iterable Operations and Functions Copying Collections Examples generate_usernames.py statistics.py Summary Exercises 107 107 108 111 113 120 121 125 126 126 135 136 138 138 146 148 149 152 156 158 Chapter Control Structures and Functions Control Structures Conditional Branching Looping Exception Handling Catching and Raising Exceptions Custom Exceptions Custom Functions Names and Docstrings Argument and Parameter Unpacking 159 159 159 161 163 163 168 171 176 177 x www.allitebooks.com Accessing Variables in the Global Scope Lambda Functions Assertions Example: make_html_skeleton.py Summary Exercise 180 182 183 185 191 192 Chapter Modules Modules and Packages Packages Custom Modules Overview of Python’s Standard Library String Handling Command-Line Programming Mathematics and Numbers Times and Dates Algorithms and Collection Data Types File Formats, Encodings, and Data Persistence File, Directory, and Process Handling Networking and Internet Programming XML Other Modules Summary Exercise 195 195 199 202 212 213 214 216 216 217 219 222 225 226 228 230 231 Chapter Object-Oriented Programming The Object-Oriented Approach Object-Oriented Concepts and Terminology Custom Classes Attributes and Methods Inheritance and Polymorphism Using Properties to Control Attribute Access Creating Complete Fully Integrated Data Types Custom Collection Classes Creating Classes That Aggregate Collections Creating Collection Classes Using Aggregation Creating Collection Classes Using Inheritance Summary Exercises 233 234 235 238 238 243 246 248 261 261 269 276 283 285 xi www.allitebooks.com Chapter File Handling Writing and Reading Binary Data Pickles with Optional Compression Raw Binary Data with Optional Compression Writing and Parsing Text Files Writing Text Parsing Text Parsing Text Using Regular Expressions Writing and Parsing XML Files Element Trees DOM (Document Object Model) Manually Writing XML Parsing XML with SAX (Simple API for XML) Random Access Binary Files A Generic BinaryRecordFile Class Example: The BikeStock Module’s Classes Summary Exercises 287 292 292 295 305 305 307 310 312 313 316 319 321 324 324 332 336 337 Chapter Advanced Programming Techniques Further Procedural Programming Branching Using Dictionaries Generator Expressions and Functions Dynamic Code Execution and Dynamic Imports Local and Recursive Functions Function and Method Decorators Function Annotations Further Object-Oriented Programming Controlling Attribute Access Functors Context Managers Descriptors Class Decorators Abstract Base Classes Multiple Inheritance Metaclasses Functional-Style Programming Partial Function Application 339 340 340 341 344 351 356 360 363 363 367 369 372 378 380 388 390 395 398 xii www.allitebooks.com Coroutines Example: Valid.py Summary Exercises 399 407 410 411 Chapter Debugging, Testing, and Profiling Debugging Dealing with Syntax Errors Dealing with Runtime Errors Scientific Debugging Unit Testing Profiling Summary 413 414 414 415 420 425 432 437 Chapter 10 Processes and Threading Using the Multiprocessing Module Using the Threading Module Example: A Threaded Find Word Program Example: A Threaded Find Duplicate Files Program Summary Exercises 439 440 444 446 449 454 455 Chapter 11 Networking Creating a TCP Client Creating a TCP Server Summary Exercises 457 458 464 471 471 Chapter 12 Database Programming DBM Databases SQL Databases Summary Exercise 475 476 480 487 488 Chapter 13 Regular Expressions Python’s Regular Expression Language Characters and Character Classes Quantifiers Grouping and Capturing Assertions and Flags The Regular Expression Module 489 490 490 491 494 496 499 xiii www.allitebooks.com Summary Exercises 509 510 Chapter 14 Introduction to Parsing BNF Syntax and Parsing Terminology Writing Handcrafted Parsers Simple Key–Value Data Parsing Playlist Data Parsing Parsing the Blocks Domain-Specific Language Pythonic Parsing with PyParsing A Quick Introduction to PyParsing Simple Key–Value Data Parsing Playlist Data Parsing Parsing the Blocks Domain-Specific Language Parsing First-Order Logic Lex/Yacc-Style Parsing with PLY Simple Key–Value Data Parsing Playlist Data Parsing Parsing the Blocks Domain-Specific Language Parsing First-Order Logic Summary Exercise 513 514 519 519 522 525 534 535 539 541 543 548 553 555 557 559 562 566 568 Chapter 15 Introduction to GUI Programming Dialog-Style Programs Main-Window-Style Programs Creating a Main Window Creating a Custom Dialog Summary Exercises 569 572 578 578 590 593 593 Epilogue 595 Selected Bibliography 597 Index 599 xiv www.allitebooks.com List of Tables 2.1 2.2 Python’s Keywords 52 Numeric Operators and Functions 55 2.3 Integer Conversion Functions 2.4 Integer Bitwise Operators 57 The Math Module’s Functions and Constants #1 60 2.5 2.6 2.7 55 The Math Module’s Functions and Constants #2 61 Python’s String Escapes 66 2.8 String Methods #1 73 2.9 String Methods #2 74 2.10 String Methods #3 75 3.1 3.2 3.3 3.4 6.1 6.2 6.3 6.4 7.1 7.2 7.3 7.4 7.5 8.1 8.2 8.3 8.4 12.1 List Methods 115 Set Methods and Operators 123 Dictionary Methods 129 Common Iterable Operators and Functions 140 Comparison Special Methods 242 Fundamental Special Methods 250 Numeric and Bitwise Special Methods 253 Collection Special Methods 265 Bytes and Bytearray Methods #1 299 Bytes and Bytearray Methods #2 300 Bytes and Bytearray Methods #3 301 File Object Attributes and Methods #1 325 File Object Attributes and Methods #2 326 Dynamic Programming and Introspection Functions 349 Attribute Access Special Methods 365 The Numbers Module’s Abstract Base Classes 381 The Collections Module’s Main Abstract Base Classes 383 DB-API 2.0 Connection Object Methods 481 12.2 DB-API 2.0 Cursor Object Attributes and Methods 482 13.1 Character Class Shorthands 492 xv www.allitebooks.com 616 Index locale module, 86 setlocale(), 86, 87 localization, 86 locals() (built-in), 81, 82, 97, 154, 188, 189, 190, 345, 349, 422, 423, 484 localtime() (time module), 217 Lock type (threading module), 452, 467 log() (math module), 60 log10() (math module), 60 log1p() (math module), 60 logging module, 229, 360 logic, short-circuit, 25, 58 logical operators; see and, or, and not LookupError (exception), 164 looping, see for loop and while loop, 161 lower() bytearray type, 300 bytes type, 300 str type, 74, 76 lshift () (), 242, 259, 379 hash (), 250, 254 iadd () (+=), 253 iand () (&=), 251, 253, 257 ifloordiv () (//=), 253 ilshift () (=), 253 isub () (-=), 253 iter (), 265, 274, 281, 335 ixor () (^=), 253 le () (