Freeman .NET 4 Parallel Programming in C# Companion eBook Available Pro this print for content only—size & color not accurate   CYAN   MAGENTA   YELLOW   BLACK   PANTONE 123 C BOOKS FOR PROFESSIONALS BY PROFESSIONALS ® Adam Freeman, Author of Pro ASP.NET 4 in C# 2010 Pro LINQ: Language Integrated Query in C# 2010 Visual C# 2010 Recipes Programming .NET Security Microsoft .NET XML Web Services Step by Step C# for Java Developers Programming the Internet with Java Active Java Shelve in: Programming Languages/C# User level: Intermediate–Advanced THE APRESS ROADMAP Pro C# 2010 and the .NET 4 Platform Pro LINQ in C# 2010 Introducing .NET 4.0 Accelerated C# 2010 Pro .NET 4 Parallel Programming in C# Pro Dynamic .NET 4.0 Applications www.apress.com SOURCE CODE ONLINE Companion eBook See last page for details on $10 eBook version ISBN 978-1-4302-2967-4 9 781430 229674 5 59 9 9 Pro .NET 4 Parallel Programming in C# Dear Reader, Normal programs perform one task at a time. Parallel programs perform several tasks simultaneously, improving performance, scalability, and responsiveness. By writing parallel programs, your projects can take complete advantage of the latent power that multi-core and multi-processor computers have to offer. This book shows you how to get things done. I focus on the practice, rather than the theory and show you how the technology works using complete code examples to illustrate my points. Each chapter not only explains the principals of parallel programming but also contains a list of common pitfalls together with details of how to recognize them, and the steps you can take to prevent them happening to you. This book is an invaluable companion when tackling a wide range of parallel programming features and techniques including: • Using the .NET 4 Task Parallel Library (TPL) • Using synchronization to share data between tasks • Coordinating parallel execution • Using parallel loops • Using Parallel LINQ • Testing and debugging parallel programs • Implementing common parallel algorithms Each topic is explained with a complete fully working code example, so you can see, in one place, everything you need to do. Adam Freeman 7.5 x 9.25 spine = 0.75" 328 page count THE EXPERT’S VOICE ®  IN .NET Pro .NET 4 Parallel Programming in C# Adam Freeman Discover how concurrent programming can improve your code www.it-ebooks.info www.it-ebooks.info Pro .NET 4 Parallel Programming in C# ■ ■ ■ Adam Freeman www.it-ebooks.info Pro .NET 4 Parallel Programming in C# Copyright © 2010 by Adam Freeman All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher. ISBN-13 (pbk): 978-1-4302-2967-4 ISBN-13 (electronic): 978-1-4302-2968-1 Printed and bound in the United States of America 9 8 7 6 5 4 3 2 1 Trademarked names may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, we use the names only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. President and Publisher: Paul Manning Lead Editor: Ewan Buckingham Technical Reviewer: André van Meulebrouck Editorial Board: Clay Andres, Steve Anglin, Mark Beckner, Ewan Buckingham, Gary Cornell, Jonathan Gennick, Jonathan Hassell, Michelle Lowman, Matthew Moodie, Duncan Parkes, Jeffrey Pepper, Frank Pohlmann, Douglas Pundick, Ben Renow-Clarke, Dominic Shakeshaft, Matt Wade, Tom Welsh Coordinating Editor: Anne Collett Copy Editor: Heather Lang Production Support: Patrick Cunningham Indexer: BIM Indexing & Proofreading Services Artist: April Milne Cover Designer: Anna Ishchenko Distributed to the book trade worldwide by Springer-Verlag New York, Inc., 233 Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax 201-348-4505, e-mail orders-ny@springer-sbm.com, or visit www.springeronline.com. For information on translations, please e-mail rights@apress.com, or visit www.apress.com. Apress and friends of ED books may be purchased in bulk for academic, corporate, or promotional use. eBook versions and licenses are also available for most titles. For more information, reference our Special Bulk Sales–eBook Licensing web page at www.apress.com/info/bulksales. The information in this book is distributed on an “as is” basis, without warranty. Although every precaution has been taken in the preparation of this work, neither the author(s) nor Apress shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the information contained in this work. The source code for this book is available to readers at www.apress.com. You will need to answer questions pertaining to this book in order to successfully download the code. www.it-ebooks.info For my wife Jacqui Griffyth and her chickens www.it-ebooks.info iv Contents at a Glance About the Author xiii About the Technical Reviewer xiv Acknowledgments xv ■Chapter 1: Introducing Parallel Programming 1 ■Chapter 2: Task Programming 7 ■Chapter 3: Sharing Data 49 ■Chapter 4: Coordinating Tasks 109 ■Chapter 5: Parallel Loops 173 ■Chapter 6: Parallel LINQ 219 ■Chapter 7: Testing and Debugging 251 ■Chapter 8: Common Parallel Algorithms 271 Index 295 www.it-ebooks.info v Contents About the Author xiii About the Technical Reviewer xiv Acknowledgments xv ■Chapter 1: Introducing Parallel Programming 1 Introducing .NET Parallel Programming 1 What’s in This Book (and What Is Not) 2 Understanding the Benefits (and Pitfalls) of Parallel Programming 3 Considering Overhead 3 Coordinating Data 3 Scaling Applications 3 Deciding When to Go Parallel 3 Deciding When to Stay Sequential 4 Getting Prepared for This Book 4 Understanding the Structure of This Book 4 Getting the Example Code 5 Summary 6 www.it-ebooks.info ■ CONTENTS vi ■Chapter 2: Task Programming 7 Hello Task 7 Creating and Starting Tasks 8 Creating Simple Tasks 9 Setting Task State 11 Getting a Result 13 Specifying Task Creation Options 15 Identifying Tasks 15 Cancelling Tasks 15 Monitoring Cancellation by Polling 17 Monitoring Cancellation with a Delegate 19 Monitoring Cancellation with a Wait Handle 20 Cancelling Several Tasks 22 Creating a Composite Cancellation Token 23 Determining If a Task Was Cancelled 24 Waiting for Time to Pass 25 Using a Cancellation Token Wait Handle 26 Using Classic Sleep 27 Using Spin Waiting 29 Waiting for Tasks 30 Waiting for a Single Task 31 Waiting for Several Tasks 33 Waiting for One of Many Tasks 34 Handling Exceptions in Tasks 35 Handling Basic Exceptions 36 Using an Iterative Handler 37 Reading the Task Properties 39 Using a Custom Escalation Policy 41 www.it-ebooks.info ■ CONTENTS vii Getting the Status of a Task 43 Executing Tasks Lazily 43 Understanding Common Problems and Their Causes 45 Task Dependency Deadlock 45 Local Variable Evaluation 46 Excessive Spinning 47 Summary 48 ■Chapter 3: Sharing Data 49 The Trouble with Data 50 Going to the Races 50 Creating Some Order 51 Executing Sequentially 52 Executing Immutably 52 Executing in Isolation 53 Synchronizing Execution 59 Defining Critical Regions 59 Defining Synchronization Primitives 59 Using Synchronization Wisely 60 Using Basic Synchronization Primitives 61 Locking and Monitoring 62 Using Interlocked Operations 67 Using Spin Locking 70 Using Wait Handles and the Mutex Class 72 Configuring Interprocess Synchronization 76 Using Declarative Synchronization 78 Using Reader-Writer Locks 79 www.it-ebooks.info ■ CONTENTS viii Working with Concurrent Collections 87 Using .NET 4 Concurrent Collection Classes 88 Using First-Generation Collections 97 Using Generic Collections 99 Common Problems and Their Causes 100 Unexpected Mutability 100 Multiple Locks 101 Lock Acquisition Order 103 Orphaned Locks 105 Summary 107 ■Chapter 4: Coordinating Tasks 109 Doing More with Tasks 110 Using Task Continuations 110 Creating Simple Continuations 111 Creating One-to-Many Continuations 113 Creating Selective Continuations 115 Creating Many-to-One and Any-To-One Continuations 117 Canceling Continuations 120 Waiting for Continuations 122 Handling Exceptions 122 Creating Child Tasks 126 Using Synchronization to Coordinate Tasks 129 Barrier 131 CountDownEvent 136 ManualResetEventSlim 139 AutoResetEvent 141 SemaphoreSlim 143 www.it-ebooks.info [...]... versus parallel code Languages have evolved to make the programmer’s life easier for writing regular programs, but little has changed for parallel programming until now, of course Microsoft has added features to C#, the NET Framework, and Visual Studio 2010 that take a big step toward pairing a modern programming language with a modern approach to parallel programming Introducing NET Parallel Programming. .. Parallel programming adds some unique problems to debugging, but the new Visual Studio 2010 parallel debugger features go a long way to addressing them 4 www.it-ebooks.info CHAPTER 1 ■ INTRODUCING PARALLEL PROGRAMMING The final chapter, Chapter 8, contains some sample implementations of common parallel algorithms In many cases, especially when you are starting with parallel programming, you will find that... 236 Forcing Sequential Execution 237 Handling PLINQ Exceptions 238 Cancelling PLINQ Queries 239 Setting Merge Options 240 Using Custom Partitioning 242 Using Custom Aggregation 245 Generating Parallel Ranges 246 x www.it-ebooks.info ■ CONTENTS Common Problems and Their Causes 247 Forgetting the PLINQ Basics... long-term interest in all things parallel xiii www.it-ebooks.info About the Technical Reviewer ■André van Meulebrouck has an interest in functional programming and the functional approach to parallel computing He has written white papers and articles on functional programming and theoretical computer science and is a beta tester for F#, which is Microsoft’s new functional programming language He lives in. .. Listing 2 -4 Creating Several Tasks Using Task State using System; using System.Threading.Tasks; namespace Listing_ 04 { class Listing_ 04 { static void Main(string[] args) { string[] messages = { "First task", "Second task", "Third task", "Fourth task" }; foreach (string msg in messages) { Task myTask = new Task(obj => printMessage((string)obj), msg); myTask.Start(); } // wait for input before exiting... www.it-ebooks.info www.it-ebooks.info CHAPTER 1 ■■■ Introducing Parallel Programming When I started programming in the mid-1990s, Java was the hot new language One of the most talkedabout features was its support for parallel programming the ability for an application to do more than one task simultaneously I was very excited; I worked in a research lab, and I finally had a way to use the four CPUs in the... whole point of the TPL is to abstract away from the details, I am comfortable leaving that material to other authors 2 www.it-ebooks.info CHAPTER 1 ■ INTRODUCING PARALLEL PROGRAMMING Understanding the Benefits (and Pitfalls) of Parallel Programming Parallel computing is, at heart, a performance play The work that a program performs is broken up into pieces, which are performed by multiple cores, processors,... Basics 247 Creating Race Conditions 248 Confusing Ordering 248 Sequential Filtering 249 Summary 250 ■Chapter 7: Testing and Debugging 251 Making Things Better When Everything Goes Wrong 251 Measuring Parallel Performance 252 Using Good Coding Strategies 252 Making Simple Performance... This book is about the parallel programming features of NET 4, specifically the Task Parallel Library (TPL), Parallel LINQ, and the legion of support classes that make writing parallel programs with C# simpler and easier than ever before I have been writing parallel programs on and off since I had that overheated office, about 15 years in all I can honestly say that the TPL is the single most impressive,... mainstream programmers and do much to drive up the utilization of all of those multicore machines out there 6 www.it-ebooks.info CHAPTER 2 ■■■ Task Programming Listing 2-1 Hello Task using System; using System.Threading.Tasks; namespace Listing_01 { class Listing_01 { static void Main(string[] args) { Task.Factory.StartNew(() => { Console.WriteLine("Hello World"); }); // wait for input before exiting . pairing a modern programming language with a modern approach to parallel programming. Introducing .NET Parallel Programming This book is about the parallel. 978-1 -43 02-2967 -4 9 78 143 0 2296 74 5 59 9 9 Pro .NET 4 Parallel Programming in C# Dear Reader, Normal programs perform one task at a time. Parallel programs