NOTE Part 2 available Fall 2012 See Table of Contents inside Part 2 Windows ® Internals Russinovich Solomon Ionescu Operating Systems/ Windows ISBN: 978-0-7356-4873-9 About the Authors Mark Russinovich is a Technical Fellow in the Windows Azure ™ group at Microsoft. He is coauthor of Windows Sysinternals Administrator’s Reference, co-creator of the Sysinternals tools available from Microsoft TechNet, and coauthor of the Windows Internals book series. David A. Solomon is coauthor of the Windows Internals book series and has taught his Windows internals class to thousands of developers and IT professionals worldwide, including Microsoft staff. He is a regular speaker at Microsoft conferences, including TechNet and PDC. Alex Ionescu is a chief software architect and consultant expert in low-level system software, kernel development, security training, and reverse engineering. He teaches Windows internals courses with David Solomon, and is active in the security research community. The denitive guide—fully updated for Windows 7 and Windows Server 2008 R2 Delve inside Windows architecture and internals—and see how core components work behind the scenes. Led by a team of internationally renowned internals experts, this classic guide has been fully updated for Windows 7 and Windows Server® 2008 R2—and now presents its coverage in two volumes. As always, you get critical, insider perspectives on how Windows operates. And through hands-on experiments, you’ll experience its internal behavior rsthand—knowledge you can apply to improve application design, debugging, system performance, and support. In Part 2, you will: • Understand how core system and management mechanisms work—including object manager, synchronization, Wow64, Hyper-V ® , and the registry • Examine the data structures and activities behind processes, threads, and jobs • Go inside the Windows security model to see how it manages access, auditing, and authorization • Explore the Windows networking stack from top to bottom— including APIs, BranchCache, protocol and NDIS drivers, and layered services • Dig into internals hands-on using the kernel debugger, performance monitor, and other tools Windows ® Internals PART 2 microsoft.com/mspress U.S.A. $39.99 Canada $41.99 [Recommended] See inside cover DEVELOPER ROADMAP Step by Step • For experienced developers learning a new topic • Focus on fundamental techniques and tools • Hands-on tutorial with practice les plus eBook Start Here! • Beginner-level instruction • Easy to follow explanations and examples • Exercises to build your rst projects Developer Reference • Professional developers; intermediate to advanced • Expertly covers essential topics and techniques • Features extensive, adaptable code examples SIXTH EDITION 6 SIXTH EDITION Focused Topics • For programmers who develop complex or advanced solutions • Specialized topics; narrow focus; deep coverage • Features extensive, adaptable code examples Windows ® Internals Part 2 6 SIXTH EDITION Mark Russinovich David A. Solomon Alex Ionescu spine = 1.2” Cyan Magenta Yellow Black PUBLISHED BY Microsoft Press A Division of Microsoft Corporation One Microsoft Way Redmond, Washington 98052-6399 Copyright © 2012 by David Solomon and Mark Russinovich All rights reserved. No part of the contents of this book may be reproduced or transmitted in any form or by any means without the written permission of the publisher. Library of Congress Control Number: 2012933511 ISBN: 978-0-7356-6587-3 Printed and bound in the United States of America. First Printing Microsoft Press books are available through booksellers and distributors worldwide. If you need support related to this book, email Microsoft Press Book Support at mspinput@microsoft.com. Please tell us what you think of this book at http://www.microsoft.com/learning/booksurvey. Microsoft and the trademarks listed at http://www.microsoft.com/about/legal/en/us/IntellectualProperty/ Trademarks/EN-US.aspx are trademarks of the Microsoft group of companies. All other marks are property of their respective owners. The example companies, organizations, products, domain names, email addresses, logos, people, places, and events depicted herein are ctitious. No association with any real company, organization, product, domain name, email address, logo, person, place, or event is intended or should be inferred. This book expresses the authors’ views and opinions. The information contained in this book is provided without any express, statutory, or implied warranties. Neither the authors, Microsoft Corporation, nor its resellers, or distributors will be held liable for any damages caused or alleged to be caused either directly or indirectly by this book. Acquisitions Editor: Devon Musgrave Developmental Editor: Devon Musgrave Project Editor: Carol Dillingham Editorial Production: Curtis Philips Technical Reviewer: Christophe Nasarre; Technical Review services provided by Content Master, a member of CM Group, Ltd. Copyeditor: John Pierce Indexer: Jan Wright Cover: Twist Creative • Seattle To our parents, who guided and inspired us to follow our dreams Contents at a Glance Windows Internals, Sixth Edition, Part 1 (available separately) CHAPTER 1 Concepts and Tools CHAPTER 2 System Architecture CHAPTER 3 System Mechanisms CHAPTER 4 Management Mechanisms CHAPTER 5 Processes, Threads, and Jobs CHAPTER 6 Security CHAPTER 7 Networking Windows Internals, Sixth Edition, Part 2 CHAPTER 8 I/O System 1 CHAPTER 9 Storage Management 125 CHAPTER 10 Memory Management 187 CHAPTER 11 Cache Manager 355 CHAPTER 12 File Systems 391 CHAPTER 13 Startup and Shutdown 499 CHAPTER 14 Crash Dump Analysis 547 vii Contents Windows Internals, Sixth Edition, Part 1 (See appendix for Part 1’s table of contents) Windows Internals, Sixth Edition, Part 2 Introduction xv Chapter 8 I/O System 1 I/O System Components 1 The I/O Manager 3 Typical I/O Processing 4 Device Drivers 5 Types of Device Drivers 5 Structure of a Driver 12 Driver Objects and Device Objects 14 Opening Devices 19 I/O Processing 25 Types of I/O 25 I/O Request to a Single-Layered Driver 33 I/O Requests to Layered Drivers 40 I/O Cancellation 48 I/O Completion Ports 53 I/O Prioritization 58 Container Notications 65 Driver Verier 65 Kernel-Mode Driver Framework (KMDF) 68 Structure and Operation of a KMDF Driver 68 KMDF Data Model 70 KMDF I/O Model 74 What do you think of this book? We want to hear from you! Microsoft is interested in hearing your feedback so we can continually improve our books and learning resources for you. To participate in a brief online survey, please visit: microsoft.com/learning/booksurvey viii Contents User-Mode Driver Framework (UMDF) 78 The Plug and Play (PnP) Manager 81 Level of Plug and Play Support 82 Driver Support for Plug and Play 82 Driver Loading, Initialization, and Installation 84 Driver Installation 94 The Power Manager 98 Power Manager Operation 100 Driver Power Operation 101 Driver and Application Control of Device Power 105 Power Availability Requests 105 Processor Power Management (PPM) 108 Conclusion 123 Chapter 9 Storage Management 125 Storage Terminology 125 Disk Devices 126 Rotating Magnetic Disks 126 Solid State Disks 128 Disk Drivers 131 Winload 132 Disk Class, Port, and Miniport Drivers 132 Disk Device Objects 136 Partition Manager 138 Volume Management 138 Basic Disks 139 Dynamic Disks 141 Multipartition Volume Management 147 The Volume Namespace 153 Volume I/O Operations 159 Virtual Disk Service 160 Virtual Hard Disk Support 162 Attaching VHDs 163 Nested File Systems 163 BitLocker Drive Encryption 163 Encryption Keys 165 Trusted Platform Module (TPM) 168 BitLocker Boot Process 170 BitLocker Key Recovery 172 Contents ix Full-Volume Encryption Driver 173 BitLocker Management 174 BitLocker To Go 175 Volume Shadow Copy Service 177 Shadow Copies 177 VSS Architecture 177 VSS Operation 178 Uses in Windows 181 Conclusion 186 Chapter 10 Memory Management 187 Introduction to the Memory Manager 187 Memory Manager Components 188 Internal Synchronization 189 Examining Memory Usage 190 Services Provided by the Memory Manager 193 Large and Small Pages 193 Reserving and Committing Pages 195 Commit Limit 199 Locking Memory 199 Allocation Granularity 199 Shared Memory and Mapped Files 200 Protecting Memory 203 No Execute Page Protection 204 Copy-on-Write 209 Address Windowing Extensions 210 Kernel-Mode Heaps (System Memory Pools) 212 Pool Sizes 213 Monitoring Pool Usage 215 Look-Aside Lists 219 Heap Manager 220 Types of Heaps 221 Heap Manager Structure 222 Heap Synchronization 223 The Low Fragmentation Heap 223 Heap Security Features 224 Heap Debugging Features 225 Pageheap 226 Fault Tolerant Heap 227 x Contents Virtual Address Space Layouts 228 x86 Address Space Layouts 229 x86 System Address Space Layout 232 x86 Session Space 233 System Page Table Entries 235 64-Bit Address Space Layouts 237 x64 Virtual Addressing Limitations 240 Dynamic System Virtual Address Space Management 242 System Virtual Address Space Quotas 245 User Address Space Layout 246 Address Translation 251 x86 Virtual Address Translation 252 Translation Look-Aside Buffer 259 Physical Address Extension (PAE) 260 x64 Virtual Address Translation 265 IA64 Virtual Address Translation 266 Page Fault Handling 267 Invalid PTEs 268 Prototype PTEs 269 In-Paging I/O 271 Collided Page Faults 272 Clustered Page Faults 272 Page Files 273 Commit Charge and the System Commit Limit 275 Commit Charge and Page File Size 278 Stacks 279 User Stacks 280 Kernel Stacks 281 DPC Stack 282 Virtual Address Descriptors 282 Process VADs 283 Rotate VADs 284 NUMA 285 Section Objects 286 Driver Verier 292 Page Frame Number Database 297 Page List Dynamics 300 Page Priority 310 Modied Page Writer 314 [...]... Services Windows Internals, Fourth Edition was the Windows XP and Windows Server 20 03 update and added more content focused on helping IT professionals make use of their knowledge of Windows internals, such as u ­ sing key tools from Windows Sysinternals (www.microsoft.com/technet/sysinternals) and analyzing crash dumps Windows Internals, Fifth Edition was the update for Windows Vista and Windows Server 20 08... technical depth Inside Windows 20 00, Third Edition (Microsoft Press, 20 00) was authored by David Solomon and Mark Russinovich It added many new topics, such as startup and shutdown, service internals, registry internals, file-system drivers, and networking It also covered kernel changes in Windows 20 00, such as the Windows Driver Model (WDM), Plug and Play, power management, Windows Management Instrumentation... facility, and Hyper-V Sixth Edition Changes This latest edition has been updated to cover the kernel changes made in Windows 7 and Windows Server 20 08 R2 Hands-on experiments have been updated to reflect changes in tools Hands-on Experiments Even without access to the Windows source code, you can glean much about Windows internals from tools such as the kernel debugger and tools from Sysinternals and W ­ insider... lkd> lm kv  start    end        module name 820 07000  823 c0000   nt         (pdb symbols)       c:\programming\symbols\ntkrpamp.pdb\37D 328 E3BAE5460F8E6 627 56ED80951D2\ntkrpamp.pdb     Loaded symbol image file: ntkrpamp.exe     Image path: ntkrpamp.exe     Image name: ntkrpamp.exe     Timestamp:        Fri Jan 18 21 :30:58 20 08 (47918B 12)     CheckSum:         003 720 38     ImageSize:        003B9000     File version:     6.0.6001.18000...     LegalCopyright:   © Microsoft Corporation. All rights reserved 823 c0000  823 f3000   hal        (deferred)                  Image path: halmacpi.dll     Image name: halmacpi.dll     Timestamp:        Fri Jan 18 21 :27 :20 20 08 (47918A38)     CheckSum:         0003859F     ImageSize:        00033000     Translations:     0000.04b0 0000.04e0 0409.04b0 0409.04e0 826 00000  826 71000   ksecdd     (deferred)                  Image path: \SystemRoot\System 32\ Drivers\ksecdd.sys... here: You can also view the list of loaded kernel-mode drivers with Process Explorer from Windows Sysinternals (http://www.microsoft.com/technet/sysinternals) Run Process Explorer, select the System process, and select DLLs from the Lower Pane View menu entry in the View menu: 10 Windows Internals, Sixth Edition, Part 2 Process Explorer lists the loaded drivers, their names, version information (including... Inside Windows NT (Microsoft Press, 19 92) , written by Helen Custer (prior to the initial release of Microsoft Windows NT 3.1) Inside Windows NT was the first book ever published about Windows NT and provided key insights into the architecture and design of the system Inside Windows NT, Second Edition (Microsoft Press, 1998) was written by David Solomon It updated the original book to cover Windows. .. 320 Windows Client Memory Limits 321 Working Sets 324 Demand Paging 324 Logical Prefetcher 324 Placement Policy 328 Working Set Management 329 Balance Set... Appendix: Contents of Windows Internals, Sixth Edition, Part 1 595 Index 603 What do you think of this book? We want to hear from you! Microsoft is interested in hearing your feedback so we can continually improve our books and learning resources for you To participate in a brief online survey, please visit: microsoft.com/learning/booksurvey xiv Contents Introduction W indows Internals, Sixth Edition is intended... and configuration settings (See “The Registry” section in Chapter 4 in Part 1 for more information.) INF files, which are designated by the inf extension, are driver installation files INF files are the link between a particular hardware device and the driver that assumes primary control of Windows Internals, Sixth Edition, Part 2 the device They are made up of script-like instructions describing the . Lists 21 9 Heap Manager 22 0 Types of Heaps 22 1 Heap Manager Structure 22 2 Heap Synchronization 22 3 The Low Fragmentation Heap 22 3 Heap Security Features 22 4 Heap Debugging Features 22 5 Pageheap 22 6 Fault. Stacks 28 0 Kernel Stacks 28 1 DPC Stack 28 2 Virtual Address Descriptors 28 2 Process VADs 28 3 Rotate VADs 28 4 NUMA 28 5 Section Objects 28 6 Driver Verier 29 2 Page Frame Number Database 29 7 Page. Tolerant Heap 22 7 x Contents Virtual Address Space Layouts 22 8 x86 Address Space Layouts 22 9 x86 System Address Space Layout 23 2 x86 Session Space 23 3 System Page Table Entries 23 5 64-Bit Address