Foreword by Holly Rushmeier Francesco Banterle Alessandro Artusi Kurt Debattista Alan Chalmers Advanced High Dynamic Range Imaging High dynamic range (HDR) imaging is the term given to the capture, storage, manipu- lation, transmission, and display of images that more accurately represent the wide range of real-world lighting levels. With the advent of a true HDR video system and its 20 year history of creating static images, HDR is finally ready to enter the “mainstream” of imaging technology. This book provides a comprehensive practical guide to facilitate the widespread adoption of HDR technology. By examining the key problems associ- ated with HDR imaging and providing detailed methods to overcome these problems, the authors hope readers will be inspired to adopt HDR as their preferred approach for imaging the real world. Key HDR algorithms are provided as MATLAB code as part of the HDR Toolbox. “This book provides a practical introduction to the emerging new discipline of high dynamic range imaging that combines photography and computer graphics. . . By providing detailed equations and code, the book gives the reader the tools needed to experiment with new techniques for creating compelling images.” —From the Foreword by Holly Rushmeier, Yale University Download MATLAB source code for the book at www.advancedhdrbook.com Advanced High Dynamic Range Imaging Francesco Banterle • Alessandro Artusi Kurt Debattista • Alan Chalmers Foreword by Holly Rushmeier Advanced High Dynamic Range Imaging This page intentionally left blankThis page intentionally left blank Advanced High Dynamic Range Imaging Theory and Practice Francesco Banterle Alessandro Artusi Kurt Debattista Alan Chalmers A K Peters, Ltd. Natick, Massachusetts CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2011 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S. Government works Version Date: 20120202 International Standard Book Number-13: 978-1-4398-6594-1 (eBook - PDF) This book contains information obtained from authentic and highly regarded sources. Reason- able efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www. copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organiza- tion that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com To my parents. —FB Dedicated to all of you: Franca, Nella, Sincero, Marco, Giancarlo, and Despo. You are a lways in my mind. —AA To Alex. Welcome! —KD To Eva, Erika, Andrea, and Thomas. You are my reality! —AC This page intentionally left blankThis page intentionally left blank Contents 1Introduction 1 1.1 Light, Human Vision, and Color Spaces . . . . . . . . . . 4 2 HDR Pipeline 11 2.1 HDRContentGeneration 12 2.2 HDRContentStoring 22 2.3 VisualizationofHDRContent 26 3 Tone Mapping 33 3.1 TMOMATLABFramework 36 3.2 GlobalOperators 38 3.3 LocalOperators 61 3.4 Frequency-BasedOperators 75 3.5 SegmentationOperators 86 3.6 New Trends to the Tone Mapping Problem . . . . . . . . 103 3.7 Summary 112 4 Expansion Operators for Low Dynamic Range Content 113 4.1 Linearization of the Signal Using a Single Image . . . . . 115 4.2 Decontouring Models for High Contrast Displays . . . . . 119 4.3 EOMATLABFramework 121 4.4 GlobalModels 122 4.5 ClassificationModels 128 4.6 ExpandMapModels 134 4.7 User-Based Models: HDR Hallucination . . . . . . . . . . 144 4.8 Summary 145 vii viii CONTENTS 5 Image-Based Lighting 149 5.1 EnvironmentMap 149 5.2 RenderingwithIBL 155 5.3 Summary 174 6 Evaluation 175 6.1 PsychophysicalExperiments 175 6.2 ErrorMetric 187 6.3 Summary 190 7 HDR Content Compression 193 7.1 HDR Compression MATLAB Framework . . . . . . . . . 193 7.2 HDRImageCompression 194 7.3 HDRTextureCompression 205 7.4 HDRVideoCompression 218 7.5 Summary 225 A The Bilateral Filter 227 B Retinex Filters 231 C A Brief Overview of the MATLAB HDR Toolbox 233 Bibliography 239 Index 258 Foreword We perceive the world through the scattering of light from objects to our eyes. Imaging techniques seek to simulate the array of light that reaches our eyes to provide the illusion of sensing scenes directly. Both photography and computer graphics deal with the generation of images. Both disciplines have to cope with the high dynamic range in the energy of visible light that human eyes can sense. Traditionally photography and computer graphics took different approaches to the high dynamic range problem. Work over the last ten years, though, has unified these disciplines and created powerful new tools for the creation of complex, compelling, and realistic images. This book provides a practical introduction to the emerging new discipline of high dynamic range imaging that combines photography and computer graphics. Historically, traditional wet photography managed the recording of high dynamic range imagery through careful design of camera optics and the material layers that form film. The ingenious processes that were invented enabled the recording of images that appeared identical to real-life scenes. Further, traditional photography facilitated artistic adjustments by the photographer in the darkroom during the development process. However, the complex relationship between the light incident on the film and the chemistry of the material layers that form the image made wet photogra- phy unsuitable for light measurement. The early days of computer graphics also used ingenious methods to work around two physical constraints—inadequate computational capabil- ities for simulating light transport and display devices with limited dynamic range. To address the limited computational capabilities, simple heuristics such as Phong reflectance were developed to mimic the final appearance of objects. By designing heuristics appropriately, images were computed that always fit the narrow display range. It wasn’t until the early 1980s ix [...]... many different contexts, such as dynamic scenes and low-light conditions There is a great deal of low dynamic range imagery that has been captured and generated in the past; how can this be expanded to be used in the same context as high dynamic range imagery? What compression techniques can be employed to deal with the increased data generated by high dynamic range imaging systems? How can we best... while users were satisfied to scale and clip computed values in somewhat arbitrary ways to map the high dynamic range of computed imagery to the low dynamic range cathode ray tube devices in use at the Foreword xi time Jack Tumblin, an engineer who had been working on the problem of presenting high dynamic range images in flight simulators, ran across the work in computer graphics lighting simulation and... technology By examining the key problems associated with HDR imaging and providing detailed methods to overcome these problems, together with supporting Matlab code, we hope readers will be inspired to adopt HDR as their preferred approach for imaging the real world xiii xiv Preface Advanced High Dynamic Range Imaging covers all aspects of HDR imaging from capture to display, including an evaluation of... University of Warwick, UK High dynamic range (HDR) imaging is the term given to the capture, storage, manipulation, transmission, and display of images that more accurately represent the wide range of real-world lighting levels With the advent of a true HDR video system, and from the experience of more than 20 years of static HDR imagery, HDR is finally ready to enter the “mainstream” of imaging technology... levels A high- quality xenon film projector may get a few times brighter than this, but it is still two orders of magnitude away from the optimal light level for human acuity and color perception This is now all changing with high dynamic range (HDR) imagery and novel capture and display HDR technologies, offering a step-change in traditional imaging approaches In the last two decades, HDR imaging has... been in generating HDR images from legacy LDR content Capturing real-world lighting generates a large amount of data The HDR video camera shown at SIGGRAPH requires 24 MB per frame, which equates to almost 42 GB for a minute of footage (compared with just 9 GB for a minute of LDR video) The final chapter of Advanced High Dynamic Range Imaging examines the issues of compressing HDR imagery to enable it... Foreword that computational capability had increased to the point that full lighting simulations were possible, at least on simple scenes I had my own first experience with the yet-unnamed field of high dynamic range imaging in the mid-1980s I was studying one particular approach to lighting simulation—radiosity I was part of a team that designed experiments to demonstrate that the lengthy computation required... contain slow-moving objects; otherwise artifacts will appear The method is thus not well suited for real-world situations Nayar and Branzoi [153] developed an adaptive dynamic range camera where a controllable liquid crystal light modulator is placed in front of the camera This modulator adapts the exposure of each pixel on the image detector, allowing the capture of scenes with a very large dynamic range. .. manner It is for this reason we have chosen to include the key HDR algorithms as Matlab code as part of what we term the HDR Toolbox An overview of the HDR Toolbox is given in Appendix C In Advanced High Dynamic Range Imaging, the common parts of Matlab code are presented at the beginning of each chapter The remaining code for each technique is then presented at the point in the chapter where the technique... simultaneously capture the full range of lighting that the eye can see at any level of adaptation continues to be a major challenge The latest step towards achieving this “holy grail” of imaging was in 2009 when a video camera capable of capturing 20 f-stops (1920 × 1080 resolution) at 30 frames a second was shown at the annual ACM SIGGRAPH conference by the German high- precision camera manufacturer . www.advancedhdrbook.com Advanced High Dynamic Range Imaging Francesco Banterle • Alessandro Artusi Kurt Debattista • Alan Chalmers Foreword by Holly Rushmeier Advanced High Dynamic Range Imaging This. Banterle Alessandro Artusi Kurt Debattista Alan Chalmers Advanced High Dynamic Range Imaging High dynamic range (HDR) imaging is the term given to the capture, storage, manipu- lation, transmission, and display. 20120202 International Standard Book Number-13: 97 8-1 -4 39 8-6 59 4-1 (eBook - PDF) This book contains information obtained from authentic and highly regarded sources. Reason- able efforts have been made to