ALBERT TARANTOLA ALBERT TARANTOLA to be published by to be published by Probability and Measurements 1 Albert Tarantola Universit´edeParis, Institut de Physique du Globe 4, place Jussieu; 75005 Paris; France E-mail: Albert.Tarantola@ipgp.jussieu.fr December 3, 2001 1 c  A. Tarantola, 2001. ii iii To the memory of my father. To my mother and my wife. iv v Preface In this book, I attempt to reach two goals. The first is purely mathematical: to clarify some of the basic concepts of probability theory. The second goal is physical: to clarify the methods to be used when handling the information brought by measurements, in order to understand how accurate are the predictions we may wish to make. Probability theory is solidly based on Kolmogorov axioms, and there is no problem when treating discrete probabilities. But I am very unhappy with the usual way of extending the theory to continuous probability distributions. In this text, I introduce the notion of ‘volumetric probability’ different from the more usual notion of ‘probability density’. I claim that some of the more basic problems of the theory of continuous probability distributions can only ne solved within this framework, and that many of the well known ‘paradoxes’ of the theory are fundamental misunderstandings, that I try to clarify. I start the book with an introduction to tensor calculus, because I choose to develop the probability theory considering metric manifolds. The second chapter deals with the probability theory per se. I try to use intrinsic notions everywhere, i.e., I only introduce definitions that make sense irrespectively of the particular coordinates being used in the manifold under investigation. The reader shall see that this leads to many develoments that are at odds with those found in usual texts. In physical applications one not only needs to define probability distributions over (typically) large-dimensional manifolds. One also needs to make use of them, and this is achieved by sampling the probability distributions using the ‘Monte Carlo’ methods described in chapter 3. There is no major discovery exposed in this chapter, but I make the effort to set Monte Carlo methods using the intrinsic point of view mentioned above. The metric foundation used here allows to introduce the important notion of ‘homogeneous’ probability distributions. Contrary to the ‘noninformative’ probability distributions common in the Bayesian literature, the homogeneity notion is not controversial (provided one has agreed ona given metric over the space of interest). After a brief chapter that explain what an ideal measuring instrument should be, the book enters in the four chapter developing what I see as the four more basic inference problems in physics: (i) problems that are solved using the notion of ‘sum of probabilities’ (just an elaborate way of ‘making histograms), (ii) problems that are solved using the ‘product of probabilities’ (and approach that seems to be original), (iii) problems that are solved using ‘conditional probabilities’ (these including the so-called ‘inverse problems’), and (iv) problems that are solved using the ‘transport of probabilities’ (like the typical [indirect] mesurement problem, but solved here transporting probability distributions, rather than just transporting ‘uncertainties). Iamvery indebted to my colleagues (Bartolom´e Coll, Georges Jobert, Klaus Mosegaard, Miguel Bosch, Guillaume ´ Evrard, John Scales, Christophe Barnes, Fr´ed´eric Parrenin and Bernard Valette) for illuminating discussions. I am also grateful to my collaborators at what was the Tomography Group at the Institut de Physique du Globe de Paris. Paris, December 3, 2001 Albert Tarantola vi Contents 1Introduction to Tensors 1 2 Elements of Probability 69 3 Monte Carlo Sampling Methods 153 4 Homogeneous Probability Distributions 169 5 Basic Measurements 185 6 Inference Problems of the First Kind (Sum of Probabilities) 207 7 Inference Problems of the Second Kind (Product of Probabilities) 211 8 Inference Problems of the Third Kind (Conditional Probabilities) 219 9 Inference Problems of the Fourth Kind (Transport of Probabilities) 287 vii viii [...]... Conditional Probability 2.4.1 Notion of Conditional Probability 2.4.2 Conditional Volumetric Probability 2.5 Marginal Probability 2.5.1 Marginal Probability Density 2.5.2 Marginal Volumetric Probability 2.5.3 Interpretation of Marginal Volumetric Probability 2.5.4 Bayes Theorem 2.5.5 Independent Probability. .. introduce a Levi-Civita capacity εijk , or a Levi-Civita density (the components of both take only the values -1 , +1 or 0) A Levi-Civita pure tensor can be defined, but it does not have that simple property The lack of clear understanding of the need to work simultaneously with densities, pure tensors, and capacities, forces some authors to juggle with “pseudo-things” like the pseudo-vector corresponding... “3-D vector space” may describe the combination of forces being applied to a particle, as well as the combination of colors The same holds for the mathematical structure “differential manifold” It may describe the 3-D physical space, any 2-D surface, or, more importantly, the 4-dimensional space-time space brought into physics by Minkowski and Einstein The same theorem, when applied to the physical 3-D... vector products of vectors and triple products of vectors 20 1.7 dr3 dr2 dr2 dr1 dr1 dr1 Figure 1.1: From vectors in a three-dimensional space we define the one-dimensional capacity j k k element d1 Σij = εijk dr1 , the two-dimensional capacity element d2 Σi = εijk dr1 dr2 and the j i k three-dimensional capacity element d3Σ = εijk dr1 dr2 dr3 In a metric space, the rank-two 1 form d Σij defines a.. . 2.1.5 Conditional Volume 2.2 Probability 2.2.1 Notion of Probability 2.2.2 Volumetric Probability 2.2.3 Probability Density 2.2.4 Volumetric Histograms and Density Histograms 2.2.5 Change of Variables 2.3 Sum and Product of Probabilities 2.3.1 Sum of Probabilities... forms and vectors (we can “raise and lower indices”) through Vi = gij V j The square root of the determinant of {gij } will be denoted g and we will see that it defines a natural bijection between capacities, tensors, and densities, like in pi = g pi , so, in addition to the rules concerning the indices, we will have rules concerning the “bars” Without a clear understanding of the concept of densities and. .. 1.4 The Levi-Civita Tensor 1.5 The Kronecker Tensor 1.6 Totally Antisymmetric Tensors 1.7 Integration, Volumes 1.8 Appendixes 2 Elements of Probability 2.1 Volume 2.2 Probability 2.3 Sum and Product of Probabilities 2.4 Conditional Probability 2.5 Marginal Probability. .. Densities Tensors and Capacities Using the scalar density g we can associate tensor densities, pure tensors, and tensor capacities Using the same letter to designate the objects related through this natural bijection, we will write expressions like ρ = gρ ; V i = g Vi or g T ij kl = Tij kl (1.29) So, if gij and g ij can be used to “lower and raise indices”, g and g can be used to “put and remove bars”... even number, and (−1)pq = +1 Remark that a multiplication and a division by g will not change the value of an expression, so that, instead of using Levi-Civita’s density and capacity we can use Levi-Civita’s true tensors For instance, εi1 ip s1 sq εj1 jp s1 sq = εi1 ip s1 sq εj1 jp s1 sq (1.47) Comment: explain better Appendix 1.8.4 gives special formulas to spaces with dimension 2 , 3 , and 4 As shown... 2.7 Central Estimators and Dispersion Estimators 2.7.1 Introduction 2.7.2 Center and Radius of a Probability Distribution 2.8 Appendixes 2.8.1 Appendix: Conditional Probability Density 2.8.2 Appendix: Marginal Probability Density 2.8.3 Appendix: Replacement Gymnastics 2.8.4 Appendix: The Gaussian Probability Distribution