Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 Table of Contents I Computational Statistics I.1 Computational Statistics: An Introduction James E Gentle, Wolfgang Härdle, Yuichi Mori II Statistical Computing II.1 Basic Computational Algorithms John Monahan 19 II.2 Random Number Generation Pierre L’Ecuyer 35 II.3 Markov Chain Monte Carlo Technology Siddhartha Chib 71 II.4 Numerical Linear Algebra ˇ zková, Pavel Cíˇ ˇ zek 103 Lenka Cíˇ II.5 The EM Algorithm Shu Kay Ng, Thriyambakam Krishnan, Geoffrey J McLachlan 137 II.6 Stochastic Optimization James C Spall 169 II.7 Transforms in Statistics Brani Vidakovic 199 II.8 Parallel Computing Techniques Junji Nakano 237 II.9 Statistical Databases Claus Boyens, Oliver Günther, Hans-J Lenz 267 II.10 Interactive and Dynamic Graphics Jürgen Symanzik 293 Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 VI Table of Contents II.11 The Grammar of Graphics Leland Wilkinson 337 II.12 Statistical User Interfaces Sigbert Klinke .379 II.13 Object Oriented Computing Miroslav Virius 403 III Statistical Methodology III.1 Model Selection Yuedong Wang 437 III.2 Bootstrap and Resampling Enno Mammen, Swagata Nandi 467 III.3 Design and Analysis of Monte Carlo Experiments Jack P.C Kleijnen 497 III.4 Multivariate Density Estimation and Visualization David W Scott 517 III.5 Smoothing: Local Regression Techniques Catherine Loader 539 III.6 Dimension Reduction Methods Masahiro Mizuta 565 III.7 Generalized Linear Models Marlene Müller 591 III.8 (Non) Linear Regression Modeling ˇ zek 621 Pavel Cíˇ III.9 Robust Statistics Laurie Davies, Ursula Gather 655 III.10 Semiparametric Models Joel L Horowitz 697 III.11 Bayesian Computational Methods Christian P Robert 719 III.12 Computational Methods in Survival Analysis Toshinari Kamakura 767 III.13 Data and Knowledge Mining Adalbert Wilhelm 787 III.14 Recursive Partitioning and Tree-based Methods Heping Zhang .813 III.15 Support Vector Machines Sebastian Mika, Christin Schäfer, Pavel Laskov, David Tax, Klaus-Robert Müller 841 III.16 Bagging, Boosting and Ensemble Methods Peter Bă uhlmann 877 Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 Table of Contents VII IV Selected Applications IV.1 Computationally Intensive Value at Risk Calculations Rafał Weron 911 IV.2 Econometrics Luc Bauwens, Jeroen V.K Rombouts 951 IV.3 Statistical and Computational Geometry of Biomolecular Structure Iosif I Vaisman 981 IV.4 Functional Magnetic Resonance Imaging William F Eddy, Rebecca L McNamee .1001 IV.5 Network Intrusion Detection David J Marchette 1029 Subject Index 1053 Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 List of Contributors Luc Bauwens Université catholique de Louvain CORE and Department of Economics Belgium bauwens@core.ucl.ac.be Claus Boyens Humboldt-Universität zu Berlin Institut für Wirtschaftsinformatik Wirtschaftswissenschaftliche Fakultät Germany Peter Bühlmann ETH Zürich Seminar für Statistik Switzerland buhlmann@stat.math.ethz.ch Siddhartha Chib Washington University in Saint Louis John M Olin School of Business chib@wustl.edu ˇ zek Pavel Cíˇ Tilburg University Department of Econometrics & Operations Research The Netherlands P.Cizek@uvt.nl ˇ zková Lenka Cíˇ Czech Technical University in Prague Faculty of Nuclear Sciences and Physical Engineering The Czech Republic lenka_cizkova@web.de Laurie Davies University of Essen Department of Mathematics Germany laurie.davies@uni-essen.de William F Eddy Carnegie Mellon University Department of Statistics USA bill@stat.cmu.edu Ursula Gather University of Dortmund Department of Statistics Germany gather@statistik.uni-dortmund.de James E Gentle George Mason University USA jgentle@gmu.edu Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 X List of Contributors Oliver Günther Humboldt-Universität zu Berlin Institut für Wirtschaftsinformatik Wirtschaftswissenschaftliche Fakultät Germany Pavel Laskov Fraunhofer FIRST Department IDA Germany laskov@first.fhg.de Wolfgang Härdle Humboldt-Universität zu Berlin Wirtschaftswissenschaftliche Fakultät Institut für Statistik und Ökonometrie Germany haerdle@wiwi.hu-berlin.de Pierre L’Ecuyer Université de Montréal GERAD and Département d’informatique et de recherche opérationnelle Canada Joel L Horowitz Northwestern University Department of Economics USA Hans-J Lenz Freie Universität Berlin Fachbereich Wirtschaftswissenschaft Institut für Produktion, Wirtschaftsinformatik und Operations Research und Institut für Statistik und Ökonometrie Germany Toshinari Kamakura Chuo University Japan kamakura@indsys.chuo-u.ac.jp Jack P.C Kleijnen Tilburg University Department of Information Systems and Management Center for Economic Research (CentER) The Netherlands Kleijnen@uvt.nl Sigbert Klinke Humboldt-Universität zu Berlin Wirtschaftswissenschaftliche Fakultät Institut für Statistik und Ökonometrie Germany sigbert@wiwi.hu-berlin.de Thriyambakam Krishnan Systat Software Asia-Pacific Ltd Bangalore India krishnan@systat.com Catherine Loader Case Western Reserve University Department of Statistics USA catherine@case.edu Enno Mammen University of Mannheim Department of Economics Germany emammen@rumms.uni-mannheim.de David J Marchette John Hopkins University Whiting School of Engineering USA dmarche@nswc.navy.mil Geoffrey J McLachlan University of Queensland Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 List of Contributors XI Department of Mathematics Australia gjm@maths.uq.edu.au Rebecca L McNamee University of Pittsburgh USA rlandes@stat.cmu.edu Sebastian Mika idalab GmbH Germany mika@idalab.de and Fraunhofer FIRST Department IDA Germany mika@first.fhg.de Masahiro Mizuta Hokkaido University Information Initiative Center Japan mizuta@cims.hokudai.ac.jp John Monahan North Carolina State University Department of Statistics USA monahan@stat.ncsu.edu and University Potsdam Department of Computer Science Germany Marlene Müller Fraunhofer ITWM Germany marlene.mueller@gmx.de Junji Nakano The Institute of Statistical Mathematics Japan nakanoj@ism.ac.jp Swagata Nandi University Heidelberg Institute of Applied Mathematics Germany nandi@statlab.uni-heidelberg.de Shu Kay Ng University of Queensland Department of Mathematics Australia skn@maths.uq.edu.au Yuichi Mori Okayama University of Science Department of Socioinformation Japan mori@soci.ous.ac.jp Christian P Robert Université Paris Dauphine CERMADE France christian.robert @ceremade.dauphine.fr Klaus-Robert Müller Fraunhofer FIRST Department IDA Germany klaus@first.fhg.de Jeroen V.K Rombouts Université catholique de Louvain CORE and Department of Economics Belgium rombouts@core.ucl.ac.be Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 XII List of Contributors Christin Schäfer Fraunhofer FIRST Department IDA Germany christin@first.fhg.de David W Scott Rice University Department of Statistics USA scottdw@rice.edu James C Spall The Johns Hopkins University Applied Physics Laboratory USA james.spall@jhuapl.edu Jürgen Symanzik Utah State University Department of Mathematics and Statistics USA symanzik@math.usu.edu David Tax Delft University of Technology The Netherlands Iosif I Vaisman George Mason University School of Computational Sciences USA ivaisman@gmu.edu Brani Vidakovic School of Industrial and Systems Engineering Georgia Institute of Technology USA brani@isye.gatech.edu Miroslav Virius Czech Technical University in Prague Faculty of Nuclear Sciences and Physical Engineering Czech Republic virius@km1.fjfi.cvut.cz Yuedong Wang University of California Department of Statistics and Applied Probability USA yuedong@pstat.ucsb.edu Rafaä Weron Hugo Steinhaus Center for Stochastic Methods Wrocław University of Technology Poland rweron@im.pwr.wroc.pl Adalbert Wilhelm International University Bremen School of Humanities and Social Sciences Germany a.wilhelm@iu-bremen.de Leland Wilkinson SPSS Inc and Northwestern University USA leland@spss.com Heping Zhang Yale University School of Medicine Department of Epidemiology and Public Health USA heping.zhang@yale.edu Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 Part I Computational Statistics Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 1056 Index − Archimedean, 943 − elliptical, 943 − estimation, 944 − simulation − − conditional distributions method, 944 cost-complexity, 819 count data, 98, 592 count model, 975 counting process, 779, 780 covariance functional, 678 covariate, 682, 774 covering numbers, 847 Cox model, 774, 776 Cox’s discrete model, 776 critical sampling, 217 cross, 344, 351, 368 cross validation, 438, 444, 450, 453, 454, 456, 551, 630, 819 crossover, 188, 189 CrystalVision, 309–311, 317 cumulative logit model, 613 cumulative mean function, 780 curse of dimension, 524, 557, 736, 741 cyclic menu, 386 data augmentation, 92, 955, 972, 975 data mining (DM), 160, 295 Data Viewer, 311, 312 data visualization, 519 DataDesk, 399 dataflow, 339 daughter node, 817 DC shifts, 1011 Decision theory, 721 decision trees, 794, 796, 814 decomposition algorithm, 232 defensive sampling, 759 degrees of freedom, 444, 453, 549 denial of service attack, 1030, 1032, 1038, 1039 density estimation, 560, 561 dependent data, 470, 472, 483–485, 491, 492 descriptive modeling, 793 design matrix, 543 design of experiments, 498 destructor, 410 deterministic annealing EM (DAEM) algorithm, 151 deterministic simulation, 499, 511 Deutscher Aktienindex (DAX), 929, 941 deviance, 597 − penalized, 728 deviance information criterion (DIC), 728 Diamond Fast, 309 differentiable, 662, 669 digamma function, 771 dimension − high, 722, 754 − matching, 748 − unbounded, 724, 729 − unknown, 748 dimension reduction, 157, 159, 557 Dimension reduction methods of explanatory variables, 584 Dirichlet distribution, 975 discrepancy, 40 discrete logistic model, 774 discrete optimization, 171, 186 dispersion parameter, 601 distributed memory, 239 distribution − α-stable, 915, 916 − − S parametrization, 917 − − S0 parametrization, 917 − − characteristic function, 917 − − density function, 918 − − direct integration method, 919 − − distribution function, 918 − − Fama–Roll method, 922 − − maximum likelihood estimation, 926 − − method of moments, 923 − − regression method, 924, 925 − − simulation, 921 − − STABLE program, 920, 927 − binomial, 593, 726 − Cauchy, 917 − folded t, 735 − Gaussian, 917 − generalized hyperbolic − − density function, 935 Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 Index 1057 early binding, 420 effective number of parameters, 550 efficiency, 664, 670 efficiency of the sample mean, 771 eigenvalues, 126 − inverse iterations, 129 − Jacobi method, 127 − LR method, 129 − power method, 127 − QR method, 128 eigenvectors, 126 electroencephalogram (EEG), 1006 elitism, 188, 189 EM algorithm, 745 − extensions, 140, 160, 154 EM mapping, 142, 144 embarrassingly parallel, 244 empirical measure, 661 encapsulation, 406 encoding, 188 entropy, 57, 817 equidissection, 51 equidistribution, 51 equivariance, 662, 673 ergodic chain, 77 estimation vs testing, 728 estimator − harmonic mean, 739 − maximum a posteriori (MAP), 728 Euler’s constant, 771 evolution strategies, 186 evolutionary computation, 186 exact distance, 576 excess kurtosis, 961, 968 expectation-conditional maximization (ECM) algorithm, 156, 164 E-step (Expectation step), 139, 141, 583 − multicycle ECM, 156, 157 − exponential family, 141 expectation-conditional maximization either − factor analysis model, 158 (ECME) algorithm, 159, 160, 164 − failure-time data, 148 expected shortfall (ES), 912 − generalized linear mixed models (GLMM), expected tail loss (ETL), 912 155 EXPLOR4, 311 − Monte Carlo, 154, 155 exploratory data analysis (EDA), 295, 788 − nonapplicability, 148 exploratory spatial data analysis (ESDA), 317 − normal mixtures, 146 ExplorN, 305, 309–311, 317 − − maximum likelihood estimation, 938, 939 − − mean, 935 − − simulation, 936 − − variance, 935 − generalized inverse Gaussian (GIG), 933 − − simulation, 936, 937 − hyperbolic, 932 − − density function, 934 − − inverse, 941 − inverse Gaussian (IG) − − simulation, 937 − Lévy, 917 − Lévy stable, 915 − mixture, 743, 749 − normal inverse Gaussian (NIG) − − density function, 935 − − simulation, 937 − − tail estimates, 939 − − tails, 936 − predictive, 723 − proposal, 733 − stable Paretian, 915 − t, 732 − target, 740 − truncated stable (TLD), 930 − − characteristic function, 931 dot plot, 519 doubledecker plot, 810 Dow Jones Industrial Average (DJIA), 913, 914, 928, 940 downweighting outlying observations, 665 dual lattice, 44, 52 dynamic duration model|analysis, 952, 961, 969 Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 1058 Index exponential density function, 770 exponential distribution, 672, 769, 770 exponential family, 141, 147, 148, 592, 593, 596, 722 − sufficient statistics, 141, 158 Extensible Markup Language (XML), 340 extreme value distribution, 959 factor analysis model, 157 failure-time data − censored, 140, 147 − exponential distribution, 148 false discovery rate (FDR), 1020 fat-shattering dimension, 847 fault detection, 185 feedforward network, 800 filter − high-pass, 223 − quadrature mirror, 223 final prediction error (FPE), 450 finite mixture, 972 − model, 952, 971, 972, 975 − of Gaussian densities, 966 finite-difference SA (FDSA), 181 Fisher consistent, 660 Fisher information, 781 − generalized linear model (GLM), 602 Fisher scoring algorithm, 599 fitness function, 187, 189 fitness proportionate selection, 189 Fitts forecasting model, 390 floating-point, 188 focusing, 301 font, 392 fork–join, 241 Fourier plot, 366 Fourier space, 1010 Fourier transform, 1010 Fréchet differentiable, 669, 676, 679 free-induction decay (FID) signal, 1004 frequency domain bootstrap, 489–491 frequency polygon, 528 Friedman’s index, 569 full conditional distributions, 89 full likelihood, 776 full-screen view, 390 Functional Image Analysis Software – Computational Olio (FIASCO), 1014, 1022 functional model, 540 functional neuroimaging, 1006 gain sequences, 182 gamma distribution, 594, 769, 970 GARCH, 474–476, 478–481, 952, 961, 973, 974 Gauss–Jordan elimination, 117 Gauss–Newton method, 646 Gauss–Seidel method, 122, 125 Gaussian quadrature, 458 Gaussian simulation smoother, 965, 966 Gaussian|normal distribution, 955–963, 965– 970, 973 − Matrix, 956 − truncated, 956 gene expression data, 164 generalized additive model, 616 generalized cross validation, 438, 444, 453, 455, 456, 460, 552 generalized degrees of freedom, 445, 450– 452 generalized EM (GEM) algorithm, 142, 156, 157 generalized estimating equations (GEE), 615 generalized feedback shift register (GFSR), 51, 54 generalized linear mixed models (GLMM), 154 generalized linear model (GLM), 592 generalized maximum likelihood method, 453, 460, 775 generalized method of moments, 954 generalized partial linear model, 616 generalized principal components, 571 generalized principal components analysis (GPCA), 571 genetic algorithm, 186, 191 geographic brushing, 317 Geometric distribution, 594 geometrically ergodic, 77 getter, 406 GGobi, 309, 311–313, 399 Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 Index Gibbs sampling algorithm, 73 Gibbs sampling|sampler, 162–165, 743, 747, 955, 957, 960, 966, 972, 973, 975 − griddy-, 975 − mixing of, 744 Givens rotations, 110 Glivenko–Cantelli theorem, 661 global optimization, 180, 186 global solutions, 175 goodness of fit, 40, 444, 547, 555 gradient approximation, 181, 183 Gram–Schmidt orthogonalization, 112 grand tour, 303, 305, 306 graphics algebra, 343 Green’s algorithm, 748 Greenwood’s formula, 769 gross error model, 676 gross error neighbourhood, 667, 684 Gumbel distribution, 958 Gustafson’s law, 243 Hall’s index, 570 Halton sequences, 959 Hampel identifier, 672 hard thresholding, 443 harmonic mean, 738 hat function, 62 hat matrix, 550 Hawkes process, 971 hazard function, 768, 779, 970 hazard rate, 768 head motion, 1015–1017 Heisenberg’s uncertainty principle, 208 Hessian (or Jacobian) matrix, 186 Hessian (second derivative) matrix, 181 heterogeneity, 955, 957, 958, 960, 970 heterogeneous populations, 727, 744 heteroscedasticity, 460 hexagonal bins, 522 hidden Markov model, 143, 164, 165 hierarchical Bayes, 960 hierarchical command sequence, 393 high breakdown affine equivariant location and scale functionals, 677 high breakdown regression functional, 685 1059 higher-order kernels, 524 highest possible breakdown point, 668 highest posterior region, 725 Householder reflections, 109 HPF (High Performance Fortran), 260 Huber distribution, 659 hue brushing, 307 human-machine interaction, 185 HyperVision, 310, 311 hypotheses, 724 hypothesis testing, 555 i.i.d resampling, 468, 473, 474, 485, 490 identifiability, 723 identification, 958, 972 − problem, 972, 974 − restrictions, 954, 972, 974 Image Analysis, 164 image grand tour (IGT), 305 image registration, 1016 immersive projection technology (IPT), 314, 316, 317 importance function, 731, 963–965 − choice of, 731 − with finite variance, 731 importance sampling, 81, 458, 500, 964, 965 − and regular Monte Carlo, 734 − degeneracy of, 737, 760 − efficient (EIS), 962–965, 967–971 − for model choice, 737 incomplete-data − likelihood, 139, 141 − missing data, 139, 140, 146, 148, 152, 155, 165 − problem, 138, 139, 146, 163 incremental EM (IEM) algorithm, 161 independence M–H, 81 independence of irrelevant alternatives (IIA), 958 independent increments, 780 indexed search, 61 inefficiency factor, 78 infinite collection of models, 729 influence function, 659 information criterion Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 1060 Index − Akaike, 438, 444, 449, 450, 452, 453, 456, 602, 628, 728, 846 − Bayesian, 438, 444, 449, 450, 452, 456, 459, 603 − Schwarz, 628 information matrix − complete-data, 152 − expected, 151, 152 − observed, 151, 152 inheritance, 413, 418 injected randomness, 174 instance, 407 integral, 952–954, 959, 960, 962, 963, 965 − approximation, 729 − high dimensional, 955 − multiple, 962 − ratio, 725 integrated mean square error, 523 intensity − function, 971 − model, 969, 971 intensity functions, 779 inter arrival time, 1035, 1036 interaction term, 609 interface, 406, 419, 423 interface for derivation, 411 Interface Symposia, International Association for Statistical Computing (IASC), internet protocol, 1031 intersection classifier, 1045, 1046 intersection graph, 1046 invariant, 74 Inverse Gaussian distribution, 594 inverse iterations, 129 inverse moments, 185 inversion method, 37, 59 inverted gamma density|prior, 966, 969, 973 inverted Wishart distribution, 956, 960 iterative refinement, 119 iterative simulation algorithms, 163 iteratively reweighted least squares, 596, 598 Jacobi method, 122, 127 Jasp, 399 Java threads, 250 k-space, 1010, 1022 Kalman filter, 965, 966, 970 − augmented, 966 Kaplan–Meier curves, 829 Kaplan–Meier method, 769 Karush–Kuhn–Tucker (KKT) condition, 860 kernel − function, 851 − kernel trick, 852 − matrix, 854 − mercer, 856 kernel density, 1034, 1036 kernel density estimation, 62, 307 kernel estimation, 531, 967 kernel smoother, 541 keystroke timings, 1043, 1045 knowledge discovery, 788 Kolmogoroff metric, 659, 661, 668 kriging, 498, 504, 510 Kuiper metric, 668 Kullback-Leibler discrepancy, 450 lagged-Fibonacci generator, 45 Lagrange multipliers, 567 Laplace approximation, 458 largest nonidentifiable outlier, 673 Larmor frequency, 1003 lasso, 639 − computation, 640 late binding, 420 latent variables, 74 Latin hypercube sampling, 512 lattice, 43, 52, 57 Law of Large Numbers, 730 learning, 843 least median of squares LMS, 684 least squares, 623 − computation, 623 − explicit form, 623 − Gauss–Markov theorem, 623 − inference, 624 − orthogonal transformations, 624 least trimmed squares, 685, 686 Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 Index length of the shortest half, 669 Levenberg–Marquardt method, 647 leverage effect, 969 leverage point, 686 library, 393 likelihood, 954, 955, 964, 965, 972 − function, 952, 962, 965, 967, 970, 972 − intensity-based, 971 − intractable, 720 − marginal, 975 − maximum, 954, 962, 972 − simulated, 954, 959 likelihood ratio test − generalized linear model (GLM), 602 likelihood smoothing, 558 limited dependent variable, 952 linear congruential generator (LCG), 43, 49, 57 linear discriminant analysis, 815 linear feedback shift register (LFSR), 51, 53, 55, 57 linear recurrence, 42 linear recurrence modulo 2, 50 linear recurrence with carry, 49 linear reduction, 566 linear regression, 592, 622, 682 linear smoother, 541, 547 linear system − direct methods, 116 − − Gauss–Jordan elimination, 117 − − iterative refinement, 119 − gradient methods, 124 − − conjugate gradient method, 125 − − Gauss–Seidel method, 125 − − steepest descent method, 125 − iterative methods, 120 − − Gauss–Seidel method, 122 − − general principle, 120 − − Jacobi method, 122 − − successive overrelaxation (SOR) method, 123 link function, 559, 592, 594, 739 − canonical, 595 linked brushing, 300 linked highlighting, 808 1061 linked views, 300 linking, 388 local bootstrap, 486, 487 local likelihood, 559 local likelihood equations, 559 local linear estimate, 543, 544 local optimization, 180 local polynomial, 547, 559 local regression, 542, 616 local reversibility, 88 localized random search, 178, 182 location functional, 659, 662, 667, 669, 674, 678 location-scale-free transformation, 772 log-likelihood, 84 − generalized linear model (GLM), 596 log-linear model, 614, 781 log-logistic distribution, 769 log-normal distribution, 769, 958, 971 log-rank statistic, 830 logistic distribution, 959 logit, 593 − mixed, 959, 960, 971 − mixed multinomial (MMNL), 953, 958– 960 − model, 959, 971 − multinomial, 958 − probability, 958, 959 logit model, 592, 596, 604 longitudinal, 98 longitudinal data, 614 loss function, 170, 844 low pass filter, 443 LR method, 129 LU decomposition, 106 M-estimation, 560 M-functional, 663, 664, 668, 669, 671, 675, 676, 682, 684 − with a redescending ψ-function, 666 M-step (Maximization step), 139, 141 − exponential family, 141, 148 − factor analysis model, 158 − failure-time data, 148 − generalized EM (GEM) algorithm, 142 Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 1062 Index − normal mixtures, 146 magnetic field inhomogeneities, 1011, 1014 magnetic resonance, 1003 magnetic resonance imaging, 1004 magnetism, 1003 magnetoencephalogram (MEG), 1006 Mallow Cp, 438, 444, 450, 452, 453, 456 MANET, 308–310 margin, 850 marginal distribution function, 773 marginal likelihood, 72 market risk, 912 marketing, 953, 958 Markov bootstrap, 483, 488, 489, 491 Markov chain, 73, 75, 192 Markov chain Monte Carlo (MCMC), 72, 162, 163, 458, 963, 966–969, 971–974 Markov chain Monte Carlo (MCMC) algorithm, 720, 740 − automated, 754 Markov random field, 165 Markov switching autoregressive model, 973 masking effect, 671 Mason Hypergraphics, 314 Mathematica, 399 mathematical programming, 181 matrix decompositions, 104 − Cholesky decomposition, 105 − Givens rotations, 110 − Gram–Schmidt orthogonalization, 112 − Householder reflections, 109 − LU decomposition, 106 − QR decomposition, 108 − SVD decomposition, 114 matrix inversion, 115 matrix linear recurrence, 50 maximally equidistributed, 52, 55 maximum full likelihood, 777 maximum likelihood, 596, 597, 954, 962, 972 − Monte Carlo (MCML), 962, 964, 965, 967– 969, 971 − quasi- (QML), 970 − simulated, 954 maximum likelihood estimate, 666, 770 maximum likelihood estimation, 138 − global maximum, 138, 143, 144 − local maxima, 138, 143, 144, 150 maximum partial likelihood, 777 maximum score method, 714 mean squared error, 184, 446, 523, 547 measurement noise, 182 median, 657, 659, 663, 668, 671, 674, 687, 689 median absolute deviation MAD, 658, 663, 668, 671, 687 median polish, 689 menu hierarchy, 386 Mersenne twister, 51, 54, 55, 58 message, 407 metaclass, 408 metamodel, 499, 501, 502, 511 method of composition, 97 method of moments, 771 Metropolis method, 72 Metropolis–Hastings algorithm (MH), 163, 740, 741, 960 Metropolis–Hastings method, 73 micromaps, 319 military conscripts, 743 MIMD (multiple instruction stream–multiple data stream), 239 MiniCAVE, 316 minimum covariance determinant (MCD), 678 minimum volume ellipsoid (MVE), 678 mirror filter, 229 misclassification cost, 818 missing variables − simulation of, 743 mixed model, 614 mixed multinomial logit (MMNL), 953, 958– 960 mixing, 74 mixing density|distribution, 958–960 mixture − Poisson distributions, 727 mixture models, 138, 150, 162 − mixture of factor analyzers, 159, 164 − normal mixtures, 145, 152, 160–162 Mixture Sampler algorithm, 966 mode Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 Index − attraction, 742 mode tree, 531 model − AR, 723, 728, 751 − averaging, 728, 750 − binomial, 726 − choice, 747 − generalised linear, 722 − generalized linear, 739 − index, 748 − mixture, 744 − probit, 736, 740 model averaging, 728 model choice, 726 − and testing, 727 − parameter space, 728 model complexity, 444 model domain, 405 model selection, 438, 972, 973 − generalized linear model (GLM), 602 modified Bessel function, 933 moment generating function, 204 moment index, 570 Mondrian, 308–310 Monte Carlo, 172 − confidence interval, 733 − Markov chain (MCMC), 963, 966–969, 971– 974 − maximum likelihood (MCML), 962 − with importance function, 731 Monte Carlo EM, 154, 155, 163 Monte Carlo maximum likelihood (MCML), 962, 964, 965, 967–969, 971 Monte Carlo method, 37, 405, 498, 499, 730 − and the curse of dimension, 736 Monte Carlo techniques, 729 − efficiency of, 734 − population, 757 − sequential, 757 Moore’s law, 238 mosaic map, 522 mosaic plot, 307, 808 mother wavelet, 221 MPI (Message Passing Interface), 256 multicollinearity, 625, 626 1063 − exact, 625, 626 − near, 626 multilevel model, 615 multimodality, 974 multinomial distribution, 975 multinomial responses, 612 multiple binary responses, 832 multiple counting processes, 780 multiple document interface, 387 multiple failures, 779 multiple recursive generator (MRG), 42 multiple recursive matrix generator, 54 multiple-block M–H algorithms, 73 multiply-with-carry generator, 49 multiresolution analysis (MRA), 217 multiresolution kd-trees, 161 multivariate smoothing, 557 multivariate-t density, 85 mutation, 188, 190 Nadaraya–Watson estimate, 474, 541 negative binomial distribution, 594 nested models, 602 network sensor, 1033 neural network, 184, 799, 853 New York Stock Exchange (NYSE), 970 Newton’s method, 181, 646 Newton–Raphson algorithm, 181, 186, 598, 959 Newton–Raphson method, 560, 782 Neyman–Pearson theory, 724 no free lunch (NFL) theorems, 175 node impurity, 817 noisy measurement, 179, 180 nominal logistic regression, 612 non-nested models, 602 nonhomogeneous Poisson process, 781, 783 nonlinear least squares, 645 − asymptotic normality, 645 − inference, 647 nonlinear regression, 622, 644 nonparametric autoregressive bootstrap, 486 nonparametric curve estimation, 471, 472, 491 nonparametric density estimation, 518 Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 1064 Index normal approximation, 733 normal distribution, 658, 670 normal equations, 543, 623 normalization property, 219, 225 normalizing constant, 731, 737 − ratios of, 737 novelty detection, 843, 868 NOW (network of workstations), 241 nuisance parameter, 781, 783 null deviance, 604 NUMA (non-uniform memory access), 240 numerical standard error, 78 nViZn, 321 Nyquist ghosts, 1011, 1014 object, 406 object composition, 410, 418 Object oriented programming (OOP), 405 object, starting, 432 Occam’s razor, 444, 457 offset, 601 Old Faithful geyser data, 519, 520 one-way analysis of variance, 686 one-way table, 686 OpenMP, 251 optimization|optimizer, 959, 964, 967, 972 order of menu items, 386 ordered probit model, 613 ordinal logistic regression, 613 ordinary least squares (OLS), 963, 964 orthogonal series, 546 orthogonality property, 219, 225 outlier, 657, 665, 670, 671, 680, 684, 689, 969 outlier detection, 843, 868 outlier identification, 681, 687 outlier region, 672 outwards testing, 673 overdispersion, 612 overfitting, 799 oversmoothing, 524, 528 panel data, 615, 952, 975 panning, 301 parallel computing, 238 parallel coordinate display, 303 parallel coordinate plot, 303, 366 parallel coordinates, 806, 1039, 1042 parameter − of interest, 723 parameter space − constrained, 720 parameter–expanded EM (PX–EM) algorithm, 160 Parseval formula, 211 partial autocorrelation, 724 partial least squares, 641 − algorithm, 642 − extensions, 643 − latent variables, 642 − modified Wold’s R, 642 − nonlinear regression, 649 − Wold’s R, 642 partial likelihood, 775, 777 partially linear models, 704 particle systems, 757 password cracking, 1043 pattern recognition, 184 Pearson statistic, 601 penalized least squares, 441, 544 penalized likelihood, 138, 160, 165, 559 perfect sampling method, 99 periodogram, 207 permutation tests, 482 physiological noise, 1012, 1017 pie chart, 307 piecewise polynomial, 545 pilot estimate, 554 pixel grand tour, 305 plug-in, 468–470 plug-in bandwidth selection, 553 PMC vs MCMC, 761 point process, 971 Poisson data, 593 Poisson distribution, 57, 779, 972 Poisson process, 63 poly-t distribution, 722 polymorphic class, 420 polymorphism, 419 polynomial lattice, 52 polynomial LCG, 53 Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 Index polynomial regression, 499, 508, 513 polynomial terms, 606 population, 187 population Monte Carlo (PMC) techniques, 757 positron emission tomography (PET), 1006 posterior density, 83, 955, 959, 960, 962, 967, 968, 974 posterior distribution, 722 posterior mean, 955–957, 967–969, 974 posterior probability, 146, 150, 161 power expansions, 778 power method, 127 power parameter, 772 prediction, 551, 723 − sequential, 724 predictive modeling, 794 predictive squared error, 446 PRESS, 454 primitive polynomial, 43, 50 principal components analysis (PCA), 566, 632 principal components regression, 632, 633 − choice of principle components, 633 principal curve, 582 prior − proper, 730 prior (density), 955–957, 960, 966, 967, 969, 972, 974 − informative, 974 − uninformative, 956, 967 prior distribution, 720 − conjugate, 722 − selection of a, 721 prior-posterior summary, 84 probability of move, 73 probit − model, 958, 972 − multinomial, 958 − multinomial multiperiod, 952–954 − multivariate, 953, 958 − static multinomial, 955 probit model, 596, 605 probit regression, 93 problem domain, 405 1065 process control, 185 process forking, 245 productivity, 390 program execution profiling, 1048 progress bar, 390 projection, 302 projection index, 569 projection pursuit, 305, 557, 568 projection pursuit guided tour, 305 projection pursuit index, 305 projection step, 583 proportion, 568 proportional hazard, 830 proportional hazards model, 710, 776 proposal − adaptive, 761 − multiscale, 759 proposal distribution, 73 prosection matrix, 302 prosections, 302 proximity, 825 pruning, 799 pseudo data, 468 pseudo-likelihood, 612 pseudorandom number generator, 37 Pthread library, 248 pulse sequence, 1008 PVM (Parallel Virtual Machine), 253 QR decomposition, 108 QR method, 128 quadratic principal components analysis (QPCA), 572 quality improvement, 184 quasi-likelihood, 612 quasi-maximum likelihood (QML), 970 queuing systems, 184 R, 399, 763 radial basis networks, 853 random effects, 98, 154, 155 random forests, 825 random graph, 1046 random noise, 172 random number generator, 37, 410 Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 1066 Index − approximate factoring, 46 − combined generators, 47, 54, 56, 58 − definition, 38 − figure of merit, 44, 51 − floating-point implementation, 46 − implementation, 46, 56 − jumping ahead, 39, 48, 51 − non-uniform, 58 − nonlinear, 55 − period length, 39, 50, 54 − physical device, 38 − power-of-two modulus, 47 − powers-of-two-decomposition, 46 − purely periodic, 39 − quality criteria, 39, 59 − seed, 38 − state, 38 − statistical tests, 41, 56 − streams and substreams, 39, 58 random numbers, 410 − common, 959, 964 − pseudo-, 959 − quasi-, 959 random permutation, 473, 474, 477, 482 random permutation sampler, 973 random perturbation vector, 184 random search, 176 random walk M–H, 80 Rao–Blackwellization, 95 rate of convergence, 144, 152, 156, 159–161 − rate matrix, 144 ratio − and normalizing constants, 731 − importance sampling for, 737 − of integrals, 731 − of posterior probabilities, 725 ratio-of-uniforms method, 63 real-number coding, 190 recursive partitioning, 366 recursive sample mean, 1034 red-green blindness, 392 redescending ψ-function, 665 reduced conditional ordinates, 96 reformatting, 302 REGARD, 308, 309, 318 regression depth, 685 regression equivariant, 683 regression functional, 682 regression splines, 545 regression trees, 835 regression-type bootstrap, 486, 487 regressor-outlier, 686 regularization, 846 rejection method, 62, 64 rejection sampling, 967 relative projection Pursuit, 571 remote sensing data, 521 resampling, 468, 469, 471–476, 478, 481–483, 485–492, 556 resampling tests, 476, 478 rescaling, 302 residual, 688 residual sum of squares, 598 residuals − generalized linear model (GLM), 601 resistant one-step identifier, 686 resolution of identity, 217 response-outlier, 686 restricted maximum likelihood, 460 reverse move, − probability, 749 reversible, 76 reversible jump MCMC, 748, 752 ridge regression, 340, 635 − almost unbiased, 637 − almost unbiased feasible, 637 − bias, 635 − choice of ridge parameter, 635–637 − feasible generalized, 637 − generalized, 636 − minimization formulation, 636 − nonlinear regression, 648 − reduced-rank data, 637 risk, 844 − empirical, 844 − expected, 845 − regularized, 846 − structural minimization, 847 risk measure, 912 Robbin–Monro algorithm, 757 Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 Index robust, 683 robust functional, 671 robust location functional, 661 robust regression, 560, 682, 684 robust scatter functional, 680 robust statistic, 657, 659, 661 robustness, 659, 663, 668 root node, 817 root-finding, 171, 182 rotation, 302 roulette wheel selection, 189 S-functional, 666, 679, 681, 685, 686 S-Plus, 399 sample mean, 771 sampler performance, 74 SAS, 399 Satterwaite approximation, 555 saturated model, 598 saturation brushing, 307 scalable EM algorithm, 161 scale functional, 663, 669 scales, 353 scaling algorithm, 742 scaling equation, 218 scaling function, 217 scanner data, 955 scatter diagram, 519, 521 scatterplot, 298, 387 scatterplot matrix, 298, 391 schema theory, 192 search direction, 172 secondary data analysis, 790 selection, 188, 189 selection sequences, 301 self-consistency, 143 semiparametric models, 557, 700 semiparametric regression, 615 sensitivity analysis, 498 sensor placement, 185 serial test, 57 setter, 406 shape parameter, 770 shared memory, 239 shortcut, 386 1067 shortest half, 661, 666, 678, 684 shrinkage estimation, 634 shrinking neighbourhood, 660 sieve bootstrap, 485, 486, 490, 491 SIMD (single instruction stream–multiple data stream), 239 simulated maximum likelihood (SML), 954, 959–961, 965, 967, 968 − quasi-random, 959 simulated moments, 954, 959 simulated scores, 959 simulated tempering, 98 simulation, 498, 499, 504, 509, 513, 729, 952, 954, 960, 962, 965, 966, 970, 975 simulation-based optimization, 173, 184 simultaneous perturbation SA (SPSA), 183 single index model, 615, 701 single trial fMRI, 1009 SISD (single instruction stream–single data stream), 239 slammer worm, 1048 slash distribution, 670 slice sampling, 92 sliced inverse regression, 584 slicing, 302 smooth bootstrap, 759 smoothed maximum score, 714 smoothing, 540 smoothing parameter, 442, 453, 459, 460, 523, 547 SMP (symmetric multiprocessor), 240 soft thresholding, 443 software reliability, 779 sparse matrices, 129 sparsity, 860 specification search, 699 spectral density, 207 spectral test, 44 spectrogram, 209 speech recognition, 814 Spider, 309 SPIEM algorithm, 161 spine plot, 307 spline, 438, 441, 447, 455, 456 spline smoother, 544, 545 Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 1068 Index spreadplots, 307 SPSA Web site, 184 SPSS, 399 SQL, 340 squeeze function, 63 SRM, see structural risk minimization stably bounded algebraic curve, 579 standard deviation, 658, 663, 668, 671 standard errors, 151–153 starting (initial) value, 143, 144, 150, 151 state space, 964, 966 state space model − Gaussian linear, 965, 970 stationarity, 724, 961, 974 statistical computing, statistical functional, 659 Statistical Parametric Mapping (SPM), 1022 steepest descent, 180 steepest descent method, 125 Stein-rule estimator, 634 stereoscopic display, 313 stereoscopic graphic, 313 stochastic approximation, 180 stochastic conditional duration (SCD) model, 970, 971 stochastic gradient, 180 Stochastic optimization, 170 stock trading system, 970 stopping rule, 757 streaming data, 1033 structural risk minimization (SRM), 847, 849 structure parameter, 783 Structured Query Language, 340 Student-t distribution, 968, 969 subsampling, 472, 482–484 subtract-with-borrow, 49 successive overrelaxation (SOR) method, 123 supervised learning, 791 supplemented EM (SEM) algorithm, 151, 154 support, 801 support vector machine, 843 − decomposition, 861 − linear, 849 − optimization, 857 − sequential minimal optimization (SMO), 863 − sparsity, 860 support vector novelty detection, 868 support vector regression, 867 surrogate data tests, 483 surrogate splits, 824 survival analysis, 147 survival function, 768 survival model, 560, 614 survival rate − variance, 760 survival trees, 829 susceptibility artifacts, 1013 SV model, 952, 961, 971 − canonical, 961, 962, 964, 965, 967, 968, 970 − multivariate, 969 SVD decomposition, 114 symmetric command sequence, 393 syn cookie, 1033 SYSTAT, 399 systems of linear equations − direct methods, 116 − − Gauss–Jordan elimination, 117 − − iterative refinement, 119 − gradient methods, 124 − − conjugate gradient method, 125 − − Gauss–Seidel method, 125 − − steepest descent method, 125 − iterative methods, 120 − − Gauss–Seidel method, 122 − − general principle, 120 − − Jacobi method, 122 − − SOR method, 123 Table Production Language (TPL), 352 tailored M–H, 81 tailoring, 88 TAQ database, 970 target tracking, 173 Tausworthe generator, 51, 53 Taylor series, 548 Taylor series expansions, 782 TCP three-way handshake, 1032 t-distribution, folded, 735 Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 Index tempering, 54 terminal nodes, 819 termination criterion, 192 Tesla, 1003 thinning, 63 threading, 247 threshold parameters, 770 thresholding, 213 time series, 470, 472–474, 480, 482–492, 952, 961, 974, 975 tissue contrast, 1005 tournament selection, 189 traffic management, 185 training data, 791 transform − continuous wavelet, 215 − discrete Fourier, 206 − discrete wavelet, 226 − empirical Fourier–Stieltjes, 204 − fast Fourier, 226 − Fourier–Stieltjes, 203 − Hilbert, 209 − integral Fourier, 208 − Laplace, 204 − short time Fourier, 209 − Wigner–Ville, 210 − windowed Fourier, 209 transformation − Box–Cox, 203 − Fisher z, 201 transformation models, 706 transformed density rejection, 63 transition kernel, 75 transition matrix, 193 translation equivariant functional, 677 transmission control protocol, 1031 transparent α-level contour, 307 trapping state, 745 tree growing, 817 tree pruning, 818 tree repairing, 833 Trellis display, 391 triangular distribution, 958 trigonometric regression, 438, 440, 447, 455, 456 1069 trojan program, 1035, 1036 Tukey’s biweight function, 665, 681 twisted generalized feedback shift register (GFSR), 51, 54, 55 two-way analysis of variance, 688 UMA (uniform memory access), 240 unbiased risk, 450, 453, 460 unbiased risk estimation, 553 under-fitting, 845 Unified Modelling Language (UML), 406, 411 uniform distribution, 37, 958, 974 uniformity measure, 40, 51 unit measurement, 355 unobserved (or latent) variables, 952 unobserved heterogeneity, 710 unpredictability, 42 unsupervised learning, 791 user profiling, 1043 utility|utilities, 953–955, 958, 959, 961 validation data, 791 Value at Risk (VaR), 912, 914 − copulas, 942 vanGogh, 399 vanishing moments, 225 Vapnik–Cervonenkis class, 674 variable − auxiliary, 747 variable selection, 438, 627 − all-subsets regression, 629 − − branch and bound, 630 − − genetic algorithms, 630 − backward elimination, 627 − cross validation, 630 − forward selection, 629 − least angle regression, 629 − stepwise regression, 627 variance estimation, 550 variance reduction, 59 variance reduction technique, 500 varset, 341, 342 VC-bound, 848 VC-dimension, 847 VC-theory, 847 Copyright Springer Heidelberg 2004 On-screen viewing permitted Printing not permitted Please buy this book at your bookshop Order information see http://www.springeronline.com/3-540-40464-3 1070 Index vector error-correction model, 955 Virtual Data Visualizer, 317 virtual desktop, 389 virtual reality (VR), 295, 313, 314, 316, 317, 322 Virtual Reality Modeling Language (VRML), 317 visual data mining (VDM), 295 volatility of asset returns, 952, 961 voting, 952, 958 VoxBo, 1022 VRGobi, 314–316 W-transformation, 772 Wasserstein metrics, 829 waterfall plot, 1037 wavelet domain, 226 wavelet regularization, 846 wavelets, 212 − Daubechies, 224 − Haar, 223 − Mexican hat, 215 − periodized, 232 − sombrero, 215 Weibull density function, 770 Weibull distribution, 769, 770, 970, 971 Weibull process model, 782 weight function, 541 weights − generalized linear model (GLM), 601, 611 wild bootstrap, 486, 487, 491 winBUGS, 720, 763 window titles, 1043 Wishart distribution, 957 working correlation, 615 XGobi, 305, 309, 311–318 XML, 340 XploRe, 399, 915, 920, 935 zooming, 301 ... paradigm of the data sciences The Cross Currents of Computational Statistics 1.2.4 Computational statistics of course is more closely related to statistics than to any other discipline, and computationally-intensive... Why This Handbook 1.3 The purpose of this handbook is to provide a survey of the basic concepts of computational statistics; that is, Concepts and Fundamentals A glance at the table of contents... the details of the computer system of numbers and operators One of the important uses of computers in statistics, and one that is central to computational statistics, is the simulation of random