Genetic and Evolutionary Computation Rick Riolo Ekaterina Vladislavleva Marylyn D. Ritchie Jason H. Moore Editors Genetic Programming Theory and Practice X www.it-ebooks.info Genetic and Evolutionary Computation Series Editors: David E. Goldberg John R. Koza For further volumes: http://www.springer.com/series/7373 www.it-ebooks.info www.it-ebooks.info Rick Riolo • Ekaterina Vladislavleva Marylyn D. Ritchie • Jason H. Moore Editors Genetic Programming Theory and Practice X Foreword by Bill Worzel 123 www.it-ebooks.info Editors Rick Riolo Center for the Study of Complex Systems University of Michigan Ann Arbor, Michigan, USA Marylyn D. Ritchie Department of Biochemistry and Molecular Biology The Pennsylvania State University University Park Pennsylvania, USA Ekaterina Vladislavleva Evolved Analytics Europe BVBA Beerse, Belgium Jason H. Moore Institute for Quantitative Biomedical Sciences Dartmouth Medical School Lebanon, New Hampshire, USA ISSN 1932-0167 ISBN 978-1-4614-6845-5 ISBN 978-1-4614-6846-2 (eBook) DOI 10.1007/978-1-4614-6846-2 Springer New York Heidelberg Dordrecht London Library of Congress Control Number: 2013937720 © Springer Science+Business Media New York 2013 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. While the advice and information in this book are believed to be true and accurate at the date of pub- lication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com) www.it-ebooks.info This tenth anniversary edition of GPTP is dedicated to the memory of Jason Daida. Jason’s presentations at the seminal GPTP workshops on structure and reachability inspired and greatly influenced our thinking and guided our research. Although his passion for teaching and education prevented his attendance at recent workshops, it was always a joy to encounter him be it at a conference or during one of many trips to UM’s sister university in Shanghai. A quick and innovative mind coupled with a ready smile and positive outlook is a tough combination not to cherish. Jason’s many students, friends and colleagues are testimony to his clear vision, dedication to learning and his love of life. We will miss him dearly. www.it-ebooks.info www.it-ebooks.info Foreword An Idiosyncratic Reflection on 10 Years of the Genetic Programming Theory and Practice Workshop Beginnings Ten years ago Carl Simon, then Director of the Program for the Study of Complex Systems (PSCS) at the University of Michigan invited me to lunch an asked me to give my input on a workshop on genetic programming (GP). Carl felt that as a growing, cutting edge field, it would be both useful and interesting for PSCS to sponsor a “state of the art” workshop of GP. As we discussed the idea, both Carl and I envisioned a one-time workshop that would bring together people actively working in the field. Little did I know that workshop would become a crucial part of my life and a regular event in my annual calendar. At the time GP was still quite a young discipline despite 20 years or more effort on the part of many researchers. Carl was looking for a unifying theme for the work- shop and after a few minutes of reflection, I suggested a theme of GP theory and practice, where computer scientists studying the theory of GP and practitioners ap- plying GP to real world problems could meet and discuss their respective progress. It was my thought that such a meeting could provide a review of the current state of theory and that GP programmers could use a better understanding of GP theory to improve the application of GP to “real-world” problems. Conversely, practical re- sults are the ultimate test of theory. Carl was enthusiastic about this idea and much to my surprise, asked me to work with Rick Riolo to organize the workshop. Working with Rick was both a pleasure and an education. As I had never been involved in organizing an academic conference or workshop, I let Rick lead the way. Rick and the PSCS staff not only handled the logistics of the conference, but he knew the right questions to ask about format and content. We decided to try to have a matched pairing of theory and practice papers where possible, knowing that this would often be difficult. We also had long discussions about the format of the workshop. It was my idea that we should have longer times for presentations than was normal for conferences as well as plenty of time for discussion. We also decided that at the end of a set of related presentations, we should provide time for discussion reflecting on the set of presentations and what bigger questions they raised. These decisions have proved to be fruitful as many times the extended discussion sessions have been the most valuable part of the workshop. vii www.it-ebooks.info viii Foreword Initially we conceived of the workshop as a place where people could present speculative ideas that they might not otherwise talk about at a peer reviewed confer- ence. Instead, we opted for chapters to be written by presenters that were reviewed by other workshop participants and published in book form. While this meant that all attendees’ submissions would be accepted, they nevertheless went through se- rious review that often radically changed the chapter as did the lengthy discussion sessions during the workshop. Another element we added was a daily keynote. Originally we planned for a gen- eralized topic for a keynote speaker on each day: One day was to be keynoted by someone in evolutionary biology, one on evolutionary computing and one by some- one who had expertise in integrating cutting edge technology into commercial appli- cations. While this strict format has not survived, its spirit has survived and over the years the keynotes have spawned many fruitful discussions both during question- and-answer sessions after the keynote and in many discussions that extended late into the evening. At the end of the first GPTP, it was by no means certain there would be a sec- ond workshop. It had been successful, but was not an unalloyed success in terms of content and quality. What was an overwhelming success was the interesting discus- sions at the workshop and deep into the nights at the end of each day. A little to my surprise, when asked whether they thought a second workshop was in order, there was an enthusiastically positive response from the attendees and from the entities that had provided financial support for the workshop, including the PSCS. Over the years that have followed, the format has modulated somewhat, PSCS be- came a Center (CSCS) but the general ideas we settled on that first year, speculative presentations, diverse keynotes, large amounts of discussion time and cross-reviews by participants, have largely stayed intact. Moreover, over time the workshop has developed its own flavor and style that has led people to return; some annually, oth- ers biannually and still others only when they had something new to say. Theory and Practice? Perhaps the best way to describe the organizing principle of GPTP is the quotation attributed to Jan Schnapsheut (and Yogi Berra!) “In theory there is no difference between theory and practice. But in practice there is.” The first thing that quickly became apparent from the early GPTP workshops is that practice always outruns theory because it is much easier to think up a new scheme that helps to solve a problem but much less so to explain the mathematical reasons why such a scheme improves the fundamental function of the underlying algorithm. The other thing that emerged was that practitioners became ersatz theorists, de- veloping tools and metrics to test and explain behaviors in GP. Not only did this lead to modifications of existing algorithms and new techniques that were clearly shown to improve outcomes, but it spurred new theoretical consideration of GP. Theorists began to move from work on such fundamentals as the building block hypothesis to www.it-ebooks.info Foreword ix broader questions that approached some of the questions the evolutionary biologists wrestle with such as: What are the constraints on evolution? What are the dynamics? What are the information theoretical underpinnings of GP? There is also a growing sense that researchers in natural and artificial evolution have something to say to each other. Selected Chapters As the title of this forward suggests, I have an idiosyncratic view of GPTP. Ap- proaching the 10th year, I decided to go back through the GPTP books published in past years, and pick some of my favorite chapters. This is totally subjective, with some of the chapters selected simply because they interested me personally, while others were chapters I selected because I thought they were particularly important to our improved understanding of GP and others, just because Whatfollowsisthe list of my choices from the first 10 years and some brief comments on them. This is by no means an exhaustive list or even a list of the “best” work done (but then evolution favors diversity over optimization), and I hope that people such as Trent McGonaghy, Erik Goodman and the many other people that I omitted from the list will not interpret this as lessening my respect for them or their work. GPTP I:“Three Fundamentals of the Biological Genetic Algorithm” by Steven Freeland. This keynote by the evolutionary biologist, Steven Freeland, outlined fundamen- tal characteristics of natural evolution that he felt should be adopted by genetic programming. Some of the items he mentions include particulate genes, an adap- tive genetic code, and the dichotomy between genotype and phenotype. He also sets a standard for measuring the success of evolutionary computing when he says “Biology will gain when evolutionary programmers place our system within their findings, illustrating the potential for biological inspiration from EC [Evolutionary Computing].” GPTP II: “The Role of Structure in Problem Solving by Computer” by Jason Daida. This chapter shows that there are natural limits on trees (and perhaps other related structures) that constrain the likely range of program-trees that can be created by standard genetic programming. This raises fundamental questions that have not been fully addressed in subsequent work. GPTP III: “Trivial Geography” by Spector and Klein. Spector and Klein showed that by creating a sense of place for individuals in a population and constraining their crossover partners to those in the near neighbor- hood, a significant improvement in efficiency and effectiveness can be realized. It also implicitly raises the question of an environment for evolution since once you have a sense of geography you can vary what is found in different locations (i.e., ecosystems). www.it-ebooks.info [...]... References Vladislavleva et al (2011) Genetic Programming Theory and Practice IX Springer, 2011 Neumann, O’Reilly, and Wagner, (2011) “Computational Complexity Analysis of Genetic Programming Initial Results and Futre Directions”, Genetic Programming Theory and Practice IX Springer, 2011 www.it-ebooks.info Contents 1 Evolving SQL Queries from Examples with Developmental Genetic Programming ... system biology, and a more systematic approach to GP can help bridge the gap between theory and practice www.it-ebooks.info xvi Preface Last year we stated that “symbolic regression and automated programming are just the two ends of a continuum of problems relevant for genetic programming: Symbolic Regression > Evolution of executable variable length structures > Automatic Programming And while the ‘simplest’... book was first presented at the Tenth Workshop on Genetic Programming, Theory and Practice, organized by the Center for the Study of Complex Systems at the University of Michigan, Ann Arbor, May 12–14, 2012 The goal of this workshop series is to promote the exchange of research results and ideas between those who focus on Genetic Programming (GP) theory and those who focus on the application of GP to... lspector@hampshire.edu R Riolo et al (eds.), Genetic Programming Theory and Practice X, Genetic and Evolutionary Computation, DOI 10.1007/978-1-4614-6846-2 1, © Springer Science+Business Media New York 2013 www.it-ebooks.info 1 2 T Helmuth and L Spector This issue can be addressed in part by a system that takes positive and negative example tuples—which is generally easy for users to provide and returns concise, comprehensible... understanding and advancing GP in theory and practice and look forward to the GPTP-2013 Acknowledgments We thank all the workshop participants for making the workshop an exciting and productive 3 days In particular we thank the authors, without whose hard work and creative talents, neither the workshop nor the book would be possible We also thank our three keynote speakers, Sean Luke, Seth Chandler and. .. the fields of genetic programming and multiagent systems His insight and experience in these areas contributed greatly to the workshop discussions about how to use genetic programming to solve complex problems Friday began with a talk by Professor Seth Chandler on “Evolving Binary decision trees that sound like law.” Chandler, professor of Law at University of Houston, gave a remarkable and enlightening... formal talks and of the informal talk during the scheduled and unscheduled discussions; and (2) acknowledgements of the many generous people and institutions who made the GPTP-2012 workshop possible by their financial and other support A Brief Summary of the Ideas from Talks and Talked About Ideas at GPTP-2012 As in the previous 10 springs, the 2012 workshop on Genetic Programming in Theory and Practice. .. between exploration and exploitation in open-ended evolution? • How to seamlessly integrate different types of data structures? www.it-ebooks.info Preface xix • If the goal of many problems we are attempting to solve is understanding of underlying process, what are innovative post processing methods for analysis and final selection of GP solutions? • Are diversity preservation and niching and expert knowledge... GPTP X (2012) www.it-ebooks.info Preface xvii • Wagy et al (Chap 6) presented a flexible distributed GP system incorporating many relaxations to evaluation and selection mechanisms, e.g data binning and island models • Moore et al (Chap 7) employed multi-objective Pareto-based selection with fitness and model size, as objectives in the computational evolution system for open-eded analysis of complex genetic. .. from Examples with Developmental Genetic Programming 5 Table 1.1: Instructions used in our PushGP runs Stack Instructions integer add, sub, mult, div, mod, stackdepth, dup, swap, rot string length, stackdepth where condition from stack, condition from index, condition distinct from index, condition from pos ex, condition from neg ex, and, or from ephemeral random constants (ERCs), which are random . Genetic and Evolutionary Computation Rick Riolo Ekaterina Vladislavleva Marylyn D. Ritchie Jason H. Moore Editors Genetic Programming Theory and Practice X www.it-ebooks.info Genetic and. to understanding and advancing GP in theory and practice and look forward to the GPTP-2013. Acknowledgments We thank all the workshop participants for making the workshop an exciting and productive. (and Yogi Berra!) “In theory there is no difference between theory and practice. But in practice there is.” The first thing that quickly became apparent from the early GPTP workshops is that practice