Early Praise for Genetic Algorithms and Machine Learning for Programmers I really like the book; it’s pitched nicely at the interested beginner and doesn’t make undue assumptions about background knowledge ➤ Burkhard Kloss Director, Applied Numerical Research Labs A unique take on the subject and should very much appeal to programmers looking to get started with various machine learning techniques ➤ Christopher L Simons Senior Lecturer, University of the West of England, Bristol, UK Turtles, paper bags, escape, AI, fun whilst learning: it’s turtles all the way out ➤ Russel Winder Retired Consultant, Self-Employed This book lifts the veil on the complexity and magic of machine learning techniques for ordinary programmers Simple examples and interactive programs really show you not just how these algorithms work, but bring real-world problems to life ➤ Steve Love Programmer, Freelance We've left this page blank to make the page numbers the same in the electronic and paper books We tried just leaving it out, but then people wrote us to ask about the missing pages Anyway, Eddy the Gerbil wanted to say “hello.” Genetic Algorithms and Machine Learning for Programmers Create AI Models and Evolve Solutions Frances Buontempo The Pragmatic Bookshelf Raleigh, North Carolina Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks Where those designations appear in this book, and The Pragmatic Programmers, LLC was aware of a trademark claim, the designations have been printed in initial capital letters or in all capitals The Pragmatic Starter Kit, The Pragmatic Programmer, Pragmatic Programming, Pragmatic Bookshelf, PragProg and the linking g device are trademarks of The Pragmatic Programmers, LLC Every precaution was taken in the preparation of this book However, the publisher assumes no responsibility for errors or omissions, or for damages that may result from the use of information (including program listings) contained herein Our Pragmatic books, screencasts, and audio books can help you and your team create better software and have more fun Visit us at https://pragprog.com The team that produced this book includes: Publisher: Andy Hunt VP of Operations: Janet Furlow Managing Editor: Susan Conant Development Editor: Tammy Coron Copy Editor: Jasmine Kwityn Indexing: Potomac Indexing, LLC Layout: Gilson Graphics For sales, volume licensing, and support, please contact support@pragprog.com For international rights, please contact rights@pragprog.com Copyright © 2019 The Pragmatic Programmers, LLC All rights reserved No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form, or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior consent of the publisher ISBN-13: 978-1-68050-620-4 Book version: P1.0—January 2019 Contents Preface ix Escape! Code Your Way Out of a Paper Bag Let’s Begin Your Mission: Find a Way Out How to Help the Turtle Escape Let’s Save the Turtle Did It Work? Over to You 11 13 Decide! Find the Paper Bag Your Mission: Learn from Data How to Grow a Decision Tree Let’s Find That Paper Bag Did It Work? Over to You 15 16 18 23 27 31 Boom! Create a Genetic Algorithm Your Mission: Fire Cannonballs How to Breed Solutions Let’s Fire Some Cannons Did It Work? Over to You 33 35 37 40 48 53 Swarm! Build a Nature-Inspired Swarm Your Mission: Crowd Control How to Form a Swarm Let’s Make a Swarm Did It Work? Over to You 57 58 66 69 76 78 Contents • vi Colonize! Discover Pathways Your Mission: Lay Pheromones How to Create Pathways Let’s March Some Ants Did It Work? Over to You 79 80 83 85 93 97 Diffuse! Employ a Stochastic Model Your Mission: Make Small Random Steps How to Cause Diffusion Let’s Diffuse Some Particles Did It Work? Over to You 99 100 109 111 119 125 Buzz! Converge on One Solution Your Mission: Beekeeping How to Feed the Bees Let’s Make Some Bees Swarm Did It Work? Over to You 127 128 131 133 143 145 Alive! Create Artificial Life Your Mission: Make Cells Come Alive How to Create Artificial Life Let’s Make Cellular Automata Did It Work? Over to You 147 149 152 154 160 161 Dream! Explore CA with GA Your Mission: Find the Best How to Explore a CA Let’s Find the Best Starting Row Did It Work? Over to You 163 164 167 169 181 185 10 Optimize! Find the Best Your Mission: Move Turtles How to Get a Turtle into a Paper Bag Let’s Find the Bottom of the Bag Did It Work? Extension to More Dimensions Over to You 187 188 189 193 198 203 205 Contents Bibliography Index • vii 207 209 Preface Have you ever heard the phrase “Coding your way out of a paper bag”? In this book, you’ll exactly that In each chapter, you’ll examine different machine learning techniques that you can use to programmatically get particles, ants, bees, and even turtles out of a paper bag While the metaphor itself may be silly, it’s a great way to demonstrate how algorithms find solutions over time Who Is This Book For? If you’re a beginner to intermediate programmer keen to understand machine learning, this book is for you Inside its pages, you’ll create genetic algorithms, nature-inspired swarms, Monte Carlo simulations, cellular automata, and clusters You’ll also learn how to test your code as you dive into even more advanced topics Experts in machine learning may still enjoy the “programming out of a paper bag” metaphor, though they are unlikely to learn new things What’s in This Book? In this book, you will: • • • • • • • • Use heuristics and design fitness functions Build genetic algorithms Make nature-inspired swarms with ants, bees, and particles Create Monte Carlo simulations Investigate cellular automata Find minima and maxima using hill climbing and simulated annealing Try selection methods, including tournament and roulette wheels Learn about heuristics, fitness functions, metrics, and clusters You’ll also test your code, get inspired to solve new problems, and work through scenarios to code your way out of a paper bag—an important skill for any competent programmer Beyond that, you’ll see how the algorithms report erratum • discuss