COOPERATIVE ALGORITHMS FOR A TEAM OF AUTONOMOUS UNDERWATER VEHICLES TAN YEW TECK (B.Sc.(Hons), M. Eng.) A THESIS SUBMITTED FOR THE DEGREE OF DOCTOR OF PHILOSOPHY ELECTRICAL AND COMPUTER ENGINEERING NATIONAL UNIVERSITY OF SINGAPORE 2014 DECLARATION I hereby declare that this thesis is my original work and it has been written by me in its entirety. I have duly acknowledged all the sources of information which have been used in the thesis. This thesis has also not been submitted for any degree in any university previously. Signed: TAN YEW TECK Date: 25 - 11 - 2014 i “Learn from yesterday, live for today, hope for tomorrow. The important thing is not to stop questioning.” Albert Einstein Acknowledgements This thesis could not have been completed without the help and support of many friends and colleagues for the last four year in NUS and MIT. First, I would like to thank my thesis advisor, Dr. Mandar Chitre for allowing me to carry out the PhD studies under his supervision. His technical guidance, support, encouragement and expertise has proved invaluable. Furthermore, I very much appreciate him for sacrificing so much of his personal time in helping me either in software design or hardware development. One of the most exciting periods during my PhD studies was when he allowed me to lead a team to design and build an unmanned surface vehicle for water quality monitoring project. I picked up many skills and experiences that otherwise would not have been obtained from my research topic. I would also like to thank Professor Nicholas Patrikalakis for helping me to secure the SMART PhD fellowship. This work would not be possible without the support of the funding. Not forgetting the members of Acoustic Research Laboratory (ARL), especially the STARFISH project team: Koay Teong Beng, Eng You Hong, Gao Rui, Chew Jee Loong, Bharath Kaylan, Shilabh Suman and Varadarajan Ganesan for their guidance and support while working with the STARFISH AUV. Their great company during numerous field trials made the experience more enjoyable. Many thanks to Dr. Venugopalan Pallayil and Mr. Mohan Panayamadam for making sure I got hold of all the tools that I needed, both software and hardware, for my research. The six months research residency in MIT last year was truly a great exposure and the most memorable experience throughout my PhD studies. Great thanks to Professor iii Franz Hover for agreeing to have me as a visiting scholar in his lab and allowing me to make use of the kayaks for experiments in the Charles River. Many thanks to members of the HoverGroup too: Mei Yi Cheung, Eric Gilbertson, Brooks Reed, Pedro Vaz Teixeira and Joshua Leighton for their help during the experiments in Boston. It has been a great pleasure knowing and working with them for the period in MIT. I would like to thank Dr. Kanna Rajan for the opportunity as visiting research scholar in Monterey Bay Aquarium Research Institute (MBARI). Although brief, the time spent in MBARI allowed me to get to know the T-REX reactive mission planner for AUVs. Thanks also to Dr. Frederic Py and Dr. Rishi Graham for their help and support during the stay in MBARI. Finally, I would also like to thank my foster father, Brian Kelly and my family for their love and continued support throughout this process. Without them, none of this work would have been possible. This work was funded by Singapore-MIT Alliance for Research and Technology (SMART) PhD fellowship. Contents DECLARATION i Acknowledgements iii Table of Contents v Summary viii List of Tables x List of Figures xi List of Abbreviations xvii List of Symbols xix Introduction 1.1 Autonomous Underwater Vehicles 1.2 Motivation . . . . . . . . . . . . . 1.3 Objectives . . . . . . . . . . . . . 1.4 Thesis Contributions . . . . . . . 1.5 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Background 2.1 Cooperative Positioning . . . . . 2.2 Bathymetry-based Localization . 2.3 Command and Control Systems 2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 10 14 17 20 Cooperative Positioning with a Single Moving Beacon 3.1 Cooperative Positioning using Acoustic Ranging . 3.2 Problem Formulation . . . . . . . . . . . . . . . . 3.3 Markov Decision Processes . . . . . . . . . . . . . 3.4 Policy Learning . . . . . . . . . . . . . . . . . . . 3.4.1 Cross-Entropy Method . . . . . . . . . . . 3.4.2 Variable-Length Genetic Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 23 25 28 29 30 36 . . . . v Table of Contents 3.5 . . . . . . . . . 43 44 46 47 49 49 51 57 58 Cooperative Bathymetry-based Localization 4.1 The Concept of Cooperative Bathymetry-based Localization . . . . . . 4.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 Process and Measurement Models . . . . . . . . . . . . . . . . 4.2.2 Marginalized Particle Filter . . . . . . . . . . . . . . . . . . . 4.3 Measurement Model for Cooperative Localization . . . . . . . . . . . . 4.3.1 Localization in Single-vehicle . . . . . . . . . . . . . . . . . . 4.3.2 Localization in Multiple Vehicles . . . . . . . . . . . . . . . . 4.4 Simualtions and Results . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.1 Measurement Models . . . . . . . . . . . . . . . . . . . . . . . 4.4.2 Influence of Communication Bandwidth . . . . . . . . . . . . . 4.4.3 Importance of Acoustic Communication and Bathymetry Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.4 Influence of Simulated Ocean Current . . . . . . . . . . . . . . 4.4.5 Influence of Compass and Thruster Biases . . . . . . . . . . . . 4.4.6 Influence of Bathymetry Map Resolution . . . . . . . . . . . . 4.5 Field Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.1 Charles River Basin, Boston . . . . . . . . . . . . . . . . . . . 4.5.2 St. John Island, Singapore . . . . . . . . . . . . . . . . . . . . 4.6 Sensitivity Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6.1 Influence of Ranging Frequency and Success Rate . . . . . . . 4.6.2 Influence of Sensor Noise Level . . . . . . . . . . . . . . . . . 4.7 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 60 62 62 62 66 67 69 73 76 77 3.6 3.7 3.8 Simulation . . . . . . . . . . . . . . . . . . . . . . . . 3.5.1 Supporting Single Survey AUV . . . . . . . . 3.5.2 Supporting Multiple Survey AUVs . . . . . . . 3.5.3 Position Estimation of the Survey AUV . . . . Field Experiments . . . . . . . . . . . . . . . . . . . . 3.6.1 Cooperative Positioning with Geo-fence . . . . 3.6.2 Cooperative Positioning around Coastal Waters Discussion . . . . . . . . . . . . . . . . . . . . . . . . Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Command and Control System for Autonomous Underwater Vehicles 5.1 Heirarchical Agent-based Control Architecture . . . . . . . . . . . 5.1.1 Agents Responsibilities . . . . . . . . . . . . . . . . . . . . 5.1.2 Backseat Driver Paradigm . . . . . . . . . . . . . . . . . . 5.2 Software Architecture . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.1 Command and Control Agents . . . . . . . . . . . . . . . . 5.2.2 Mission Planning . . . . . . . . . . . . . . . . . . . . . . . 5.2.3 Mission Execution . . . . . . . . . . . . . . . . . . . . . . 5.3 Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi . . . . . . . . . . . . . . . . . . . . . . . . . 79 84 85 86 87 89 90 94 94 95 96 97 98 99 100 103 106 107 110 111 113 Table of Contents 5.4 5.5 5.6 Field Experiments . . . . . . . . . . . 5.4.1 System Identification Mission 5.4.2 Surveying Mission . . . . . . 5.4.3 Adaptive Mission . . . . . . . Discussion . . . . . . . . . . . . . . . Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 116 117 120 124 126 Conclusions and Future Research 128 6.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 6.2 Future Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 Appendix A Error Estimate Covariance Due to Range Updates 132 Appendix B Command and Control System Software Specifications 135 Appendix C Java Command and Control System Developer Guide 139 Bibliography 162 Publications 173 vii Summary Multi-vehicle missions offer several advantages over single-vehicle missions in terms of mission complexity and tolerance to single-vehicle failure. However, missions involving multiple underwater vehicles pose two main challenges – the absence of a reliable positioning reference (GPS) and the extremely limited communication bandwidth among the vehicles – both of which limit the application of multi-vehicle cooperation techniques that are commonly used by their land and aerial counterparts. This thesis develops two cooperative algorithms for a team of Autonomous Underwater Vehicles (AUVs) that address the challenges. First, we design a cooperative navigation strategy for a beacon vehicle to serve as navigation beacon for a team of AUVs. The exchange of navigation information between the beacon and other vehicles improves their individual position estimates. We propose dynamic positioning algorithms for the beacon vehicle and analyse their performances in minimizing the position errors of other vehicles in the team. Second, given the bathymetric terrain maps, we develop cooperative localization using a team of sensor-limited AUVs. The localization of each vehicle is performed via decentralized particle filtering on its bathymetric measurements, assisted by acoustic range and information obtained from peer vehicles through acoustic communication. We extend the filter of an individual vehicle to incorporate information received from another vehicle to better estimate its position, and investigate the impact of communication interval, sensor noise and biases on the localization performance. Summary Designing a Command and Control (C2) system for a single AUV that is robust and easily extensible to accommodate the requirements of multi-vehicle cooperative missions is another focus of the thesis. In particular, we develop a hierarchical agentbased C2 system for a low-cost modular AUV - the STARFISH AUV - that allocates mission, navigation and vehicle tasks to individual self-contained agents. The collective interactions among the pool of agents enables the AUV to achieve its mission objectives autonomously. The C2 system has been developed and successfully deployed for various single-vehicle, adaptive missions as well as multi-vehicle cooperative missions. Using both simulations and field testings, we demonstrate the feasibility and capability of the developed algorithms in minimizing the position errors accumulated by the AUVs during mission execution. ix 5.8 Command and Control (C2) The C2 panel allows the operator to send different C2 command to the vehicle. The Destination field must match to the name of the vehicle specified in Listing line number 6. Figure 16: The command and control panel. 5.9 Map Viewer The map viewer shows the current location of the vehicle during the simulation. The trajectory of the vehicle is shown in yellow dot/lines in the map, while the vehicle’s current location (x and y) is shown at the right top corner of the window. 21 Figure 17: The map viewer showing the location of the vehicle (yellow dot/lines) and the location of the vehicle in the map (two numbers on the right top corner of the window). 5.10 LogFile Extraction Once the simulation has completed, the user can copy the log files to their desired directory. This can be done by clicking on the Administration>Extract Logs menu. Once the user has chosen the destination directory, all the files in the underlying “logs” directory inside the StarControl.app will be copied to the specified location. The content of the simLog folder are : 1. c2log-⇤.log : The main log file of the JC2 Agents. 2. m⇤ : Folders containing all the logs extracted from the log-0.txt. 3. time.txt : file contains all the start and end time for each of the missions. 4. guilog-0.txt : The log from the GUI agent. Only used for debugging purpose. The format of the files in the m⇤ folder are as follows: 1. mPoints.txt : The planned mission points. 2. waypt.txt : The planned way points. 22 3. AuvBearing.txt : The vehicle’s positions, bearings and distances to the next way points. 4. m⇤.txt : The original c2 logs of the particular mission. with the extracted data, one can easily plot the resultant trajectories using any plotting programs. Figure 18: Folder “simLog” contains the original log files and the extracted log files. 23 Bibliography [1] J. Elvander and G. Hawkes. ROVs and AUVs in support of marine renewable technologies. In Oceans, 2012, pages 1–6, Oct 2012. [2] C. von Alt, B. Allen, T. Austin, and R. Stokey. Remote environmental measuring units. In Autonomous Underwater Vehicle Technology, 1994. AUV ’94., Proceedings of the 1994 Symposium on, pages 13–19, Jul 1994. [3] T.B. Koay, Y.T. Tan, Y.H. Eng, R. Gao, Mandar Chitre, J.L. Chew, N. Chandhavarkar, R.R. Khan, T. Taher, and J. Koh. STARFISH – a small team of autonomous robotic fish. Indian Journal of Geo-Marine Sciences, 20(2):157–167, April 2011. [4] C.C. Eriksen, T.J. Osse, R.D. Light, T. Wen, T.W. Lehman, P.L. Sabin, J.W. Ballard, and AM. Chiodi. Seaglider: a long-range autonomous underwater vehicle for oceanographic research. Oceanic Engineering, IEEE Journal of, 26(4):424–436, Oct 2001. ISSN 0364-9059. [5] J. Sherman, R. Davis, W. B. Owens, and J. Valdes. The autonomous underwater glider “spray”. Oceanic Engineering, IEEE Journal of, 26(4):437–446, Oct 2001. ISSN 0364-9059. [6] Amy Nevala. A glide across the gulf stream. OCEANUS, 44(1), June 2005. [7] W. J. Kirkwood. Development of the DORADO mapping vehicle for multibeam, subbottom, and sidescan science missions. Journal of Field Robotics, 24(6):487– 495, 2007. ISSN 1556-4967. doi: 10.1002/rob.20191. 162 Bibliography [8] P. Wadhams and M. J. Doble. Digital terrain mapping of the underside of sea ice from a small AUV. Geophysical Research Letters, 35(1), 2008. ISSN 1944-8007. doi: 10.1029/2007GL031921. [9] StefanB. Williams, Oscar Pizarro, Michael Jakuba, and Neville Barrett. AUV benthic habitat mapping in south eastern Tasmania. In Andrew Howard, Karl Iagnemma, and Alonzo Kelly, editors, Field and Service Robotics, volume 62 of Springer Tracts in Advanced Robotics, pages 275–284. Springer Berlin Heidelberg, 2010. ISBN 978-3-642-13407-4. doi: 10.1007/978-3-642-13408-1 25. [10] Yanwu Zhang, J.G. Bellingham, M.A. Godin, and J.P. Ryan. Using an autonomous underwater vehicle to track the thermocline based on peak-gradient detection. Oceanic Engineering, IEEE Journal of, 37(3):544 –553, july 2012. ISSN 03649059. doi: 10.1109/JOE.2012.2192340. [11] B. Anderson and J. Crowell. Workhorse AUV - a cost-sensible new autonomous underwater vehicle for surveys/soundings, search, rescue, and research. In OCEANS, 2005. Proceedings of MTS/IEEE, pages 1–6, Sept 2005. doi: 10.1109/ OCEANS.2005.1639923. [12] A. Matos, N. Cruz, A. Martins, and F. Lobo Pereira. Development and implementation of a low-cost LBL navigation system for an AUV. In OCEANS ’99 MTS/IEEE. Riding the Crest into the 21st Century, volume 2, pages 774 –779 vol.2, 1999. doi: 10.1109/OCEANS.1999.804906. [13] P. Rigby, O. Pizarro, and S.B. Williams. Towards geo-referenced AUV navigation through fusion of USBL and DVL measurements. In OCEANS 2006, pages –6, 2006. doi: 10.1109/OCEANS.2006.306898. [14] A. Alcocer, P. Oliveira, and A. Pascoal. Study and implementation of an EKF GIB-based underwater positioning system. Control Engineering Practice, 15(6): 689 – 701, 2007. ISSN 0967-0661. doi: 10.1016/j.conengprac.2006.04.001. [15] Sarah E Webster, Ryan M Eustice, Hanumant Singh, and Louis L Whitcomb. Advances in single-beacon one-way-travel-time acoustic navigation for underwater vehicles. The International Journal of Robotics Research, 31(8):935–950, 2012. 163 Bibliography [16] J. C. Alleyne. Position estimation from range only measurements. Master’s thesis, Naval Postgraduate School, Monterey CA, September 2000. [17] Alexander Bahr, John J. Leonard, and Maurice F. Fallon. Cooperative localization for autonomous underwater vehicles. The International Journal of Robotics Research, 28(6):714–728, 2009. [18] Maurice F Fallon, Georgios Papadopoulos, John J Leonard, and Nicholas M Patrikalakis. Cooperative AUV Navigation using a Single Maneuvering Surface Craft. The International Journal of Robotics Research. [19] A.P. Scherbatyuk. The AUV positioning using ranges from one transponder LBL. In OCEANS ’95. MTS/IEEE. Challenges of Our Changing Global Environment. Conference Proceedings., volume 3, pages 1620–1623 vol.3, Oct 1995. [20] J. Hartsfiel. Single transponder range only navigation geometry (STRONG) applied to REMUS autonomous under water vehicles. Master’s thesis, MIT, 2005. [21] A. Bahr, J.J. Leonard, and A. Martinoli. Dynamic positioning of beacon vehicles for cooperative underwater navigation. In Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on, pages 3760–3767, 2012. doi: 10. 1109/IROS.2012.6386168. [22] M. Chitre. Path planning for cooperative underwater range-only navigation using a single beacon. In Autonomous and Intelligent Systems (AIS), 2010 International Conference on, pages –6, 2010. doi: 10.1109/AIS.2010.5547044. [23] D.K. Meduna, S.M. Rock, and R.S. McEwen. Closed-loop terrain relative navigation for AUVs with non-inertial grade navigation sensors. In Autonomous Underwater Vehicles (AUV), 2010 IEEE/OES, pages 1–8, 2010. [24] S. Carreno, P. Wilson, P. Ridao, and Y. Petillot. A survey on terrain based navigation for AUVs. In OCEANS 2010, pages 1–7, 2010. doi: 10.1109/OCEANS.2010. 5664372. [25] Bharath Kalyan and Mandar Chitre. A feasibility analysis on using bathymetry for navigation of autonomous underwater vehicles. In Proceedings of the 28th 164 Bibliography Annual ACM Symposium on Applied Computing, SAC ’13, pages 229–231, New York, NY, USA, 2013. ACM. ISBN 978-1-4503-1656-9. doi: 10.1145/2480362. 2480411. [26] I. Nygren and M. Jansson. Terrain navigation for underwater vehicles using the correlator method. Oceanic Engineering, IEEE Journal of, 29(3):906–915, 2004. ISSN 0364-9059. doi: 10.1109/JOE.2004.833222. [27] K.B. Anonsen and O. Hallingstad. Terrain aided underwater navigation using point mass and particle filters. In Position, Location, And Navigation Symposium, 2006 IEEE/ION, pages 1027–1035, April 2006. doi: 10.1109/PLANS.2006.1650705. [28] R. Karlsson and F. Gustafsson. Particle filter for underwater terrain navigation. In Statistical Signal Processing, 2003 IEEE Workshop on, pages 526 – 529, sept.-1 oct. 2003. doi: 10.1109/SSP.2003.1289507. [29] P.-J. Nordlund and F. Gustafsson. Sequential monte carlo filtering techniques applied to integrated navigation systems. In American Control Conference, 2001. Proceedings of the 2001, volume 6, pages 4375–4380 vol.6, 2001. doi: 10.1109/ACC.2001.945666. [30] T. Schon, F. Gustafsson, and P.-J. Nordlund. Marginalized particle filters for mixed linear/nonlinear state-space models. Signal Processing, IEEE Transactions on, 53 (7):2279–2289, 2005. ISSN 1053-587X. doi: 10.1109/TSP.2005.849151. [31] F Teixeira, Ant´onio Pascoal, and Pramod Maurya. A novel particle filter formulation with application to terrain-aided navigation. In Proc. IFAC Workshop on Navigation, Guidance and Control of Underwater Vehicles (NGCUV’2012), Porto, Portugal, pages 10–12, 2012. [32] Arnaud Doucet, Nando de Freitas, Kevin P. Murphy, and Stuart J. Russell. Raoblackwellised particle filtering for dynamic bayesian networks. In Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, UAI ’00, pages 176– 183, San Francisco, CA, USA, 2000. Morgan Kaufmann Publishers Inc. ISBN 1-55860-709-9. 165 Bibliography [33] Stephen Barkby, Stefan B. Williams, Oscar Pizarro, and Michael V. Jakuba. A featureless approach to efficient bathymetric SLAM using distributed particle mapping. Journal of Field Robotics, 28(1):19–39, 2011. ISSN 1556-4967. [34] Nathaniel Fairfield, George A Kantor, and David Wettergreen. Towards particle filter SLAM with three dimensional evidence grids in a flooded subterranean environment. In Proceedings of ICRA 2006, pages 3575 – 3580, May 2006. [35] N. Fairfield and D. Wettergreen. Active localization on the ocean floor with multibeam sonar. In OCEANS 2008, pages 1–10, 2008. doi: 10.1109/OCEANS.2008. 5151853. [36] G.T. Donovan. Position error correction for an autonomous underwater vehicle inertial navigation system (INS) using a particle filter. Oceanic Engineering, IEEE Journal of, 37(3):431 –445, July 2012. ISSN 0364-9059. doi: 10.1109/JOE.2012. 2190810. [37] C. Roman and H. Singh. Improved vehicle based multibeam bathymetry using sub-maps and SLAM. In Intelligent Robots and Systems, 2005. (IROS 2005). 2005 IEEE/RSJ International Conference on, pages 3662–3669, Aug 2005. [38] M. F. Fallon, M. Kaess, H. Johannsson, and J. J. Leonard. Efficient AUV navigation fusing acoustic ranging and side-scan sonar. In IEEE International Conference on Robotics and Automation (ICRA),Shanghai, China, May 2011. [39] Francisco Curado Teixeira, Jo˜ao Quintas, and Ant´onio Pascoal. AUV terrain-aided doppler navigation using complementary filtering. In Proc IFAC Conference on Manoeuvring and Control of Marine Craft (MCMC’2012), Arenzano, Italy, 2012. [40] Pramod Maurya, Francisco Curado Teixeira, and Ant´onio Pascoal. Complementary terrain/single beacon-based AUV navigation. In Proc IFAC Workshop on Navigation, Guidance and Control of Underwater Vehicles (NGCUV’2012), Porto, Portugal, pages 10–12, 2012. [41] Matt Rosencrantz, Geoffrey Gordon, and Sebastian Thrun. Decentralized sensor fusion with distributed particle filters. In Proceedings of the Nineteenth conference 166 Bibliography on Uncertainty in Artificial Intelligence, UAI’03, pages 493–500, San Francisco, CA, USA, 2003. Morgan Kaufmann Publishers Inc. ISBN 0-127-05664-5. [42] Bo Jiang and B. Ravindran. Completely distributed particle filters for target tracking in sensor networks. In Parallel Distributed Processing Symposium (IPDPS), 2011 IEEE International, pages 334–344, 2011. doi: 10.1109/IPDPS.2011.40. [43] Xiaohong Sheng, Yu-Hen Hu, and P. Ramanathan. Distributed particle fil- ter with GMM approximation for multiple targets localization and tracking in wireless sensor network. In Information Processing in Sensor Networks, 2005. IPSN 2005. Fourth International Symposium on, pages 181–188, 2005. doi: 10.1109/IPSN.2005.1440923. [44] H. Yavuz and A. Bradshaw. A new conceptual approach to the design of hybrid control architecture for autonomous mobile robots. Journal of Intelligent and Robotic Systems, 34(1):1–26, 2002. [45] Alexander M. Meystel and James Sacra Albus. Intelligent Systems: Architecture, Design, and Control. John Wiley & Sons, Inc., New York, NY, USA, 1st edition, 2000. ISBN 0471193747. [46] Ronald C. Arkin. An Behavior-based Robotics. MIT Press, Cambridge, MA, USA, 1st edition, 1998. ISBN 0262011654. [47] P. Ridao, J. Yuh, J. Batlle, and K. Sugihara. On AUV control architecture. In Intelligent Robots and Systems, 2000. (IROS 2000). Proceedings. 2000 IEEE/RSJ International Conference on, volume 2, pages 855–860 vol.2, 2000. doi: 10.1109/ IROS.2000.893126. [48] A. Yavnai. Architecture for an autonomous reconfigurable intelligent control system (ARICS). In Autonomous Underwater Vehicle Technology, 1996. AUV ’96., Proceedings of the 1996 Symposium on, pages 238–245, Jun 1996. doi: 10.1109/AUV.1996.532421. [49] A. Brooks, T. Kaupp, A. Makarenko, S. Williams, and A. Oreback. Towards component-based robotics. In Intelligent Robots and Systems, 2005. (IROS 2005). 167 Bibliography 2005 IEEE/RSJ International Conference on, pages 163–168, Aug 2005. doi: 10.1109/IROS.2005.1545523. [50] Morgan Quigley, Ken Conley, Brian Gerkey, Josh Faust, Tully Foote, Jeremy Leibs, Rob Wheeler, and Andrew Y Ng. ROS: an open-source robot operating system. ICRA workshop on open source software, 3(3.2), 2009. [51] Paul Michael Newman. MOOS-mission orientated operating suite. Massachusetts Institute of Technology, Tech. Rep, 2299(08), 2008. [52] Michael R. Benjamin, Henrik Schmidt, Paul M. Newman, and John J. Leonard. Nested autonomy for unmanned marine vehicles with MOOS-IvP. Journal of Field Robotics, 27(6):834–875, 2010. ISSN 1556-4967. doi: 10.1002/rob.20370. [53] Nicholas R. Jennings. On agent-based software engineering. Artificial Intelligence, 117(2):277 – 296, 2000. ISSN 0004-3702. doi: http://dx.doi.org/10.1016/ S0004-3702(99)00107-1. [54] Y. T. Tan and Mandar Chitre. Single beacon cooperative path planning using crossentropy method. In IEEE/MTS OCEANS, KONA, Hawaii, September 2011. [55] Y. T. Tan and Mandar Chitre. Direct policy search with variable-length genetic algorithm for single beacon cooperative path planning. In International Symposium on Distributed Autonomous Robotic Systems (DARS) 2012, Baltimore, Maryland, USA, November 2012. [56] Y. T. Tan, Rui Gao, and M. Chitre. Cooperative path planning for range-only localization using a single moving beacon. Oceanic Engineering, IEEE Journal of, 39(2):371–385, April 2014. ISSN 0364-9059. doi: 10.1109/JOE.2013.2296361. [57] Marco Wiering and Martijn van Otterlo. Reinforcement Learning: State of the Art, volume 12 of Adaptation, Learning, and Optimization. Springer Berlin Heidelberg, 2012. [58] RonaldJ. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8(3-4):229–256, 1992. 168 Bibliography [59] Jonathan Baxter, Peter L. Bartlett, and Lex Weaver. Experiments with infinitehorizon, policy-gradient estimation. J. Artif. Int. Res., 15(1):351–381, November 2001. ISSN 1076-9757. [60] Thomas Philip Runarsson. Learning heuristic policies – A reinforcement learning problem. In Learning and Intelligent Optimization, volume 6683 of Lecture Notes in Computer Science, pages 423–432. Springer Berlin Heidelberg, 2011. [61] Pieter tjerk De Boer, Dirk P. Kroese, Shie Mannor, and Reuven Y. Rubinstein. A tutorial on the cross-entropy method. Annals of Operations Research, 134:19–67. doi: 10.1007/s10479-005-5724-z. [62] Shie Mannor, Reuven Rubinstein, and Yohai Gat. The cross entropy method for fast policy search. In In International Conference on Machine Learning, pages 512–519. Morgan Kaufmann, 2003. [63] Warren B. Powell. Approximate Dynamic Programming: Solving the Curses of Dimensionality. Wiley Series in Probability and Statistics. Wiley, 2nd edition, 2011. [64] D E Goldberg. Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, 1989. [65] Jianping Tu and S.X. Yang. Genetic algorithm based path planning for a mobile robot. In Robotics and Automation, 2003. Proceedings. ICRA ’03. IEEE International Conference on, volume 1, pages 1221 – 1226, sept. 2003. doi: 10.1109/ROBOT.2003.1241759. [66] Chang Wook Ahn and R.S. Ramakrishna. A genetic algorithm for shortest path routing problem and the sizing of populations. Evolutionary Computation, IEEE Transactions on, 6(6):566 – 579, dec 2002. ISSN 1089-778X. doi: 10.1109/ TEVC.2002.804323. [67] Jun Wei, Haining Zheng, Haoliang Chen, Boon Hooi Ooi, M. H. Dao, Wonjoon Cho, Paola M. Rizzoli, P. Tkalich, and Nicholas M. Patrikalakis. Multi-layer model simulation and data assimilation in the Serangoon Harbor of Singapore. In International Offshore (Ocean) and Polar Engineering Conference, June 2010. 169 Bibliography [68] P.O. Arambel, Constantino Rago, and R.K. Mehra. Covariance intersection algorithm for distributed spacecraft state estimation. In American Control Conference, 2001. Proceedings of the 2001, volume 6, pages 4398–4403 vol.6, 2001. doi: 10.1109/ACC.2001.945670. [69] A. Bahr, M.R. Walter, and J.J. Leonard. Consistent cooperative localization. In IEEE International Conference on Robotics and Automation (ICRA), Kobe, Japan, May 2009. [70] Y. T. Tan, Mandar Chitre, and F. Hover. Collaborative bathymetry-based localization of a team of autonomous underwater vehicles. In Robotics and Automation (ICRA), 2014 IEEE International Conference on, May 2014. [71] F Teixeira. Terrain-Aided Navigation and Geophysical Navigation of Autonomous Underwater Vehicles. PhD thesis, Dynamical Systems and Ocean Robotics Lab, Lisbon, 2007. [72] Per-Johan Nordlund. Sequential Monte Carlo Filters and Integrated Navigation. PhD thesis, Linkopings universitet, 2002. [73] G. Grisetti, C. Stachniss, and W. Burgard. Improved techniques for grid mapping with Rao-Blackwellized particle filters. Robotics, IEEE Transactions on, 23(1): 34–46, Feb 2007. ISSN 1552-3098. [74] Jun S. Liu and Rong Chen. Sequential monte carlo methods for dynamic systems. Journal of the American Statistical Association, 93:1032–1044, 1998. [75] Paul Fearnhead. Sequential Monte Carlo methods in filter theory. PhD thesis, University of Oxford, 1998. [76] Thomas M. Cover and Joy A. Thomas. Elements of Information Theory 2nd Edition (Wiley Series in Telecommunications and Signal Processing). WileyInterscience, July 2006. ISBN 0471241954. [77] Oswald Lanz. An information theoretic rule for sample size adaptation in particle filtering. Image Analysis and Processing, International Conference on, 0:317–322, 2007. 170 Bibliography [78] S.E. Webster, J.M. Walls, L.L. Whitcomb, and R.M. Eustice. Decentralized extended information filter for single-beacon cooperative acoustic navigation: Theory and experiments. Robotics, IEEE Transactions on, 29(4):957–974, 2013. ISSN 1552-3098. doi: 10.1109/TRO.2013.2252857. [79] D.P. Eickstedt and S.R. Sideleau. The backseat control architecture for au- tonomous robotic vehicles: A case study with the Iver2 AUV. In OCEANS 2009, MTS/IEEE Biloxi - Marine Technology for Our Future: Global and Local Challenges, pages –8, 26-29 2009. [80] Y. T. Tan, Mandar Chitre, and Prahlad Vadakkepat. Hierarchical agent-based command and control system for autonomous underwater vehicles. In International Conference on Autonomous and Intelligent Systems (AIS) 2010, Povoa de Varzim, Portugal, June 2010. [81] Y. T. Tan and Mandar Chitre. Hierarchical multi-agent command and control system for autonomous underwater vehicles. In IEEE AUV 2012, Southampton, UK, September 2012. [82] M. Chitre. DSAAV - A distributed software architecture for autonomous vehicles. In OCEANS 2008, pages –10, sept. 2008. doi: 10.1109/OCEANS.2008. 5151848. [83] G. Hollinger and S. Singh. Multi-robot coordination with periodic connectivity. In Robotics and Automation (ICRA), 2010 IEEE International Conference on, pages 4457 –4462, May 2010. [84] C. McGann, F. Py, K. Rajan, H. Thomas, R. Henthorn, and R. McEwen. A deliberative architecture for AUV control. In Robotics and Automation, 2008. ICRA 2008. IEEE International Conference on, pages 1049 –1054, May 2008. [85] Mandar Chitre, R. Bhatnagar, and W.-S. Soh. UnetStack: an agent-based software stack and simulator for underwater networks. In IEEE OCEANS, St. John’s, Canada, September 2014. 171 Bibliography [86] Morgan Quigley, Ken Conley, Brian P. Gerkey, Josh Faust, Tully Foote, Jeremy Leibs, Rob Wheeler, and Andrew Y. Ng. ROS: an open-source robot operating system. In ICRA Workshop on Open Source Software, 2009. [87] P. M Newman. MOOS—A mission oriented operating suite (Tech. Rep. OE200307). Technical report, Department of Ocean Engineering, MIT, Cambridge, MA:, 2003. [88] Eng You Hong, Teo Kwong Meng, and Mandar Chitre. Online system identification of the dynamics of an autonomous underwater vehicle. In Underwater Technology Symposium (UT), 2013 IEEE International, pages 1–10, March 2013. doi: 10.1109/UT.2013.6519846. [89] P.S. Dias, G.M. Goncalves, R.M.F. Gomes, J.B. Sousa, J. Pinto, and F.L. Pereira. Mission planning and specification in the Neptus framework. In Robotics and Automation, 2006. ICRA 2006. Proceedings 2006 IEEE International Conference on, pages 3220–3225, May 2006. [90] Chee-Loon Ng, Schuyler Senft-Grupp, and Harold F Hemond. A multi-platform optical sensor for in situ sensing of water chemistry. Limnol. Oceanogr. Methods, 10:978–990, 2012. [91] M. Shaukat, Mandar Chitre, Y. T. Tan, and Ashish Raste. Bio-CAST: A bioinspired control algorithm for small team of robots using implicit communication for cooperative source localization. Bioinspired & Biomimetics (Submitted), 2014. [92] Frederic Py, Kanna Rajan, and Conor McGann. A systematic agent framework for situated autonomous systems. In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: Volume 2, AAMAS ’10, pages 583–590, Richland, SC, 2010. International Foundation for Autonomous Agents and Multiagent Systems. ISBN 0-98265-712-9. 172 Publications [1] Mansoor Shaukat, Mandar Chitre, Y. T. Tan, and Ashish Raste. Bio-CAST: A bioinspired control algorithm for small team of robots using implicit communication for cooperative source localization. Bioinspired & Biomimetics (Submitted), 2014. [2] Y. T. Tan, Mandar Chitre, and Franz Hover. Collaborative bathymetry-based localization of a team of autonomous underwater vehicles. In Robotics and Automation (ICRA), 2014 IEEE International Conference on, May 2014. [3] Y. T. Tan, R. Gao, and Mandar Chitre. Cooperative path planning for range-only localization using a single moving beacon. Oceanic Engineering, IEEE Journal of, 39(2):371–385, April 2014. ISSN 0364-9059. doi: 10.1109/JOE.2013.2296361. [4] Y. T. Tan and Mandar Chitre. Direct policy search with variable-length genetic algorithm for single beacon cooperative path planning. In International Symposium on Distributed Autonomous Robotic Systems (DARS) 2012, Baltimore, Maryland, USA, November 2012. [5] Y. T. Tan and Mandar Chitre. Hierarchical multi-agent command and control system for autonomous underwater vehicles. In IEEE AUV 2012, Southampton, UK, September 2012. [6] T. B. Koay, Y. T. Tan, Y. H. Eng, R. Gao, Mandar Chitre, J. L. Chew, N. Chandhavarkar, R. Khan, T. Taher, and J. Koh. STARFISH - A small team of autonomous robotics fish. In 3rd International Conference on Underwater System Technology: Theory and Applications 2010, (Cyberjaya, Malaysia), November 2011. [7] Y. T. Tan and Mandar Chitre. Single beacon cooperative path planning using crossentropy method. In IEEE/MTS OCEANS, KONA, Hawaii, September 2011. 173 Publications [8] T.B. Koay, Y.T. Tan, Y.H. Eng, R. Gao, Mandar Chitre, J.L. Chew, N. Chandhavarkar, R.R. Khan, T. Taher, and J. Koh. STARFISH – A small team of autonomous robotic fish. Indian Journal of Geo-Marine Sciences, 20(2):157–167, April 2011. [9] J. L. Chew, T. B. Koay, Y. T. Tan, Y. H. Eng, R. Gao, Mandar Chitre, and N. Chandhavarkar. STARFISH: An Open-Architecture AUV and its Applications. Defence Technology Asia, 2011, February 2011. [10] Y. T. Tan, Mandar Chitre, and Prahlad Vadakkepat. Hierarchical agent-based command and control system for autonomous underwater vehicles. In International Conference on Autonomous and Intelligent Systems (AIS) 2010, Povoa de Varzim, Portugal, June 2010. 174 [...]... Altitude(t-1) Altitude(t) (a) (b) F IGURE 2.1: Different approaches for measurement model’s update stage (a) Sequential approach (b) Batch approach from [26] 2.2 Bathymetry-based Localization Bathymetry-based localization and navigation, also known as Terrain Relative Navigation (TRN) [23], Terrain-aided Navigation (TAN) [24], and Bathymetric-aided Navigation (BAN) [25] has been used for decades in aircraft and... MPF has been employed in [29], in an integrated navigation system of an aircraft with a state vector of more than 15 dimensions, and simulation results showed good performance with a much lower computational load In the domain of underwater navigation, the authors in [31] have shown the feasibility of applying the MPF for an AUV with a particle set as low as 500 and was able to achieve good localization... underside of sea ice [8] mapping More recently, cameras have also been attached to AUVs for mapping coral reefs around shallow waters [9] Due to strong attenuation of light underwater, the camera can only capture a small area at a time A complete picture can obtained 3 Chapter 1 Introduction by mosaicking a series of pictures taken around the coral reefs Elsewhere, in order to understand the evolution of. .. using a single moving beacon, and presents the formulation of the beacon’s path planning policy within a MDP framework Two approaches are adopted to automatically learn the resulting policy: the cross-entropy method and the variable-length genetic algorithm Simulation and field trial results are also presented Chapter 4 presents cooperative localization of a team of AUVs using terrain information from a. .. the water column between the start and the end point of a mission Examples of buoyancy-driven AUVs (Fig 1.2) are the Seaglider [4] and Spray glider [5] This class of AUVs is capable of cruising around 0.2-0.5 m/s, and covering a range of 6000 km [6] Apart from ocean exploration, AUVs have been used for a wide range of applications AUVs equipped with sonar systems are deployed for sea floor [7] and underside... ocean 1 National Oceanic and Atmospheric Administration – Ocean http://www.noaa.gov/ocean.html 1 Chapter 1 Introduction In recent years, the advancement in the Autonomous Underwater Vehicles (AUVs) technology provides an attractive alternative They require less efforts to operate, and the cost of maintenance is marginal compared to those of manned vessels Furthermore, the levels of autonomy that can... indicator of the performance of the localization filter, it is not the focus of this thesis Often a particle filter is designed to estimate and track a large number of system variables which requires a large number of particles for the filter to converge This poses a challenge for the AUVs’ limited computational power onboard In order to alleviate this, a number of researchers have adopted an approach called... that its position broadcasts can be used to minimize the uncertainties in the position estimates of a team of low-cost, sensor-limited AUVs 2 To develop a cooperative localization algorithm using terrain information and acoustic communications among a team of low-cost, sensor-limited AUVs 2 Software-In-The-Loop simulation allows an actual system software to be tested in a simulation environment, before... this day, due to the lack of available data According to NOAA 1 , More than 70 % of the Earth’s surface is covered by the ocean, yet only about 5 % has been explored by humans Classical ocean exploration relies on static buoys, manned surface and underwater vehicles The high cost and substantial deployment and retrieval efforts have limited their effectiveness in exploring and gathering scientific data... filters’ particle distribution and assist the position estimation 5 Empirical studies of the impact of various parameters on the performance of the cooperative localization filter 6 A hierarchical agent-based C2 system for a single AUV that is robust and easily extensible to accommodate the requirements of multi-vehicle cooperative missions The C2 system that clearly allocates mission, navigation and vehicle . Many thanks to Dr. Venu- gopalan Pallayil and Mr. Mohan Panayamadam for making sure I got hold of all the tools that I needed, both software and hardware, for my research. The six months research. Resampling SLAM Simultaneous Localization and Mapping TAN Terrain Aided Navigation TOA Time of Arrival TOT Time of Transmission TWTT Two Way Travel Time USBL Utra-Short Baseline VLGA Variable-length. cooperative navigation strategy for a beacon vehicle to serve as navigation beacon for a team of AUVs. The exchange of navigation information between the beacon and other vehicles improves their individual position