Lecture Notes in Electrical Engineering 501 Sukhan Lee · Hanseok Ko Songhwai Oh   Editors Multisensor Fusion and Integration in the Wake of Big Data, Deep Learning and Cyber Physical System An Edition of the Selected Papers from the 2017 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI 2017) Lecture Notes in Electrical Engineering Volume 501 Board of Series editors Leopoldo Angrisani, Napoli, Italy Marco Arteaga, Coyoacán, México Bijaya Ketan Panigrahi, New Delhi, India Samarjit Chakraborty, München, Germany Jiming Chen, Hangzhou, P.R China Shanben Chen, Shanghai, China Tan Kay Chen, Singapore, Singapore Rüdiger Dillmann, Karlsruhe, Germany Haibin Duan, Beijing, China Gianluigi Ferrari, Parma, Italy Manuel Ferre, Madrid, Spain Sandra Hirche, München, Germany Faryar Jabbari, Irvine, USA Limin Jia, Beijing, China Janusz Kacprzyk, Warsaw, Poland Alaa Khamis, New Cairo City, Egypt Torsten Kroeger, Stanford, USA Qilian Liang, Arlington, USA Tan Cher Ming, Singapore, Singapore Wolfgang Minker, Ulm, Germany Pradeep Misra, Dayton, USA Sebastian Möller, Berlin, Germany Subhas Mukhopadyay, Palmerston North, New Zealand Cun-Zheng Ning, Tempe, USA Toyoaki Nishida, Kyoto, Japan Federica Pascucci, Roma, Italy Yong Qin, Beijing, China Gan Woon Seng, Singapore, Singapore Germano Veiga, Porto, Portugal Haitao Wu, Beijing, China Junjie James Zhang, Charlotte, USA ** Indexing: The books of this series are submitted to ISI Proceedings, EI-Compendex, SCOPUS, MetaPress, Springerlink ** Lecture Notes in Electrical Engineering (LNEE) is a book series which reports the latest research and developments in Electrical Engineering, namely: • • • • • • Communication, Networks, and Information Theory Computer Engineering Signal, Image, Speech and Information Processing Circuits and Systems Bioengineering Engineering The audience for the books in LNEE consists of advanced level students, researchers, and industry professionals working at the forefront of their fields Much like Springer’s other Lecture Notes series, LNEE will be distributed through Springer’s print and electronic publishing channels For general information about this series, comments or suggestions, please use the contact address under “service for this series” To submit a proposal or request further information, please contact the appropriate Springer Publishing Editors: Asia: China, Jessie Guo, Assistant Editor (jessie.guo@springer.com) (Engineering) India, Swati Meherishi, Senior Editor (swati.meherishi@springer.com) (Engineering) Japan, Takeyuki Yonezawa, Editorial Director (takeyuki.yonezawa@springer.com) (Physical Sciences & Engineering) South Korea, Smith (Ahram) Chae, Associate Editor (smith.chae@springer.com) (Physical Sciences & Engineering) Southeast Asia, Ramesh Premnath, Editor (ramesh.premnath@springer.com) (Electrical Engineering) South Asia, Aninda Bose, Editor (aninda.bose@springer.com) (Electrical Engineering) Europe: Leontina Di Cecco, Editor (Leontina.dicecco@springer.com) (Applied Sciences and Engineering; Bio-Inspired Robotics, Medical Robotics, Bioengineering; Computational Methods & Models in Science, Medicine and Technology; Soft Computing; Philosophy of Modern Science and Technologies; Mechanical Engineering; Ocean and Naval Engineering; Water Management & Technology) (christoph.baumann@springer.com) (Heat and Mass Transfer, Signal Processing and Telecommunications, and Solid and Fluid Mechanics, and Engineering Materials) North America: Michael Luby, Editor (michael.luby@springer.com) (Mechanics; Materials) More information about this series at http://www.springer.com/series/7818 Sukhan Lee Hanseok Ko Songhwai Oh • Editors Multisensor Fusion and Integration in the Wake of Big Data, Deep Learning and Cyber Physical System An Edition of the Selected Papers from the 2017 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI 2017) 123 Editors Sukhan Lee Intelligent Systems Research Institute Sungkyunkwan University Suwon Korea (Republic of) Songhwai Oh Department of Electrical and Computer Engineering Seoul National University Seoul Korea (Republic of) Hanseok Ko School of Electrical Engineering Korea University Seoul Korea (Republic of) ISSN 1876-1100 ISSN 1876-1119 (electronic) Lecture Notes in Electrical Engineering ISBN 978-3-319-90508-2 ISBN 978-3-319-90509-9 (eBook) https://doi.org/10.1007/978-3-319-90509-9 Library of Congress Control Number: 2018940915 © Springer International Publishing AG, part of Springer Nature 2018 This work is subject to copyright All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed The use of general descriptive names, registered names, trademarks, service marks, etc in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations Printed on acid-free paper This Springer imprint is published by the registered company Springer International Publishing AG part of Springer Nature The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland Preface Multisensor fusion and integration is playing a critical role in harnessing the smart technologies as we ride the big wave of the 4th Industrial Revolution Deployment of the Internet of Things, Cyber-Physical Systems and Robotics in distributed environment is rapidly rising as our society seeks to transition from being ambient to being smart and, at the same time, to enable human to curate information and knowledge between ubiquitous and collective computing environments What surround us are the networks of sensors and actuators that monitor our environment, health, security and safety, as well as the service robots, intelligent vehicles and autonomous systems of ever heightened autonomy and dependability with integrated heterogeneous sensors and actuators Developing fundamental theories and advancing implementation tools to address the emerging key issues in multisensor fusion and integration in the wake of big data and deep learning would make the above transition smooth and rewarding This volume is an edition of the papers selected from the 13th IEEE International Conference on Multisensor Integration and Fusion, IEEE MFI 2017, held in Daegu, Korea, 16–22 November 2017 Only 17 papers out of the 112 papers accepted for IEEE MFI 2017 were chosen and requested for revision and extension to be included in this volume The 17 contributions to this volume are organized into two chapters: Chapter is dedicated to the theories in data and information fusion in distributed environment and Chapter to the multisensor fusion in robotics To help readers understand better, a chapter summary is included in each chapter as an introduction It is the wish of the editors that readers find this volume informative and enjoyable We would also like to thank Springer-Verlag for undertaking the publication of this volume Sukhan Lee Hanseok Ko Songhwai Oh v Contents Multi-sensor Fusion: Theory and Practice Covariance Projection as a General Framework of Data Fusion and Outlier Removal Sukhan Lee and Muhammad Abu Bakr State Estimation in Networked Control Systems with Delayed and Lossy Acknowledgments Florian Rosenthal, Benjamin Noack, and Uwe D Hanebeck 22 Performance of State Estimation and Fusion with Elliptical Motion Constraints Qiang Liu and Nageswara S V Rao 39 Relevance and Redundancy as Selection Techniques for Human-Autonomy Sensor Fusion Justin D Brody, Anna M R Dixon, Daniel Donavanik, Ryan M Robinson, and William D Nothwang Classification of Reactor Facility Operational State Using SPRT Methods with Radiation Sensor Networks Camila Ramirez and Nageswara S V Rao Improving Ego-Lane Detection by Incorporating Source Reliability Tran Tuan Nguyen, Jens Spehr, Jonas Sitzmann, Marcus Baum, Sebastian Zug, and Rudolf Kruse 52 76 98 Applying Knowledge-Based Reasoning for Information Fusion in Intelligence, Surveillance, and Reconnaissance 119 Achim Kuwertz, Dirk Mühlenberg, Jennifer Sander, and Wilmuth Müller Multiple Classifier Fusion Based on Testing Sample Pairs 140 Gaochao Feng, Deqiang Han, Yi Yang, and Jiankun Ding vii viii Contents Multi-sensor Fusion Applications in Robotics Bayesian Estimator Based Target Localization in Ship Monitoring System Using Multiple Compact High Frequency Surface Wave Radars 157 Sangwook Park, Chul Jin Cho, Younglo Lee, Andrew Da Costa, SangHo Lee, and Hanseok Ko SLAM-Based Return to Take-Off Point for UAS 168 Daniel Bender, Wolfgang Koch, and Daniel Cremers Underwater Terrain Navigation During Realistic Scenarios 186 Mårten Lager, Elin A Topp, and Jacek Malec Supervised Calibration Method for Improving Contrast and Intensity of LIDAR Laser Beams 210 Mohammad Aldibaja, Noaki Suganuma, Keisuke Yoneda, Ryo Yanase, and Akisue Kuramoto Multi-object Tracking Based on a Multi-layer Particle Filter for Unclustered Spatially Extended Measurements 219 Johannes Buyer, Martin Vollert, Mihai Kocsis, Nico Smann, and Raoul Zưllner Ensemble Kalman Filter Variants for Multi-Object Tracking with False and Missing Measurements 239 Fabian Sigges and Marcus Baum Fall Detection with Unobtrusive Infrared Array Sensors 253 Xiuyi Fan, Huiguo Zhang, Cyril Leung, and Zhiqi Shen Subtle Hand Action Recognition in Factory Based on Inertial Sensors 268 Yanyan Bao, Fuchun Sun, Xinfeng Hua, Bin Wang, and Jianqin Yin Kinematics, Dynamics and Control of an Upper Limb Rehabilitation Exoskeleton 284 Qingcong Wu and Ziyan Shao Author Index 299 Multi-sensor Fusion: Theory and Practice Multi-sensor Fusion: Theory and Practice Sukhan Lee and Hanseok Ko Multisensor fusion and integration in a distributed environment is becoming of utmost importance, especially, in the wake of the growing deployment of Internet of Things (IoT) as well as Cyber Physical Systems (CPS) Although the fundamental theory behind multisensor fusion and integration has been well-established through several decades of investigations, in practice, there still remain a number of technical challenges to overcome, in particular, for dealing with multisensor fusion and integration in a distributed environment Specifically speaking, multisensor fusion with the known cross-correlations among multiple data sources can be handled ideally, for instance, by Bar-Shalom Campo and Generalized Millman’s formula However, in a distributed environment, a number of critical issues arise that are yet to be addressed and solved, including (1) the difficulty of estimating exact cross-correlations among multiple data sources due to the physical relationships possibly existing among their observations as well as the possible double counting by sharing prior information or data sources, (2) the presence of inconsistency or outliers among data sources, (3) the existence of transmission delays as well as data losses and (4) the incorporation of various constraints that may be available among states and observations into fusion The papers collected for this chapter are to address some of the critical issues as described above in a theoretical and/or a practical point of view, as follows: The paper, entitled “Covariance Projection as General Framework of Data Fusion and Outlier Removal,” by Sukhan Lee and Muhammad Abu Bakr proposes a general framework of distributed data fusion for distributed sensor networks of arbitrary redundancies, where inconsistent data are identified simultaneously within the framework The paper, entitled “State Estimation in Networked Control Systems with Delayed and Lossy Acknowledgments,” by Florian Rosenthal, Benjamin Noack and Uwe D Hanebeck deals with the state estimation in networked control systems where the control inputs and measurements transmitted via networks as well as the acknowledgements packets sent by the actuator upon reception of control inputs are subject to data losses and random transmission delays The paper, entitled “Performance of State Estimation and Fusion with Elliptical Motion Constraints,” by Qiang Liu and Nageswara Rao investigates target tracking in the presence of elliptical nonlinear constraints on its motion dynamics, where the state estimates generated by sensors are considered to be sent over long-haul lossy links to a remote fusion center The paper, entitled “Relevance and Redundancy as Selection Techniques for Human-Autonomy Sensor Fusion,” by Justin David Brody, Anna Marie Rogers Dixon, Daniel Donavanik, Ryan M Robinson and William D Nothwang addresses the problem of sensor fusion in a human-autonomy system where the dynamic nature of sensors makes it difficult to model their variability The paper examines the application of information theoretic entities, such as the relevance between sensors and target classes and the redundancy among the selected sensors, as the criteria for evaluating the importance for fusion The paper, entitled “Classification of Reactor Facility 286 Q Wu and Z Shao Fig The architecture of the upper limb rehabilitation exoskeleton.1 Self-aligning platform; AC servo motor; Bowden cable components; Elbow flexion/extension; Wrist flexion/ extension; Wrist ulnal/radial deviation; Forearm pronation/supination; Shoulder flexion/ extension; Shoulder abduction/adduction; 10 Shoulder internal/external Unlike the spherical joint rotating around should joint, there are three independent active joints working together to realize the function of should joint, including internal/ external, abduction/adduction, and flexion/extension movements The three rotation centerlines intersect at the center of shoulder joint The three joints are driven using the wire-steel noose known as the Bowden cable actuation system As the shoulder moves there is also a positional change caused by shoulder girdle movement The passive mobile platform is used to ensure the joints of both human and exoskeleton keeping in the process of coincidence Also the platform can help to eliminate the potential harm caused by the misalignment of the two shoulder joints The platform can realize the linear motion with three degrees of freedom The springs on the platform can help to keep the system in a balance state An active DOF and a passive DOF are used to enable the flexion/extension, and pronation/supination of elbow joint The two freedoms are driven by wire noose too The auxiliary parallel links and zero-length spring are placed on the elbow joint used as gravity balance system And the auxiliary parallel links can also help to keep the gesture of the exoskeleton The flexion/extension, and ulnal/radial function of wrist joint are enabled by two DOFs dividedly Since the driven torque of wrist in the rehabilitation training process is small, two motor drivers are used directly on the joints In order to further diminish weight and energy consumption of the exoskeleton, the Bowden cable actuation system [21, 22] with high torque-to-mass ratio is used as the Kinematics, Dynamics and Control of an Upper Limb 287 actuator for the robot The Schematic of the system is shown in Fig Bowden cable is used to attach the proximal pulley with the distal pulley The proximal pulley is connected to the driven motor and the distal pulley is connecter with the joint on the exoskeleton As the motor drives the proximal pulley to rotate, the driving torque is transmitted to the distal pulley to pull the exoskeleton joint rotating The pretention device provides a movement restriction with positioning hole and a positioning groove And the preloaded bolts are used to adjust the preloading force by changing the preces‐ sion depth in case the Bowden cable becoming flabby Several fixed parts can be added between the cable supports helping to keep certain gesture of the Bowden cable The system can realize the two-way rotation of the joints The AC servo motor is able to provide a maximum torque of 35 Nm which can satisfy the requirement of driving the rotation movement And the motors and reducers are fixed on the support frame to reduce the load of the exoskeleton Fig Kinematic configuration and D-H parameters of the developed rehabilitation exoskeleton with human arm Kinematics of Exoskeleton Figure describes the forward kinematic configuration of the proposed exoskeleton robot The D-H convention strategy is used to describe the robotic coordinate system 288 Q Wu and Z Shao [23], and the corresponding D-H parameters are presented in Table The transformation from the base coordinate frame (internal/external joint of the shoulder) to the endeffector can be given as: ( ) ( ) ( ) ( ) ( ) ( ) ( ) T8 = T2 𝜃 T3 𝜃 T 𝜃 T 𝜃 T 𝜃 T 𝜃 T8 𝜃 (1) where iTj denotes the × homogeneous transformation matrix θi represents the joint rotation variable L1 denotes the link length of upper arm L2 denotes the link length of forearm L3 denotes the link lengths of palm handle The detailed transformation matrix can be presented as:: ⎡ c1 ⎢s T2 = ⎢ ⎢ ⎣0 s1 ⎤ ⎡ c2 s2 ⎤ ⎢ s −c2 ⎥ −c1 ⎥ T3 = ⎢ 0⎥ 0⎥ ⎥ ⎢ ⎥ 1⎦ ⎣ 0 1⎦ / √ / √ / ⎡ c3 − 3s3 s3 − 3L1 s3 ⎤ / ⎥ ⎢ √ / / √ ⎢ s3 3c3 −c3 3L1 c3 ⎥ T4 = ⎢ ⎥ √ / ⎢0 ⎥ 1∕ L1 ∕ ⎢ ⎥ ⎣0 ⎦ 0 0 ⎡ c4 ⎢s T5 = ⎢ ⎢ ⎣0 0 s4 −c4 0 0⎤ 0⎥ 0⎥ ⎥ 1⎦ ⎡ c5 ⎢s T6 = ⎢ ⎢ ⎣0 0 s5 −c5 0 ⎡ c6 ⎢s T7 = ⎢ ⎢ ⎣0 0 s6 −c6 0 0⎤ 0⎥ 0⎥ ⎥ 1⎦ ⎡ c7 ⎢s T8 = ⎢ ⎢ ⎣0 0 s7 L3 c7 ⎤ −c7 L3 s7 ⎥ 0 ⎥ ⎥ ⎦ (2) ⎤ ⎥ L2 ⎥ ⎥ ⎦ Table D-H parameters for the upper limb exoskeleton link i θi/home (deg) αi (deg) (mm) di (mm) ROM_ADL (deg) ROM_EXO (deg) θ1/180 θ2/–60 θ3/–90 θ4/–90 θ5/0 θ6/90 90 90 *** 90 90 90 0 *** 0 0 *** L2 130~245 −195~−35 −135~45 −180~−45 −90~75 80~115 150~240 −180~−45 −120~30 −165~−45 −85~60 80~120 θ7/0 90 L3 −30~55 −30~60 Here, si represents sin (θi), and ci represents cos (θi) Therefore, the forward kine‐ matics is a function of joint variables and limb lengths of exoskeleton Kinematics, Dynamics and Control of an Upper Limb 289 In order to analyze the accessible workspace, the Monte Carlo method [24] was used based on the forward kinematic model, which can be described as: { } W = P||𝜃imin ≤ 𝜃i ≤ 𝜃imax (i = 1, 2, ⋯ , n) (3) where θimin and θimax are the upper limit and lower limit of the ith joint variable By using the pseudo random uniformity number RAND (⋅) ∈ [0, 1], the random joint variables can be shown as: ( ) 𝜃ik = 𝜃imin + 𝜃imax − 𝜃imin RAND(i = 1, 2, ⋯ , 7; k = 1, 2, ⋯ , 𝜆) (4) Here λ is the total number of random samples Setting the link lengths of exoskeleton as L1 = 320 mm, L2 = 250 mm, L3 = 85 mm, and the total number of random samples is set as λ = 6000 And then, the accessible workspace of the exoskeleton can be calcu‐ lated and shown in Fig (a) Three-dimensional accessible workspaces of exoskeleton (c) X-Z plane accessible workspaces (b) X-Y plane accessible workspaces (d) Y-Z plane accessible workspaces Fig Accessible workspaces of the upper limb rehabilitation exoskeleton based on Monte Carlo method 290 Q Wu and Z Shao Singular configuration is an important character of robotic mechanical structure In this exoskeleton robot, two ball-like hinge joints are composed by three DOFs provided by shoulder joint and three DOFs provided by wrist joint, as shown in Fig known as point S and W Singular configuration would come into being in the two hinge joints since the three rotation axis are coplanar and this would lead to the loss of DOFs Point S is used here to analyze the singular level of the exoskeleton using a mixed product of axis’ unit vector, which can be express as ( )| | S = − |𝐳𝟏 ⋅ 𝐳𝟐 × 𝐳𝟑 | | | (5) where S is the singular configuration, z1, z2 and z3 denote the unit vector of the shoulder joint The singular level is and the exoskeleton is in a singular position when the axes are coplanar As the axes change to perpendicular to each other, the singular level would be At this time the operation space becomes the biggest In order to avoid the singular position an optimization is used in the shoulder joint as is shown in Fig The angle between axis z1 and horizontal surface changes to 30 degree to avoid the perpendicularity to the plane composed by z2 and z3 Suppose the two schemes have a same range of movement, singularity distribution characteristics are analyzed with a result shown in Fig The optimized scheme is more reason‐ able since the singular position of the optimized scheme distributes in the top edge of the workspace while the previous’ singular position occupies the center section of the workspace (a) Vertical scheme (b) Optimized scheme Fig Structure scheme of shoulder joint’s rotation axes Kinematics, Dynamics and Control of an Upper Limb (a) Singular level of vertical scheme 291 (b) Singular level of optimized scheme Fig Singular configuration characteristics of the exoskeleton Dynamics of Exoskeleton The dynamic model of the exoskeleton robot is shown in Fig There are several potential restrictions of the model First, the parts of the exoskeleton are assumed to be Fig Dynamic model parameters of the system 292 Q Wu and Z Shao rigid The length of each part remains constant Second, the geometries of the part are assumed to be axially symmetric, so the exoskeleton is simplified as a model with 6link linkage And the mass of each part is supposed to centralize into one end of the link Finally, the inertia of links will not change while moving Table shows several defi‐ nitions for the parameters in the Fig 6: Table Parameters definitions ( ) oi eix , eiy , eiz Parameter Definition Axis build using D-H method Bi mi li qi Rigid body i Weight of Bi Length of link Rotation angle The Kane method can be described as follow: Define ur as generalized velocity of a system with n degrees of freedom, where r has values from to n then the partial velocity and partial angular velocity of each rigid body can be calculated, generalized active forces and generalized inertia forces can be calculated either Letting the summation of generalized active force and generalized inertia force corresponding to certain general‐ ized velocity be zero, n scalar equations would be obtained which known as the equations of motion using the Kane method The general form of Kane’s equation: F(r) + F∗(r) = 𝟎 (r = 1, 2, ⋯ , n) (6) where F(r) is the generalized active force and F∗(r) is the generalized inertia force The exoskeleton robot consists of rigid links Bi (i = 0, 1, ⋯ 6) Notice that there is ◦ a 30 angle between B2 and B3, and The other bars are perpendicular to each other at the initial state Building the fixed reference frame as (O0 , e1) at the top of the exoskeleton, and the body frame can be defined as (Or , er) (r = 1, 2, ⋯ 6) while using the rotation centerline as the z axis for the frame The centroid position could not easily be determined since the rigid body structure is not composed of standard shaped parts The analysis make use of the point Or as the points of quality and inertia concentration qr is used to denote the relative rotation angle between adjacent rigid bodies, so the generalized velocity ur can be obtained by derivation of qr: ur = q̇ r (r = 1, 2, ⋯ 6) (7) ur are combined together to be a matrix: [ ]T u = u𝟏 , ⋯ , ur (r = 𝟏, 𝟐, ⋅ ⋅ ⋅ 𝟔) Define pr as the unit vector for each axis of rotation So qr is the angle who rotates around er And pr form a matrix: Kinematics, Dynamics and Control of an Upper Limb 293 [ ]T p = p1 , ⋯ , pr (r = 𝟏, 𝟐, ⋅ ⋅ ⋅ 𝟔) The formula using the recursive method solving the velocity and angle velocity of each centroid Or are as follows: { ⎧ v𝟏 = 𝟎 ⎪ 𝝎 =u e 1( 𝟏z ) ⎪{ 𝟏 ⎪ v𝟐 = u1 l1 cq1 e𝟏x + l1 sq1 e𝟏y + l2 e𝟏z ⎪ 𝝎𝟐 = 𝝎𝟏 + u2 e𝟐z ) ( ⎪{ ⎪ v𝟑 = v𝟐 + u2 l3 cq2 e𝟐x + l3 sq1 e𝟐y + l4 e𝟐z ⎪ 𝝎𝟑 = 𝝎𝟐 + u3 e𝟑z (√ ) √ ⎪⎧ 3 ⎨⎪v = v +u l cq e + l sq e + l e 𝟑 ⎪⎨ 𝟒 𝟑x 𝟑y 𝟑z ⎪⎪ ⎪{ ⎩ 𝝎𝟒 = 𝝎𝟑 + u4 e𝟒z ⎪ v = v + u (l cq e + l sq e ) 𝟓 𝟒 𝟒x 𝟒y ⎪ ⎪ { 𝝎𝟓 = 𝝎𝟒 + u(5 e𝟓z ) ⎪ v𝟔 = v𝟓 + u5 l7 cq5 e𝟓x + l7 sq5 e𝟓y + l8 e𝟓z ⎪ 𝝎 =𝝎 +u e 𝟔 𝟓 𝟔z ⎩ (8) The relation between axis ei and ei−𝟏 is calculated by the equation: ei = R(q)ei−𝟏 (9) where R(q) is the rotation transformation matrix The partial velocity vi and partial angular velocity 𝜔i relative to the generalized velocity ur can be solved by the following equation: ⎧ v(r) = 𝜕vi ⎪ i ur (i, r = 1, 2, ⋯ 6) ⎨ (r) 𝜕𝜔 i ⎪ 𝜔i = ⎩ ur (10) And the centroid acceleration and angular acceleration of each rigid body are calcu‐ lated using the following formula by taking a derivative with respect to time t: dvi ⎧ ⎪ v̇ l = dt ⎨ d𝜔i (i = 1, 2, ⋯ 6) ⎪ 𝜔̇ l = ⎩ dt (11) The generalized active force is equal to the summation of products of main force vector and main moment acting on the simplified center of the rigid body by the r’s partial velocity and partial angular velocity The formula used to calculate the general‐ ized active force is as follows: 294 Q Wu and Z Shao F(r) = F ⋅ vO(r) + M ⋅ 𝝎(r) (12) The generalized inertia force and can be expressed in the following equation: F∗(r) = F∗ ⋅ vO(r) + M∗ ⋅ 𝝎(r) (13) Where F∗ is the principal vector and M∗ is the principal moment acting on the system Notice that there is a connecting belt on body3, and two gravity balance systems including the free-length springs and auxiliary parallel links are used in the mechanic structure, the principle of gravity balance system is shown in previous work [21], so this analysis just simplifies the force and moment on to joint4 with values of f0 and M0 The quality for body Bi is mi, the principal vector and principal moment caused by gravity could be obtained by the equation: { Fi = mi ge𝟎x (i, r = 1, 2, ⋯ 6) Mi = And the principal vector and principal moment caused by f0 and M0 is: { F𝟎 = f0 e𝟎x M𝟎 = M0 e𝟒z Suppose the main inertia moment about the eiz is Ji, the principal vector and the principal moment for inertia force can be described by follows: { F∗i = mi v̇ l (i, r = 1, 2, ⋯ 6) M∗i = Ji 𝝎̇ l Transform the values of Fi, Mi, F∗i and M∗i into axis e0 According to the Kane equation, the sum of the generalized force and generalized inertial forces corresponding to the generalized velocity is F ⋅ v(r) + M ⋅ 𝝎(r) + F∗ ⋅ vO(r) + M∗ ⋅ 𝝎(r) = O Substituting the parameters into the Kane equation then the kinetic equation of the system can be obtained Admittance Control Strategy In order to induce active participation of patients, an admittance-based patient-active control strategy is developed in this paper The proposed control strategy is depicted in Fig The admittance model can be defined as follow: ) ( ) ( ) ( 𝐌d 𝐏̈ c − 𝐏̈ d +𝐁d 𝐏̇ c − 𝐏̇ d +𝐊d 𝐏c − 𝐏d = 𝐅 − 𝐅d (14) Kinematics, Dynamics and Control of an Upper Limb 295 Fig Overall block diagram of the proposed admittance control strategy Here F and Fd are the actual human-robot interaction force and desired interaction force; 𝐏d , 𝐏̇ d , and 𝐏̈ d are the desired position, velocity, and acceleration of end-effector; 𝐏c , 𝐏̇ c , and 𝐏̈ c are the corresponding control position, velocity, and acceleration of endeffector; 𝐌d , 𝐁d and 𝐊d ∈ R3 are the objective inertial matrix, damping matrix, and stiffness matrix of admittance filter The desired admittance characteristics can be re-expressed in the frequency domain as follow: 𝐏c (s) − 𝐏d (s) Δ𝐏(s) = = 𝐅(s) − 𝐅d (s) 𝐅(s) − 𝐅d (s) 𝐌d s2 +𝐁d s+𝐊d ] [ 𝐅d (s) = 𝐅(s) − Δ𝐅(s) = 𝐅(s) − Δ𝐏(s) 𝐌d s2 +𝐁d s+𝐊d (15) (16) where ΔP denotes the value of position regulation The position controller is developed on the base of a fuzzy sliding mode controller developed in our previous research [25] In the virtual environment module, the feedback signals, which come from the force/ torque sensors, the position sensors, and the switches and pushbuttons mounted at the end-effector, are transmitted into a virtual keyboard unit developed in Visual C program‐ ming environment to handle the game running in virtual environment The virtual keyboard unit is the interface between the wearer and virtual game The effectiveness of rehabilitation training can be analyzed via the game results The desired admittance parameters and interaction force should be rationally selected to adjust the difficulty level of virtual game and the training intensity The effectiveness of the proposed control strategy were demonstrated via a prelimi‐ nary experiment conducted by a male healthy subject The subject was demanded to 296 Q Wu and Z Shao manipulate the end-effector to execute horizontal reciprocating movement along the xaxis of base coordinate system The desired position of end-effector, i.e., Pd, was prede‐ fined as the robot configuration shown in Fig The range of position deviation ∆x was restricted within the interval of [–400 mm, 400 mm] To compare the training perform‐ ance with different admittance parameters, the experiment was carried out with three different groups of parameters More specifically, the first group of admittance param‐ eters was set as: Md = diag [0.06, 0.06, 0.06] Ns2/mm, Bd = diag [0.06, 0.06, 0.06] Ns/ mm, Kd = diag [0.06, 0.06, 0.06] N/mm The second group of admittance parameters was set as: Md = diag [0.025, 0.025, 0.025] Ns2/mm, Bd = diag [0.025, 0.025, 0.025] Ns/ mm, Kd = diag [0.025, 0.025, 0.025] N/mm The third group of admittance parameters was set as: Md = diag [0.015, 0.015, 0.015] Ns2/mm, Bd = diag [0.015, 0.015, 0.015] Ns/ (a) Experimental results of the first group admittance parameters (b) Experimental results of the second group admittance parameters (c) Experimental results of the third group admittance parameters Fig Relationships between interaction force and position variation with different admittance parameters Kinematics, Dynamics and Control of an Upper Limb 297 mm, Kd = diag [0.015, 0.015, 0.015] N/mm The experimental results were presented in Fig It can be seen that during the active rehabilitation training, the value of position deviation is in positive correlation to the interaction force Moreover, the increase of admittance parameters may cause larger motion resistance and training intensity while executing the same training task Conclusion In this research a kind of upper-limb exoskeleton robot system was developed for reha‐ bilitation usage An introduction about the mechanical structure was given And then kinematic and dynamic models were built to carry out the dynamic analysis using D-H method and Kane method An admittance-based controller is proposed to provide patient-active rehabilitation training and induce active participation of subject Prelimi‐ nary experiment was carried out to verify the feasibilities of the developed rehabilitation exoskeleton and control algorithm Acknowledgement This work was supported in part by the National Natural Science Foundation of China (Grant No 51705240), the Natural Science Foundation of Jiangsu Province of China (Grant No BK20170783), and the State Key Laboratory of Robotics and System (HIT, Grant No SKLRS-2018-KF-10) References Le, F., Markovsky, I., Freeman, C.T., Rogers, E.: Identification of electrically stimulated muscle models of stroke patients Control Eng Pract 18(4), 396–407 (2010) Tormene, P., Giorgino, T., Quaglini, S., Stefanelli, M.: Matching incomplete time series with dynamic time warping: an algorithm and an application to post-stroke rehabilitation Artif Intell Med 45(1), 11–34 (2009) Song, A., Pan, L., Xu, G., Li, H.: Adaptive motion control of arm rehabilitation robot based on impedance identification Robotica 33(9), 1795–1812 (2015) Giovacchini, F., et al.: A light-weight active orthosis for hip movement assistance Robot Auton Syst 73, 123–134 (2015) Zoss, A.B., Kazerooni, H., Chu, A.: Biomechanical design of the Berkeley lower extremity exoskeleton (BLEEX) IEEE/ASME Trans Mechatron 11(2), 128–138 (2006) Dollar, A.M., Herr, H.: Lower extremity exoskeletons and active orthoses: challenges and state-of-the-art IEEE Trans Robot 24(1), 144–158 (2008) Mohammed, S., Amirat, Y.: Towards intelligent lower limb wearable robots: Challenges and perspectives - state of the art In: 2008 IEEE International Conference on Robotics and Biomimetics, ROBIO 2008, pp 312–317 (2008) Nef, T., Mihelj, M., Riener, R.: ARMin: a robot for patient-cooperative arm therapy Med Biol Eng Comput 45(9), 887–900 (2007) Wu, Q.C., Wang, X.S., Du, F., Zhang, X.: Design and control of a powered hip exoskeleton for walking assistance Int J Adv Robot Syst 12, 18 (2015) 10 Zierath, J., Woernle, C.: Multibody Dynamics Computational Methods and Applications, vol 28 Springer, Berlin (2013) 298 Q Wu and Z Shao 11 Omar, M.: Multibody dynamics formulation for modeling and simulation of roller chain using spatial operator In: MATEC Web of Conferences, vol 3, pp 1–8 (2016) 12 Schiehlen, W.: Multibody system dynamics: roots and perspectives Multibody Syst Dyn 1(2), 149–188 (1997) 13 Zhou, L., Li, Y., Bai, S.: A human-centered design optimization approach for robotic exoskeletons through biomechanical simulation Robot Auton Syst 91, 337–347 (2017) 14 Hernandez, S., Raison, M., Baron, L.: Refinement of exoskeleton design using multibody modeling: an overview, pp 1–10 (2015) 15 Carignan, C.R., Naylor, M.P., Roderick, S.N.: Controlling shoulder impedance in a rehabilitation arm exoskeleton In: Proceedings of IEEE International Conference on Robotics and Automation, pp 2453-2458 (2008) 16 Jiang, X.Z., Huang, X.H., Xiong, C.H., Sun, R.L., Xiong, Y.L.: Position control of a rehabilitation robotic joint based on neuron proportion-integral and feedforward control J Comput Nonlinear Dyn 7(2), 024502 (2012) 17 Frisoli, A., Sotgiu, E., Procopio, C., Bergamasco, M., Rossi, B., Chisari, C.: Design and implementation of a training strategy in chronic stroke with an arm robotic exoskeleton In: Proceedings of IEEE International Conference on Rehabilitation Robotics, pp 1–8 (2011) 18 Pehlivan, A.U., Losey, D.P., OrMalley, M.K.: Minimal assist-as-needed (mAAN) controller for upper limb robotic rehabilitation IEEE Trans Robot 32(1), 113–124 (2016) 19 Luna, C.O., Rahman, M.H., Saad, M., Archambault, P.S., Ferrer, S.B.: Admittance-based upper limb robotic active and active-assistive movements Int J Adv Robot Syst 12, 117 (2015) 20 Duygun, E., Mallapragada, V., Sarkar, N., Taub, E.: A new control approach to robot assisted rehabilitation In: Proceedings of IEEE International Conference on Rehabilitation Robotics, pp 323–328 (2005) 21 Wu, Q.C., Wang, X.S.: Design of a gravity balanced upper limb exoskeleton with bowden cable actuators In: Proceedings of IFAC Symposium on Mechatronic Systems, pp 679–683 (2013) 22 Wu, Q.C., Wang, X.S., Chen, L., Du, F.P.: Transmission model and compensation control of double-tendon-sheath actuation system IEEE Trans Ind Electron 62(3), 1599–1609 (2015) 23 Spong, M.W., Hutchinson, S., Vidyasagar, M.: Robot Modeling and Control Wiley, New York (2006) 24 Qingxuan, L., Gang, C.: Calculation of space robot workspace by using Monte Carlo method Spacecr Eng 4(14), 79–85 (2011) 25 Wu, Q.C., Wang, X.S., Du, F.P.: Modeling and position control of a therapeutic exoskeleton targeting upper extremity rehabilitation P I Mech Eng C- J Mech 231, 4360–4373 (2016) Author Index A Aldibaja, Mohammad, 210 Kuramoto, Akisue, 210 Kuwertz, Achim, 119 B Bakr, Muhammad Abu, Bao, Yanyan, 268 Baum, Marcus, 98, 239 Bender, Daniel, 168 Brody, Justin D., 52 Buyer, Johannes, 219 L Lager, Mårten, 186 Lee, SangHo, 157 Lee, Sukhan, Lee, Younglo, 157 Leung, Cyril, 253 Liu, Qiang, 39 C Cho, Chul Jin, 157 Cremers, Daniel, 168 M Malec, Jacek, 186 Mühlenberg, Dirk, 119 Müller, Wilmuth, 119 D Da Costa, Andrew, 157 Ding, Jiankun, 140 Dixon, Anna M R., 52 Donavanik, Daniel, 52 F Fan, Xiuyi, 253 Feng, Gaochao, 140 N Nguyen, Tran Tuan, 98 Noack, Benjamin, 22 Nothwang, William D., 52 P Park, Sangwook, 157 H Han, Deqiang, 140 Hanebeck, Uwe D., 22 Hua, Xinfeng, 268 R Ramirez, Camila, 76 Rao, Nageswara S V., 39, 76 Robinson, Ryan M., 52 Rosenthal, Florian, 22 K Ko, Hanseok, 157 Koch, Wolfgang, 168 Kocsis, Mihai, 219 Kruse, Rudolf, 98 S Sander, Jennifer, 119 Shao, Ziyan, 284 Shen, Zhiqi, 253 Sigges, Fabian, 239 © Springer International Publishing AG, part of Springer Nature 2018 S Lee et al (Eds.): MFI 2017, LNEE 501, pp 299–300, 2018 https://doi.org/10.1007/978-3-319-90509-9 299 300 Sitzmann, Jonas, 98 Spehr, Jens, 98 Suganuma, Noaki, 210 Sun, Fuchun, 268 Sußmann, Nico, 219 Author Index W Wang, Bin, 268 Wu, Qingcong, 284 T Topp, Elin A., 186 Y Yanase, Ryo, 210 Yang, Yi, 140 Yin, Jianqin, 268 Yoneda, Keisuke, 210 V Vollert, Martin, 219 Z Zhang, Huiguo, 253 Zöllner, Raoul, 219 Zug, Sebastian, 98 ... Editors Multisensor Fusion and Integration in the Wake of Big Data, Deep Learning and Cyber Physical System An Edition of the Selected Papers from the 2017 IEEE International Conference on Multisensor. .. theories and advancing implementation tools to address the emerging key issues in multisensor fusion and integration in the wake of big data and deep learning would make the above transition smooth and. .. especially, in the wake of the growing deployment of Internet of Things (IoT) as well as Cyber Physical Systems (CPS) Although the fundamental theory behind multisensor fusion and integration