AdvancesinHaptics192 Fig. 14. Measured test point in physical workspace. Comparing to the model rendered in the virtual environment, as the operator positioned the device to the test point, the calculated coordinate based on measured motor angles, forward kinematics, and a proper scaling of the model is (0.346, 0.050, 0.000). Fig. 15. Test points in physical workspace. By adjusting the position of a camera along the positive x-axis, Figure 15 shows the top view of the device and the previous test point 1 projected onto this view plane at a distance approximately 0.35m parallel to the y-z plane. Table 4 shows the comparison between the measurements on the actual device and the calculated coordinates on the virtual model as the operator manipulated and positioned the tip of the handle to all the test points shown. Note that the experiments were conducted by the operator determining the position of the tip of the handle and estimating a home reference with all motor angles resetting to zero at the starting origin. The imperfect zero-home reference, estimated location of the handle tip, and camera displacements may introduce source of errors during the experiments. Test Point Measured y-z coordinates on physical device (metres) Calculated y-z coordinates on virtual model (metres) 1 (0.050, 0.000) 0.050, 0.000 2 (0.050, -0.050) 0.050, -0.049 3 (-0.050, -0.050) -0.049, -0.049 4 (-0.050, 0.050) -0.049. 0.051 5 (0.050, 0.050) 0.050, 0.051 Table 4. Comparison between measured coordinates and calculated coordinates 4.3 Haptic Feedback In addition to the observation and tracking of a virtual tool, an example haptic scene is prepared for the operator to experience haptic feedback in the environment. A virtual wall is predefined in the scene prior to the experiment. Figure 16 shows the virtual environment (left) and the home position of the device model (right). The sphere rendered in the left region indicates the haptic interface point in the scene. Fig. 16. Virtual environment for haptic exploration (virtual wall into page) AnalysisandExperimentalStudyofa4-DOFHapticDevice 193 Fig. 14. Measured test point in physical workspace. Comparing to the model rendered in the virtual environment, as the operator positioned the device to the test point, the calculated coordinate based on measured motor angles, forward kinematics, and a proper scaling of the model is (0.346, 0.050, 0.000). Fig. 15. Test points in physical workspace. By adjusting the position of a camera along the positive x-axis, Figure 15 shows the top view of the device and the previous test point 1 projected onto this view plane at a distance approximately 0.35m parallel to the y-z plane. Table 4 shows the comparison between the measurements on the actual device and the calculated coordinates on the virtual model as the operator manipulated and positioned the tip of the handle to all the test points shown. Note that the experiments were conducted by the operator determining the position of the tip of the handle and estimating a home reference with all motor angles resetting to zero at the starting origin. The imperfect zero-home reference, estimated location of the handle tip, and camera displacements may introduce source of errors during the experiments. Test Point Measured y-z coordinates on physical device (metres) Calculated y-z coordinates on virtual model (metres) 1 (0.050, 0.000) 0.050, 0.000 2 (0.050, -0.050) 0.050, -0.049 3 (-0.050, -0.050) -0.049, -0.049 4 (-0.050, 0.050) -0.049. 0.051 5 (0.050, 0.050) 0.050, 0.051 Table 4. Comparison between measured coordinates and calculated coordinates 4.3 Haptic Feedback In addition to the observation and tracking of a virtual tool, an example haptic scene is prepared for the operator to experience haptic feedback in the environment. A virtual wall is predefined in the scene prior to the experiment. Figure 16 shows the virtual environment (left) and the home position of the device model (right). The sphere rendered in the left region indicates the haptic interface point in the scene. Fig. 16. Virtual environment for haptic exploration (virtual wall into page) AdvancesinHaptics194 The position of the wall is located at 0.020m into the page (positive z-axis) relative to the origin. The wall (rectangle) is parallel to the x-y plane. The home position of the tool (or straight up) is along the positive x-axis. Figure 16 shows the scene with the camera behind (on negative z-axis) and looking at the origin. The operator performed the experiment by moving the sphere towards the wall along the positive z-axis and colliding the sphere with the virtual wall. Figure 17 shows the position of the sphere as the operator manipulated the tool and moved the sphere accordingly. Figure 18 shows the calculated Cartesian force Fz (Fx = Fy = 0) at the haptic interface point when the sphere collided with the virtual wall. 0 5 10 15 20 25 30 -0.08 -0.06 -0.04 -0.02 0 0.02 0.04 Time (second) Position (metre) x y z Fig. 17. Position of the sphere with respect to the reference coordinate system. 0 5 10 15 20 25 30 0 500 1000 1500 2000 2500 3000 Time (second) Force (mN) Fig. 18. Cartesian force at the haptic interface point along the z-axis. 0 5 10 15 20 25 30 0 5 10 15 20 25 30 35 Time (second) Torque (mNm) Torque 3 Torque 1 Torque 2 Fig. 19. Measured torque versus time plot. Figure 19 shows the measured torque versus time plot. The torque was measured by the application of the current monitor output (signals available from the servo amplifiers) and the integration of a low-pass filter circuit and the ADC IC on the DAS. Three axes of the motors were monitored for the torque output during the experiment. The results show the torque experienced by the operator holding the tool and repeatedly colliding with a virtual wall. As seen in the above plots, the period during which the sphere position is at the threshold, i.e. when the spring model is in effect, the force started to increase as the operator attempted to push the sphere further onto the wall. Figure 19 shows the decomposition of the torque into three motor torques felted by the operator. In this example, actuator 1 and actuator 3 were operating in order to generate the reaction force. Using the same virtual scene, the following plots show the experimental results as the operator moved the sphere back and forth from the origin to the wall experiencing a greater reaction force while attempting to penetrate the sphere further into the wall. AnalysisandExperimentalStudyofa4-DOFHapticDevice 195 The position of the wall is located at 0.020m into the page (positive z-axis) relative to the origin. The wall (rectangle) is parallel to the x-y plane. The home position of the tool (or straight up) is along the positive x-axis. Figure 16 shows the scene with the camera behind (on negative z-axis) and looking at the origin. The operator performed the experiment by moving the sphere towards the wall along the positive z-axis and colliding the sphere with the virtual wall. Figure 17 shows the position of the sphere as the operator manipulated the tool and moved the sphere accordingly. Figure 18 shows the calculated Cartesian force Fz (Fx = Fy = 0) at the haptic interface point when the sphere collided with the virtual wall. 0 5 10 15 20 25 30 -0.08 -0.06 -0.04 -0.02 0 0.02 0.04 Time (second) Position (metre) x y z Fig. 17. Position of the sphere with respect to the reference coordinate system. 0 5 10 15 20 25 30 0 500 1000 1500 2000 2500 3000 Time (second) Force (mN) Fig. 18. Cartesian force at the haptic interface point along the z-axis. 0 5 10 15 20 25 30 0 5 10 15 20 25 30 35 Time (second) Torque (mNm) Torque 3 Torque 1 Torque 2 Fig. 19. Measured torque versus time plot. Figure 19 shows the measured torque versus time plot. The torque was measured by the application of the current monitor output (signals available from the servo amplifiers) and the integration of a low-pass filter circuit and the ADC IC on the DAS. Three axes of the motors were monitored for the torque output during the experiment. The results show the torque experienced by the operator holding the tool and repeatedly colliding with a virtual wall. As seen in the above plots, the period during which the sphere position is at the threshold, i.e. when the spring model is in effect, the force started to increase as the operator attempted to push the sphere further onto the wall. Figure 19 shows the decomposition of the torque into three motor torques felted by the operator. In this example, actuator 1 and actuator 3 were operating in order to generate the reaction force. Using the same virtual scene, the following plots show the experimental results as the operator moved the sphere back and forth from the origin to the wall experiencing a greater reaction force while attempting to penetrate the sphere further into the wall. AdvancesinHaptics196 0 5 10 15 20 25 30 -0.08 -0.06 -0.04 -0.02 0 0.02 0.04 Time (second) Position (metre) z y x Fig. 20. Position of the sphere with respect to the reference coordinate system. 0 5 10 15 20 25 30 0 500 1000 1500 2000 2500 3000 Time (second) Force (mN) Fig. 21. Cartesian force at the haptic interface point along the z-axis. 0 5 10 15 20 25 30 0 5 10 15 20 25 30 35 Time (second) Torque (mNm) Torque 3 Torque 1 Fig. 22. Measured torque versus time plot. 4. Summary This chapter presented the design, modelling and hardware integration of a haptic device. The design of the haptic device is based on the notion of the hybrid spherical mechanism which consists of both a passive and active spherical joints. The passive joint is responsible for supporting the static load and user interaction forces whereas the active joint is responsible for creating the haptic feedback to the user. Closed-form solution for kinematic analysis and force mapping of the device is presented. A novel distributed computational platform is also proposed. The platform exploits the notion of scalability and modularity in the design. Performance of the closed loop system is presented in the context of interacting with a rigid environment and achieving a high sampling rate using either the UDP or the TCP communication protocols. 5. References Angeles, J. & Gosselin, C. (1990). Singularity analysis of closed-loop kinematic chains, IEEE Transactions on Robotics and Automation, vol. 6, no. 3, pp. 281-290, 1990 Birglen, L.; Gosselin, C.; Pouliot, N.; Monsarrat, B. & Laliberté, T. (2002). SHaDe, a New 3- DOF Haptic Device, IEEE Transactions on Robotics and Automation, vol. 18, no. 2, pp. 166-175, 2002 Boudreau, R., Darenfed, S. & Gosselin, C. (1998). On the Computation of the Direct Kinematics of Parallel Manipulators Using Polynomial Networks, IEEE Transactions on Systems, Man, and Cybernetics – Part A: Systems and Humans, vol. 28, no. 2, pp. 213-220, 1998 Buttolo, P., Oboe, R. & Hannaford, B. (1997). Architectures for Shared Haptic Virtual Environments, Computers & Graphics, vol. 21, no. 4, pp. 421-429, 1997 Craver, W. (1989). Master Thesis: Structural Analysis and Design of a Three-Degree-Of-Freedom Robotic Shoulder Module, The University of Texas at Austin, 1989 Gosselin, C. & Hamel, J. (1994). The Agile-Eye: a High-Performance Three-Degree-Of- Freedom Camera Orienting Device, Proc. IEEE Int. Conf. Robotics and Automations, vol. 1, pp. 781-788, 1994 Li, T. & Payandeh, S. (2002). Design of Spherical Parallel Mechanisms for Application to Laparoscopic Surgery, Robotica, pp. 133-138, 2002 Ma, A. & Payandeh, S. (2008). Analysis and Experimentation of a 4-DOF Haptic Device, 16th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 351-356, March 2008 Mishra, R. & Srikanth, S. (2000). GENIE – An Haptic Interface for Simulation of Laparoscopic Surgery, Intelligent Robots and Systems, vol. 1, pp. 714-719, 2000 AnalysisandExperimentalStudyofa4-DOFHapticDevice 197 0 5 10 15 20 25 30 -0.08 -0.06 -0.04 -0.02 0 0.02 0.04 Time (second) Position (metre) z y x Fig. 20. Position of the sphere with respect to the reference coordinate system. 0 5 10 15 20 25 30 0 500 1000 1500 2000 2500 3000 Time (second) Force (mN) Fig. 21. Cartesian force at the haptic interface point along the z-axis. 0 5 10 15 20 25 30 0 5 10 15 20 25 30 35 Time (second) Torque (mNm) Torque 3 Torque 1 Fig. 22. Measured torque versus time plot. 4. Summary This chapter presented the design, modelling and hardware integration of a haptic device. The design of the haptic device is based on the notion of the hybrid spherical mechanism which consists of both a passive and active spherical joints. The passive joint is responsible for supporting the static load and user interaction forces whereas the active joint is responsible for creating the haptic feedback to the user. Closed-form solution for kinematic analysis and force mapping of the device is presented. A novel distributed computational platform is also proposed. The platform exploits the notion of scalability and modularity in the design. Performance of the closed loop system is presented in the context of interacting with a rigid environment and achieving a high sampling rate using either the UDP or the TCP communication protocols. 5. References Angeles, J. & Gosselin, C. (1990). Singularity analysis of closed-loop kinematic chains, IEEE Transactions on Robotics and Automation, vol. 6, no. 3, pp. 281-290, 1990 Birglen, L.; Gosselin, C.; Pouliot, N.; Monsarrat, B. & Laliberté, T. (2002). SHaDe, a New 3- DOF Haptic Device, IEEE Transactions on Robotics and Automation, vol. 18, no. 2, pp. 166-175, 2002 Boudreau, R., Darenfed, S. & Gosselin, C. (1998). On the Computation of the Direct Kinematics of Parallel Manipulators Using Polynomial Networks, IEEE Transactions on Systems, Man, and Cybernetics – Part A: Systems and Humans, vol. 28, no. 2, pp. 213-220, 1998 Buttolo, P., Oboe, R. & Hannaford, B. (1997). Architectures for Shared Haptic Virtual Environments, Computers & Graphics, vol. 21, no. 4, pp. 421-429, 1997 Craver, W. (1989). Master Thesis: Structural Analysis and Design of a Three-Degree-Of-Freedom Robotic Shoulder Module, The University of Texas at Austin, 1989 Gosselin, C. & Hamel, J. (1994). The Agile-Eye: a High-Performance Three-Degree-Of- Freedom Camera Orienting Device, Proc. IEEE Int. Conf. Robotics and Automations, vol. 1, pp. 781-788, 1994 Li, T. & Payandeh, S. (2002). Design of Spherical Parallel Mechanisms for Application to Laparoscopic Surgery, Robotica, pp. 133-138, 2002 Ma, A. & Payandeh, S. (2008). Analysis and Experimentation of a 4-DOF Haptic Device, 16th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 351-356, March 2008 Mishra, R. & Srikanth, S. (2000). GENIE – An Haptic Interface for Simulation of Laparoscopic Surgery, Intelligent Robots and Systems, vol. 1, pp. 714-719, 2000 AdvancesinHaptics198 AHapticallyEnhancedOperationalConceptforaHydraulicExcavator 199 AHapticallyEnhancedOperationalConceptforaHydraulicExcavator HenningHaynandDieterSchwarzmann 0 A Haptically Enhanced Operational Concept for a Hydraulic Excavator Henning Hayn and Dieter Schwarzmann Robert Bosch GmbH Germany 1. Introduction In mobile hydraulic machines, like excavators, backhoe loaders, wheel loaders, and forklift trucks, haptic human-machine interfaces are not in use. Today, the machines are operated with mechanical-hydraulic joysticks. Each joystick axis controls a single hydraulic actuator. This leads to not very easy to use operational concepts. Since electrohydraulic systems with electronic joysticks are available for serial applications, alternative operational concepts be- come feasible. A known alternative is to control the machine using an operating device whose segments resemble the manipulator geometry, as shown in Fig. 1 (Uchino et al., 1977). This operational concept is often called coordinated control. Typically, a master-slave system is employed where the operating device (master) outputs the reference position to the position control loop of the machine (slave). This concept promises an intuitive operation of the machine. (a) Excavator (b) Operating device Fig. 1. Operating device resembles the manipulator geometry This concept can be enhanced by haptic assistance systems in order to improve the operator’s performance. These haptically enhanced coordinated control operational concepts aim at • increasing the machine efficiency (handling capacity) by providing driver assistance systems, • reducing the time needed by the driver to learn the operation of the machine, and • reducing operating errors especially for unexperienced drivers. 10 AdvancesinHaptics200 In this work, a SensAble Phantom Omni haptic device is used to generate the position refer- ence signal for the tool center point (TCP) of the hydraulic manipulator of an 18 ton excavator. The two arm segments of the operating device resemble the boom and stick of the excavator, as shown in Fig. 2. Fig. 2. Analogy of the geometry of a Phantom Omni device and an excavator manipulator This chapter is organized as follows: Section 2 gives an overview of haptic feedback in mobile hydraulic machines. In Section 3 an alternative operational concept for excavators is pro- posed. Section 4 discusses the interconnection of the position controllers of the haptic device and the hydraulic excavator leading to a control methodology for bilateral master-slave sys- tems. Section 5 introduces the applied controller design method, namely internal model con- trol (IMC), for integrating plants with input constraints. After describing the controller design for the electric actuators of the haptic device and for the hydraulic actuators of the excava- tor, experimental results are given in Section 6. Section 7 shows the results of initial usability experiments with test drivers and Section 8 offers some conclusions. 2. Haptic Feedback in Mobile Hydraulic Machines The automation level of available mobile machines is low, with the exception of some special applications like forest machines or robotic cargo loading systems. Due to the increasing use of automation technology, research and development activities focus on new human-machine interfaces and operational concepts. They aim at improving efficiency, safety and comfort. A prerequisite for innovative human-machine interfaces is the availability of electrohydraulics and the corresponding controllers. Then, new functions like driver assistance and safety sys- tems can be integrated in the controller software as well as alternative operating devices. In the future, the level of automation in mobile machines will increase up to complete automa- tion (Haas & Kim, 2002). The integration of the sense of touch in the human-machine interface promises an easier, faster and more intuitive operation than the typically encountered visual information. Haptic inter- faces have advantages, compared to human-machine interfaces that do not integrate the sense of touch, especially if the operator has to work delicately and accurately, or if various materials – for example with different elasticity – are handled. Haptically enhanced assistance systems can support the driver of a mobile machine in per- forming his working task by providing tactile sensations via the operating device. Haptic driver assistance systems for hydraulic excavators could for example • warn the driver of damaging obstacles, • feed back digging or gripping forces, • imitate open-center hydraulic systems, • enable the driver to sense the inertia of the machine’s manipulator, • simplify leveling and slope cutting, • limit the excavator’s workspace, • guide the bucket on a specific trajectory, or • assist the collaborative manipulation of a heavy building element by multiple operators. A significant advantage of haptic systems compared to semi- or fully automated assistance systems is, that the operator always has complete control over his machine. The driver is able to manually overrule the assistance systems, assuming that the human operator is always stronger than the actuators of the haptic device. The application of haptic technologies in human-machine interfaces of mobile machines is not prior art in series-production vehicles. It can be found in some scientific contributions and sporadic industrial projects, only. One finds that either the machines are controlled with conventional force-feedback joysticks (Cemenska et al., 1989; Parker et al., 1993; Yamada & Muto, 2003; Augustine, 2005) or special haptic operating devices whose segments resemble the manipulator geometry (Ostoja-Starzewski & Skibniewski, 1989; Yoshinada & Takeda, 1990; Kraft, 1991; Kontz, 2007). The same principle is known from industrial robots and similar manipulators. An overview of haptic interface technology for mobile machines, like hydraulic excavators, can be found in Hayn & Schwarzmann (2008). 3. Development of an Intuitive Operational Concept for Hydraulic Excavators Alternatives to conventional operational concepts are known but did not become widely ac- cepted. The most important reasons are: • Electrohydraulics was not available for series-production at reasonable prices, • mechanical-hydraulic components are known for being robust and reliable, and • the mobile machinery industry has a rather conservative attitude towards alternative operational concepts. In order to design an intuitive operational concept for the hydraulic manipulator of excava- tors the coordinated control approach was developed further. It was assumed that operating elements that resemble the manipulator geometry are intuitive and easy to use. These operat- ing elements have the same degree of freedom, the same types of joints and the same moving direction as the machine that has to be controlled. This property is called compatibility of the moving directions (Sachs et al., 1994). This means in detail for the manipulator of an excava- tor: • The rotation of the cabin has to be controlled using a rotary operating element, • the translation of the tool center point has to be controlled using an element which is free-moving within a vertical plane, and • tilting the bucket has to be controlled using a rotary element. The realization of this idea was expected to result in an unambiguous, predictable, and con- sistent, thus intuitive operational concept. When developing new operational concepts for hydraulic excavators it is additionally important to consider the concept being ergonomic and suitable for earthmoving machinery. [...]... hydraulic cylinders and the slew drive of the excavator) show an integrating behavior, the same control method, namely internal model control for integrating linear SISO systems, is chosen for all plants A review of the design is given in the following, starting with non-integrating plants, i.e., plants Σ having i poles pi with Re{ pi } < 0 for all i 5.1 IMC for Non-Integrating Stable Systems ˜ The main idea... compute the constraints for the highest derivative ¨ yhd according to Section 5.4: ¨ yhd,min = 1 ˙ (−yhd + T1,j · uhd,min ) kj , (30) ¨ yhd,max = 1 ˙ (−yhd + T1,j · uhd,max ) kj (31) 214 Advances in Haptics Fig 14 State-variable filter for the haptic device controller including input constraints 6. 3 Controller Design for the Hydraulic Boom The actuating variables for the hydraulic cylinders of the excavator... multifinger haptic interface joined to an arm can provide a wide operation space But, most of them are mounted on the back of the human hand like the CyberForce (CyberGlove Systems, 2009) Fixing the haptic interface to the hand gives oppressive feeling to the operator since the interface binds the human hand firmly In order to reduce the oppressive feeling and increase safety, a three-finger haptic interface... (22b) u∈[umin ,umax ] u∈[umin ,umax ] 212 Advances in Haptics This means that the actuating variable u is constrained exactly to [umin , umax ] if the highest (r ) ˜ derivative yd is restricted to these limits 6 Controller Design for the Operating Device and for the Excavator 6. 1 Design Procedure The controllers in Fig 8 were designed utilizing the IMC method The following procedure was used to design... rotating platform (slew drive) The rotating platforms of both systems are single-input, singleoutput (SISO) plants In order to control the position of the TCP, a reference signal generator is used The reference signal generator converts the desired position into reference variables for the electric joint actuators ϕref and the hydraulic cylinders lref,z1 , lref,z3 using the inverse kinematics In spite... experience with hydraulic excavators performed a predefined working task They were filmed to analyze the working cycle times and operating errors Finally the test persons were surveyed using a questionnaire 2 16 Advances in Haptics w(m), y(m) 6. 5 (a) 6 5.5 yex,x wex,x 5 4.5 0 w(m), y(m) 3 1 2 3 Time (s) 4 5 6 yex,y wex,y (b) 2 1 0 −1 0 1 2 3 Time (s) 4 5 6 Fig 17 Position on x- and y-axis of the excavator and... Mechanical Engineering, Georgia Institute of Technology Kraft, B W (1991) Force feedback control for backhoe, Patent US 5019 761 Lawrence, P., Sauder, B., Wallersteiner, U & Wilson, J (1990) Teleoperation of forest harvesting machines, in J Coutteau (ed.), Proceedings of the Symposium Robotics in Forestry Forest Operations in the Age of Technology, number SR-75, Vaudreuil, Quebec, pp 36 39 Lunze, J (20 06) Regelungstechnik... consider individual effect of each point on the other points and how the object should be deformed based on all fingers interaction Using a local model is not a solution because the entire model, or sufficiently large independent part of it, must be dealt with to take into account the effects of each finger on the model and on the other fingers To do so, we take an “elementary displacement” approach in finite... swiveling the light gray, spring centered element The slew drive for the rotating platform with the cabin is operated with the left hand using the turning knob shown in Fig 4(b) The turning knob is divided into an inner dark gray disc and an outer light gray wheel The inner disc can be used to set a desired swing angle The outer wheel is used to control the yaw rate The outer wheel allows positioning... index finger and the haptic finger is 281 and 535 [cm3], respectively The product space at the optimum pose of the haptic finger is 259 [cm3] The allocation of fingers in haptic hand was designed taken in consideration the above geometrical relation In the second link of each finger, a 6- axes force sensor (NANO sensor by BL AUTOTEC LTD.) is installed The force sensor is used to control the human fingertip . sphere rendered in the left region indicates the haptic interface point in the scene. Fig. 16. Virtual environment for haptic exploration (virtual wall into page) Advances in Haptics1 94 . design is given in the following, starting with non-integrating plants, i.e., plants Σ having i poles p i with Re{p i } < 0 for all i. 5.1 IMC for Non-Integrating Stable Systems The main idea of. method, namely internal model control for integrating linear SISO systems, is chosen for all plants. A review of the design is given in the following, starting with non-integrating plants, i.e.,