Title: Scaling Population-Based Reinforcement Learning with GPU Accelerated Simulation
Author of the experiment: Asad Ali Shahid, Yashraj Narang, Vincenzo Petrone, Enrico Ferrentino, Ankur Handa, Dieter Fox, Marco Pavone, Loris Roveda
Description:
The proposed Population-Based Reinforcement Learning (PBRL) framework applies a genetic algorithm to RL to optimize policy hyperparameters. It consists in using a population of independent agents learning the same RL task. The agents are periodically evaluated and ranked according to the cumulative reward, and the hyperparameters of the best performing agents are mutated and transferred to the worst performing agents. Compared to standard RL, PBRL reaches higher rewards with a shorter convergence time, tested on 4 different tasks on the IsaacGym simulator (Anymal Terrain, Humanoid, Shadow Hand, and Franka Nut Pick) against 3 state-of-the-art policy adaptation algorithms (PPO, SAC, and DDPG). A PBRL policy trained in simulation for the Franka Nut Pick task is deployed on the real system, without any real-word adaptation or fine-tuning, showcasing successful sim-to-real transfer.
More details in the paper A. A. Shahid, Y. Narang, V. Petrone, E. Ferrentino, A. Handa, D. Fox, M. Pavone and L. Roveda (2024). "Scaling Population-Based Reinforcement Learning with GPU Accelerated Simulation". arXiv print, Submitted for Publication to IEEE Robotics and Automation Letters (RA-L). DOI: 10.48550/arXiv.2404.03336, link.
Title: Assistive force control in collaborative human-robot transportation
Author of the experiment: Bruno G. C. Lima, Enrico Ferrentino, Pasquale Chiacchio, Mario Vento
Description:
The proposed Assistive Force Control (AFC) is an interaction control approach based on assistive forces, well suited for human-robot collaborative tasks involving physical interactions such as collaborative transportation. It offers an alternative to the commonly used indirect approaches such as admittance/impedance control. The proposed architecture is based on two parts. First, the human intention, modeled as a passivity index, is estimated through recent interaction forces and robot velocities. Second, an assistive force is computed in the direction of the human-applied forces, based on the passivity index and a chosen activation function. By sending the assistive force as an input to a lower-level direct force controller, the novel approach acts directly on the causes of motion, instead of acting on the motion itself. We validate the proposed architecture on two real-case collaborative transportation scenarios involving an industrial robot. Our preliminary results show that low effort is required for human operators to manipulate heavy objects.
More details in the paper B. G. C. Lima, E. Ferrentino, P. Chiacchio and M. Vento (2023), "Assistive force control in collaborative human-robot transportation", 2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), Busan, Korea, Republic of, 2023, pp. 2646-2651, doi: 10.1109/RO-MAN57019.2023.10309518, link.
Title: Experimental Validation of an Actor-Critic Model Predictive Force Controller for Robot-Environment Interaction Tasks
Author of the experiment: Luca Puricelli, Alessandro Pozzi, Vincenzo Petrone, Enrico Ferrentino, Pasquale Chiacchio, Francesco Braghin, Loris Roveda
Description:
An Actor-Critic Model Predictive Force Controller (ACMPFC) is a control strategy based on reinforcement learning. An ensemble of feed-forward neural networks learns the robot-environment interaction dynamics, embedding a model from which two additional networks compute the optimal action to track a desired force reference, implementing a Model Predictive Control (MPC). The Actor outputs the control action, namely the setpoint to command to a low-level impedance control, that guarantees compliance with respect to the environment. The Critic computes the value of the Actor's output, according to the Bellman's equation, based on a local reward function to minimize the force-tracking error. The ACMPFC, tested in both simulated and real scenarios on a Panda robot, reaches convergence after 10 episodes, showing good force-tracking performance and generalization capabilities.
More details in the paper Pozzi, A.; Puricelli, L.; Petrone, V.; Ferrentino, E.; Chiacchio, P.; Braghin, F. and Roveda, L. (2023). "Experimental Validation of an Actor-Critic Model Predictive Force Controller for Robot-Environment Interaction Tasks". In Proceedings of the 20th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO; ISBN 978-989-758-670-5; ISSN 2184-2809, SciTePress, pages 394-404. DOI: 10.5220/0012160700003543, link.
Title: Assessment of haptic teleoperations in robotized planetary sample collection and assembly
Author of the experiment: Lorenzo Pagliara, Vincenzo Petrone, Enrico Ferrentino, Pasquale Chiacchio
Description:
The experiment is a demonstration of a novel operational concept, where suitable Human-Robot Interfaces (HRI) are designed to aid mission operations and to simplify training of new operators. A mobile manipulator performs retrieval and assembly of a sample collected from the soil in order to simulate an In-Situ Resource Utilization (ISRU) scenario. Large arm movements are automatically planned and assessed by ground operators before uplink to the robot. Precision movements and operations are teleoperated through a haptic interface. Interaction forces between the robot and the sample and between the sample and the assembly socket are sensed by on-board algorithms, downlinked and rendered on the haptic interface for the operator to be able to feel them. Pose commands for the robot, generated by the haptic device, follow the opposite path.
More details in the paper L. Pagliara, V. Petrone, E. Ferrentino and P. Chiacchio, "Human- Robot Interface for Teleoperated Robotized Planetary Sample Collection and Assembly", 2023 IEEE 10th International Workshop on Metrology for AeroSpace (MetroAeroSpace), Milan, Italy, 2023, pp. 171-176, link.
Title: Optimized Residual Action for Interaction Control with Learned Environments (ORACLE)
Author of the experiment: Luca Puricelli, Alessandro Pozzi, Vincenzo Petrone, Enrico Ferrentino, Pasquale Chiacchio, Francesco Braghin, Loris Roveda
Description:
The ORACLE is able to predict the force the robot will exert on the environment it interacts with. The prediction is computed by an ensemble of feed-forward neural networks, properly pre-trained to learn the robot-environment interaction dynamics. This model is exploited in an optimization problem, whose solution minimizes the force tracking error with respect to a reference trajectory, regularized to avoid overshoots, slow transients and rapid changes in control actions. ORACLE's output is a residual action, added to a general purpose interaction controller's action, so as to guarantee force tracking and safety.
More details in the paper V. Petrone, L. Puricelli, A. Pozzi, E. Ferrentino, P. Chiacchio, F. Braghin and L. Roveda, "Optimized Residual Action for Interaction Control with Learned Environments", TechRxiv preprint, Mar. 2024, Submitted to IEEE Transactions on Control System Technology, DOI: 10.36227/techrxiv.21905433.v2, link.
Title: Robot-agnostic interaction controllers based on ROS
Author of the experiment: Federica Storiale, Enrico Ferrentino, Pasquale Chiacchio
Description:
Robot-agnostic interaction controllers are tested on the UR10 and Comau Smart-Six robots, as well as on their simulated versions in Gazebo in four different scenarios: trajectory tracking with Cartesian compliance (admittance control), hybrid position-force control (direct force control), human-robot collaboration (admittance control) and teaching by showing (direct force control). The controllers are based on ros_control: they assume a position-controlled robot at joint level and can work with any flange-mounted force/torque sensor, whose driver is abstracted through a generic interface.
More details in the paper Storiale, F.; Ferrentino, E.; Chiacchio, P. Robot-Agnostic Interaction Controllers Based on ROS. Appl. Sci. 2022, 12, 3949, link.
Title: Time-optimal trajectory planning with interaction with the environment
Author of the experiment: Vicenzo Petrone, Enrico Ferrentino, Pasquale Chiacchio
Description:
The UR10 robot executes a time-optimally planned trajectory, accounting for its velocity and effort limits. The trajectory involves interaction with the environment: the robot writes on a blackboard with some chalk. The execution relies on admittance control. The video shows details about the planned Maximum Velocity Curve and Phase Plane Trajectory, and analyzes the execution performance in terms of trajectory tracking and measured contact force.
More details in the paper V. Petrone, E. Ferrentino and P. Chiacchio, "Time-Optimal Trajectory Planning With Interaction With the Environment," in IEEE Robotics and Automation Letters, 2022, doi: 10.1109/LRA.2022.3191813, link.
Title: Planning of efficient trajectories in robotized assembly of aerostructures exploiting kinematic redundancy
Author of the experiment: Federica Storiale, Enrico Ferrentino, Pasquale Chiacchio
Description:
Optimal trajectories are planned for the Fanuc M20iA/35M robot placed on a slide in the LABOR cell, simulating a sequence of operations on a fuselage. The planning is made through a dynamic programming algorithm that exploits the kinematic redundancy given by the slide to minimize the joint displacements and to maximize the stiffness at the working pose, while respecting joint position and velocity bounds and avoiding collisions. It is shown that a higher trajectory duration allows the robot to move its kinematic structure to achieve a stiffer posture without violating joint limits. Then, a comparison is made with a traditional approach based on the heuristics of fixing the slide in front of the holes to be drilled. In this case, time, stiffness and/or safety must be sacrificed for complex hole pattern geometries, while the dynamic programming solution allows for a faster, smoother, stiffer and safer movement of the whole kinematic chain along the desired paths.
More details in the paper F. Storiale, E. Ferrentino, P. Chiacchio, “Planning of efficient trajectories in robotized assembly of aerostructures exploiting kinematic redundancy”, Manufacturing Review, 8, 8, 2021, link.
Title: Redundancy resolution for energy minimization and obstacle avoidance with Franka Emika's Panda robot
Author of the experiment: Enrico Ferrentino, Federico Salvioli
Description: Globally-optimal trajectories along a pre-scribed path are planned for Franka Emika's Panda robot using dynamic programming. On a first trial, the square norm of joint velocities is minimized, to achieve indirect minimization of energy. In this case, the robotic arm collides with the wall on the side. On a second trial, a second objective function is added, corresponding to the distance from the obstacle, in a Pareto-optimal setup. The solution corresponding to the minimum norm of the objective vector is shown in the video. In this case, the robotic arm avoids the obstacle.
Source code here.
More details in the paper Ferrentino, E.; Salvioli, F.; Chiacchio, P. Globally Optimal Redundancy Resolution with Dynamic Programming for Robot Planning: A ROS Implementation. Robotics 2021, 10, 42, link.
Title: Case study of constructive commissioning
Author of the experiment: Luigi Ferrara, Francesco Basile
Description: The virtual process in the video is made of three conveyors, each equipped with three retroflective sensors, transporting boxes from the left to the right. The process must be controlled to fulfil two specifications: (1) At most two robots can simultaneously work; (2) A robot can not grasp more than two boxes consecutively if one of the other robots has not grasped a box At this aim, a supervisor control architecture is adopted. First, the supervisor is designed either as an automaton (using Supremica) or a Petri net (using PetriBaR). Then, the toolbox SUP2PLC (available at the software section) is used to automatically generate the PLC code, starting from a supervisor model. Finally, the closed-loop system is simulated: a virtual PLC is instantiated with PLC-SIM (Step 7), and the code is run. I/O signals are exchanged with Factory I/O through TCP/IP. More details in the paper F. Basile, L. Ferrara (2020), From supervisory control to PLC code: a way to speed-up Constructive/Virtual Commissioning of Manufacturing Systems, 15th IFAC Workshop on Discrete Event Systems (WODES 2020)
Title: Execution of time-optimal trajectories with the Franka Emika's Panda robot
Author of the experiment: Enrico Ferrentino, Heitor Judiss Savino
Description:
Time-optimal trajectories are planned with a dynamic programming algorithm that exploits kinematic redundancy to minimize trajectory tracking time. The trajectories are planned for Franka Emika's Panda robot by considering a dynamic model with friction together with joint velocity, acceleration, jerk, torque and torque rate bounds. The planned trajectories are executed on the real robot to assess feasibility and tracking accuracy. Feasibility is obtained through interpolation and smoothing of the planned joint position references, while the tracking error in the joint space is lower than 0.8 degrees. Two trajectories are planned corresponding to a straight line of 0.50 m and an ellipse-shaped line of 1.45 m. Execution times are 0.62 s and 1.70 s respectively.
More details in the paper E. Ferrentino, H. J. Savino, A. Franchi and P. Chiacchio (2023), "A Dynamic Programming Framework for Optimal Planning of Redundant Robots Along Prescribed Paths With Kineto-Dynamic Constraints", in IEEE Transactions on Automation Science and Engineering, link.
Title: Decentralized multi-arm planning
Author of the experiment: Alessandro Marino & Jolanda Coppola
Description: A two-layer decentralized framework for kinematic control of cooperative and collaborative multi-robot systems is developed and tested. The motion of the system is specified at the workpiece level, by adopting a task-oriented formulation for cooperative tasks. The first layer computes the motion of the single arms in the system. In detail, the control unit of each robot computes the end-effector motion references in a decentralized fashion on the basis of the knowledge of the assigned cooperative task and the motion references computed by its neighbors. Then, in the second layer, each control unit computes the reference joint motion of the corresponding manipulator from the end-effector reference motion.
Setup: 2 Comau SmartSix robots installed at the Automatica Laboratory of University of Salerno.
Title: Cooperative drilling of aeronautic hybrid stacks
Author of the experiment: Alessandro Marino
Description: The proposed technology consists in a general robot architecture and a cooperative drilling process using only standard low-cost robots and off-the-shelf components. A first robot is in charge of drilling the hybrid stack, while a second manipulator ensures the right clamping force between the parts of the stack. Both robots are equipped with force control capabilities to control the generalized forces raising during the interaction with the stack. Thanks to the adoption of a fuzzy inference system, the tuning of the force controllers might be carried out by operators that have knowledge of the drilling process but not of control system technology.
Setup: 2 Comau SmartSix robots equipped with force/torque sensors installed at the Automatica Laboratory of University of Salerno.