Session 5A – Robot Learning
Date: Thursday, 19th November 2026
Time: 14:00 – 15:30
Room: Sala d’Actes
Session Chair: TBA
Presentation format: Each paper is allocated 15 minutes: approximately 10–12 minutes for the presentation and 3–5 minutes for questions.
14:00 – Battery-Aware Task Scheduling for Robotic Fleets using Reinforcement Learning
Authors: Miguel Marmelo¹, Catarina Rema², Pedro Costa¹, Gonçalo Meneses³ and Vaz Martins⁴
Affiliations: ¹ Faculty of Engineering, University of Porto, Porto, Portugal; ² INESC TEC – Institute for Systems and Computer Engineering, Technology and Science, Porto, Portugal; ³ CeNTI, V. N. Famalicão, Portugal; ⁴ Apineq, Maia, Portugal.
Abstract
With the increasing adoption of AMRs in industry 4.0, the need for highly efficient coordination strategies arises. Traditional MRTA methods often rely on short-term focused heuristics that prioritise immediate throughput, neglecting long-term energy efficiency and battery degradation. Task scheduling for multi-robot systems in dynamic industrial environments requires simultaneous management of robot coordination, battery constraints and task urgency. This paper proposes a RL-based task scheduling architecture, integrating a cost function designed to simultaneously optimise task execution time and battery-feasible fleet operation. In this approach, battery management is integrated natively into the learned policy through a unified state representation and action masking. The model is extended to handle priority tasks, which must be addressed before non-priority work while respecting energy feasibility constraints. An attention-based architecture processes each task node as a feature vector comprising coordinates and a priority flag, enabling the model to learn coordinated scheduling strategies that balance makespan minimisation, battery safety and task priority. Experimental results over 1,000 dynamic missions show that the proposed RL policy outperforms the mission completion rate of an exact OR-Tools solver (99.8% vs. 97.0%) at a fraction of its computational cost, substantially outperforming a greedy NN baseline (74.3%).
14:15 – Reinforcement Learning-based MAV Navigation With Parameterized Waypoints: Incorporating Orientation, Speed Limits, and Corridor Constraints
Authors: André Kirsch and Jan Rexilius
Affiliations: Bielefeld University of Applied Sciences and Arts, Minden, Germany.
Abstract
Reinforcement learning has achieved state-of-the-art performance in MAV control, waypoint flight, and obstacle avoidance. However, existing RL approaches often assume fixed objectives and constraints, with flight behavior largely limited by vehicle dynamics and orientation considered only when required for locomotion. Classical planning and model predictive control handle such constraints explicitly, but require optimization or replanning. This motivates methods that combine learned local control with explicit constraint handling. We combine reinforcement learning with control barrier functions to improve constraint-aware execution. We propose parameterized waypoints that encode orientation, velocity, and corridor constraints. Simulation and real-world experiments show that a single policy can execute different constraint-parameterized navigation scenarios, revealing scenario-dependent trade-offs between traversal time, tracking accuracy, and constraint satisfaction.
14:30 – Residual Reinforcement Learning with a SINDy Surrogate for Magnetic Levitation Control
Authors: Hashem Ghorab¹ and Ayman El-Badawy²
Affiliations: ¹ Department of Engineering, German International University Berlin, Berlin, Germany; ² Department of Mechatronics Engineering, German University in Cairo, Cairo, Egypt.
Abstract
Reinforcement learning can rival classical control in simulation, yet PID remains the default in industrial and motion-control loops because it is interpretable, predictable, and well understood. This paper presents a residual control framework that augments, rather than replaces, a trusted baseline controller, combining a SINDy-identified dynamics model, a PID plus current-feedback baseline, and a Soft Actor-Critic residual policy trained with Dyna-style model augmentation. The learned residual is bounded and attenuated by an error-dependent action gate, and an error-gated reward aligned with the gate at a single threshold focuses learning on the regime where the residual is allowed to act. The framework is evaluated on an open-loop unstable magnetic levitation system across ten reference-tracking and disturbance-rejection scenarios. In simulation, the residual improves RMSE on the canonical velocity-disturbance scenario by 52.76% over the PID baseline, and by 21.5% on the common pre-failure window before the baseline reaches the gap bound, settling in 2.01 s where the baseline fails to recover, while contributing under 2% of control authority across the remaining scenarios.
14:45 – PISTY: Curriculum-Based Deep Reinforcement Learning of Ground and Aerial Striking in 1v1 Rocket League
Authors: Afonso Coelho¹,², Diogo Amaral¹,², Miguel Camocho Carvalho¹,², Gonçalo Leão²,³, Armando Sousa²,³ and Luís Paulo Reis²,⁴
Affiliations: ¹ FCUP – Faculty of Sciences, University of Porto, Portugal; ² FEUP – Faculty of Engineering, University of Porto, Portugal; ³ INESC TEC – INESC Technology and Science, Portugal; ⁴ LIACC – Artificial Intelligence and Computer Science Laboratory, Portugal.
Abstract
Autonomous control of agile vehicles in dynamic three-dimensional environments is a demanding problem for both classical controllers and standalone Deep Reinforcement Learning (DRL), due to continuous dynamics, complex physics, and extreme reward sparsity. We present PISTY, a car-like agent trained with DRL to perform ground and aerial striking maneuvers in a competitive 1v1 Rocket League simulation. To overcome the sparse feedback of terminal scoring events, we combine Proximal Policy Optimization (PPO) with a five-phase Curriculum Learning (CL) strategy whose phases are named after the game’s rank progression: Bronze (locomotion and ball control), Silver/Gold (ground scoring), Platinum (aerial interception), Diamond (competitive striking under near zero-sum rewards), and a Diamond+ refinement that reprioritizes final, goal-directed rewards. Training runs on GigaLearn, a high-throughput C++ framework, for roughly 4.2 billion timesteps, paired with self-play against a non-stationary adversary. The agent converges steadily across the five phases, scores consistently on the ground, and transfers to aerial striking while keeping optimization stable; it also makes increasing use of lightly rewarded bumps and demolitions to disrupt the opponent. These results posit structured task decomposition with dense-to-sparse reward shaping as an effective route to adversarial continuous control.
15:00 – From Language to Navigation Goals: A Vision-Language Approach for Semantic Navigation of Mobile Robots Using RGB-D Perception
Authors: Jose Martinez Fajardo, Pablo Pueyo, Fernando Caballero and Luis Merino
Affiliations: Univ. Pablo de Olavide.
Abstract
Natural language interaction provides an intuitive way for non-expert users to communicate with robotic platforms. However, transforming user requests into executable navigation actions remains a challenging task, requiring the integration of language understanding, environment perception, and autonomous navigation. This work presents a language-driven navigation framework that enables mobile robots to interpret user requests in natural language to move the robot to a destination, and autonomously navigate towards it. Given a natural language request referring to a target in the environment (e.g. “go to the mail box”), the system identifies the referenced object, estimates its position using RGBD data, and generates a navigation goal, which is then executed through the ROS 2 Nav2 navigation stack. The ROS 2-based implementation facilitates portability across different robotic platforms, requiring only the configuration of the corresponding topics and services. The system is evaluated in both simulation and real-world scenarios using a TurtleBot3 Waffle and a Unitree Go2 robot with a Realsense camera. Experimental results show that the framework successfully interprets both direct commands and contextual requests, generates meaningful natural-language feedback, and navigates towards the desired target. These results demonstrate the feasibility of combining semantic perception and autonomous navigation to provide an intuitive human-robot interaction paradigm.
15:15 – Embedded Seam-Tracking Segmentations on Edge Accelerators
Authors: Lucas Souza Oliveira Paula, João Braun and José Lima
Affiliations: CeDRI, SusTEC, Bragança Polytechnic University, Campus de Santa Apolónia, 5300-253 Bragança, Portugal.
Abstract
Visual navigation along welded waterproofing membranes requires a seam reference updated at a rate compatible with robot motion. This work runs the same YOLOv8n-seg weights (512×512 input) on three embedded configurations: the Hailo-8 and the Coral USB Edge TPU in INT8, and the Raspberry Pi 5 CPU in FP32 through ONNX Runtime. On common offline inputs, the mean intersection over union (IoU) with manual labels was 0.595, 0.539 and 0.605, and the median visual cycle 26.5, 97.2 and 164.4 ms. In 72 closed-loop trials on a differential-drive robot, Test 1 scaled speed and gains to the offline cycle of each configuration: all three tracked the seam in 3/3 trials, acquiring it in 1.6, 5.8 and 9.9 s, with executed lateral offsets of 15, 10 and 9 mm measured in an overhead video. In Test 2, with identical parameters, all three tracked at 0.10 and 0.15 m/s with steady-state errors of 7–9 mm; the highest speeds tracked in 3/3 trials were 0.45, 0.30 and 0.15 m/s, with 95th-percentile live cycles of 49.6, 129.7 and 320.5 ms. With the same controller, the perception update rate set the operating speed range.