High-Speed Dexterous Manipulation with the ORCAv2 Robot Hand
The project has two parts. First, we taught an ORCAv2 hand, driven by Feetech servo motors, to rotate a cube in its fingers. Then we designed and built a second hand around stronger and faster quasi-direct-drive (QDD) finger actuators to push manipulation speed as far as we could.
Details
- In-hand cube rotation: trained the policy with massively parallel reinforcement learning in Isaac Lab and PyTorch, then transferred it from simulation to the physical hand, where it reaches 30 rotations per minute on average.
- High-speed hand: redesigned the hand mechanically and electrically for the QDD actuators. I developed the C++ CAN FD driver and ROS 2 interface for them, including joint-to-motor mapping, telemetry handling and a 600 Hz control loop.
- The new hand clicks a computer mouse more than 2,000 times per minute and fastens a nut onto a screw at a very high speed.
Relocalizing an Event Camera in a Map Built from Ordinary Images
Standard cameras struggle in the dark, under glare and during fast motion, while event cameras keep working there. We build a 3D map from RGB recordings made in ideal conditions, and then localize an event camera within that map when conditions are far from ideal, for example in the dark.
Details
- Map: a Structure-from-Motion reconstruction built with COLMAP from the RGB frames of the EDS dataset, which provides frames, events and ground-truth poses from the same viewpoint.
- Baseline: reconstruct intensity images from short event windows with E2VID, then run a standard image-localization pipeline: global retrieval, local feature matching, and PnP with RANSAC.
- Direct event-to-image matching: match events against the map images directly with the EI-Nexus event-image matcher, skipping the image reconstruction step, and fine-tune the matcher for this setting.
- Evaluation: compare the two approaches by localization success rate and pose error, and study how event-window length, event representation, motion speed and illumination affect them.
Scalable Road Slope Mapping
A multi-sensor fusion method that maps road slope from the sensors a vehicle already has, without needing any vehicle parameters.
Details
- Combines an Extended Kalman Filter on IMU, velocity and wheel-speed data with Bayesian fusion of repeated traversals of the same road.
- Adaptive covariance weighting rejects suspension-induced pitch motion.
- Validated over 68 km of public roads against ground truth: 0.13° RMSE, maximum error below 1°.
Formula Student Driverless Race Car
Our software stack, written in C++ on ROS 2, is split into state estimation, perception, planning and control. I joined the team as an autonomous control engineer and later led the Autonomous Systems and Control group responsible for the whole stack, while remaining one of its main developers.
Details
- Path tracking on the car: a lateral controller that blends the Stanley and Pure Pursuit steering laws depending on the upcoming curvature, giving faster simulated laps than either alone.
- Model predictive control: three nonlinear MPC variants (reference tracking, contouring and curvilinear) with FORCESPRO. In simulation they cut the RMS lateral error from 0.62 m to 0.19 m against the Pure Pursuit baseline, at solve times around 3 ms.
- State estimation: a 100 Hz Extended Kalman Filter on a kinematic point-mass model, built in Simulink, fusing LiDAR odometry, GPS and IMU data.
- SLAM: deployed FAST-LIO2-based LiDAR-inertial SLAM on the car, first coping with a sparse and unreliable 16-beam LiDAR, then keeping it real-time with the much denser point clouds of a Seyond LiDAR.
- Perception: proposed and led the development of the LiDAR-camera fusion.
- Infrastructure: led the migration from ROS 1 to ROS 2 and introduced a Docker development environment adopted across the team.
- Result: 5th place in Autocross and in Trackdrive at Formula Student East 2024.