Drives a 6-DOF arm to draw shapes on a whiteboard mounted on a moving linear cart, without stopping the target or fixing the workpiece. I built the full stack: optical 3D tracking, sensor fusion, and a predictive controller.

Architecture

• Tracking: an optical tracker (96 Hz, ±0.08 mm) reads passive marker frames on the end-effector and on the target board.
• Endpoint open-loop PBVS: the end-effector pose comes from forward kinematics (clean), and the target pose comes from vision (noisy). The control error is only ever the difference between the two.
• LTV-MPC: every cycle (100 Hz, OSQP), the controller re-linearizes and solves a QP to track the moving target within velocity limits.
• Dual Kalman filters: an LKF for translation and an ESKF for orientation turn the raw, noisy pose into clean velocity, cutting noise by 26% at only 20–60 ms of lag.
• Compliant end-effector: a spring-loaded pen holder absorbs surface and depth errors passively, so no force sensor is needed.

Results

• 0.24 mm shape error tracing a static circle (10-trial repeatability).
• 26% noise reduction from the Kalman filters, at about 20–60 ms of lag.
• Sub-millimeter accuracy holds across ±40° of board tilt: orientation barely affects it.
• 0.8 mm shape error tracking a continuous curve at a target speed of 6 cm/s. Sharp corners degrade faster (4.9 mm).
• 4 cm/s is the recommended operating limit for reliable tracking of complex geometry.