MRover — autonomous keyboard typing

A new competition task for the 2025–26 season: position the rover manually from fifteen feet, then everything after is autonomous — approach a keyboard, align, and type a ten-character sequence handed to you at the task. You don't know which keyboard in advance, the camera rides near the end effector, and scoring is all-or-nothing: one wrong key voids the sequence.

Rather than read the keys, find the keyboard's four corners and extrapolate every key from the standard QWERTY layout — the geometry does the work. A teammate's idea made it robust: render the training set synthetically from 3D keyboard models in Blender, which gave us roughly 12K images covering conditions worse than anything realistic (near-parallel angles, only a sliver of the face visible). The detector is a ~20M-parameter network — scaling it up bought nothing once the problem was framed as corner detection. A peer and our subteam lead backed OCR instead; we settled it with a head-to-head in the team simulator rather than an argument. Corner detection got more characters right in roughly eight of ten runs, and on full sequences something like six of ten against two.

Corner detection shipped into the rover's control loop on the Jetson alongside ROS 2 and landed some successful sequences at competition — not reliably; the grading was harsh. What I got wrong was treating it as either/or: right as the effector closes on a key the camera loses the keyboard, exactly where OCR would be near-perfect as a confirmation step, and a failed confirmation is recoverable because the layout tells you how far to move. This season we're building the hybrid — corner detection primary, OCR as the close-range check, on their code.