FieldMOT: A Field-Registered Multi-Object Tracking for Sports Videos

Accurate player movement data is crucial for both athletes and coaches in game analysis, providing valuable insights into decision-making, movement patterns, and tactical execution. This work addresses the challenge of multi-object tracking (MOT) in sports broadcast videos, where video clips are switched among multiple cameras. Most tracking frameworks operate in image space, making them vulnerable to camera movements and frequent transitions. To overcome this, we propose a "register-then-track" paradigm, as opposed to the conventional "track-then-register" approach. By leveraging the object detection capabilities of an open-vocabulary foundation model, our method retrieves field anchors, keypoints, and positional information from image frames, enabling more stable and consistent tracking across varying camera views. The new paradigm allows us to extract player trajectories directly from broadcast footage. However, unlike single-camera tracking setups, broadcast videos present the additional challenges of frequent camera switches and limited field coverage, which can disrupt consistent tracking. Through these field keypoints, our method is designed to mitigate these issues, ensuring more reliable tracking across multiple viewpoints. We evaluate our approach on a synthesized football dataset, demonstrating its effectiveness in tracking players, and further test it on a street view dataset, demonstrating its broader applicability beyond sports.