Over the course of its rich history, object tracking has been tackled under many disguises: multi-object tracking, single-object tracking, video object segmentation, video instance segmentation, and more. A significant majority of these disguises are closed-world benchmarks where the methods are expected to detect and track a predefined set of frequently observed classes. However, an intelligent system should go beyond a closed-world setting: It should be able to detect and track objects that have never been seen before. To this end, we have introduced TAO-OW: Tracking Any Object in an Open World benchmark for measuring tracking performance in an open-world setting where all objects must be tracked. Methods are specifically evaluated on how well they can track object classes that were missing from the training set (unknown objects), as well as objects which were in the training set (known objects).
|Time||June 18, 2023|
|Venue||CVPR 2023 Vancouver, Canada|
|09:00-09:20 PST||Organizers||Introduction, Challenge Description|
|09:20-09:50 PST||Fisher Yu||TBA|
|09:50-10:00 PST||TAO Long-Tail Challenge 2nd Place Winner||TBA|
|10:00-10:30 PST||Adam Harley||TBA|
|10:30-10:40 PST||TAO Open-World Challenge 2nd Place Winner||TBA|
|10:40-11:00 PST||Coffee Break|
|11:00-11:30 PST||Zeynep Akata||TBA|
|11:30-11:40 PST||TAO Long-Tail Challenge 1st Place Winner||TBA|
|11:40-12:10 PST||Laura Leal-Taixé||TBA|
|12:10-12:20 PST||TAO Open-World Challenge 1st Place Winner||TBA|
|13:40-14:10 PST||Jiri Matas||TBA|
|14:10-14:20 PST||TAO Long-Tail Challenge Submission Track Best Paper||TBA|
|14:20-14:50 PST||Du Tran||TBA|
|14:50-15:00 PST||TAO Open-World Challenge Submission Track Best Paper||TBA|
|15:00-15:20 PST||Closing Remarks / Award Ceremony||Organizers|
|15:20-16:00 PST||Round table: Quo Vadis, Tracking?||All Speakers|
We are excited to announce two Multi-Object Tracking (MOT) competitions: the Long-Tail Challenge and the Open-World Challenge. With these challenges, we aim to advance multi-object tracking and segmentation research in challenging few-shot and open-world conditions.
We base our challenges on TAO (Tracking Any Object) dataset and BURST (Benchmark for Unifying Object Recognition, Segmentation, and Tracking) video segmentation labels. We provide 2,914 videos with pixel-precise labels for 16,089 unique object tracks (600,000 per-frame masks) spanning 482 object classes!
In the Long-Tail Challenge, we focus on tracking and classifying all objects within the TAO/BURST object class vocabulary. In the Open-World Challenge, we investigate multi-object tracking and segmentation in a setting where labels for only a subset of target classes are available during model training. All objects need to be tracked but not classified.
In summary, the Long-Tail and Open-World Challenges offer a unique opportunity for researchers to investigate how far we can get with object tracking in long-tailed and open-world regimes and advance the field.
The submission deadline for both challenges is June 5th, 2023. Participants can submit their results through the MOTChallenge platform. Winners will be invited to present their work at our workshop.
In the Long-Tail Tracking Challenge we ask to track and classify all objects specified in the TAO/BURST object class vocabulary. Models can leverage labeled data for all 482 semantic classes during training. The challenge emphasizes the long-tail distribution of object classes, with a few classes occurring frequently and the majority occurring rarely. Participants are expected to develop methods that can handle long-tail distribution and are robust to highly imbalanced datasets.
The challenge’s goal is to advance the state-of-the-art in multi-object tracking and segmentation. Participants are encouraged to use creative and innovative approaches to achieve the highest possible performance on this challenging dataset.
The Open-World Challenge focuses on multi-object tracking and segmentation in a setting where only a limited number of labeled classes are available during training (see Opening up Open-World Tracking Paper). This is a challenging problem, as methods need to track all objects, including those not presented as labeled instances, during the model training.
Unlike the Long-Tail Challenge, in the Open-World Challenge, we (i) limit the number of labeled classes used for model training (i.e., only labels for classes within COCO class vocabulary can be used), and (ii) do not require classifying tracked object instances.