PASCAL VOC datatsets - two new features added

PASCAL VOC datatsets - two new features added

Posted by Rebecca Martin on Mon, 10/02/2014 - 10:58

We are pleased to announce two new features for the PASCAL Visual Object Classes evaluation server:

1) Leaderboards (similar to the scoreboard comparisons for evaluations on the Middlebury datasets) are now available for the VOC 2010, 2011 and 2012 datasets, see http://host.robots.ox.ac.uk:8080/leaderboard/ .
A submission is by default private, but you can choose to "publish"
it to the relevant leaderboard.

2) The ability to generate an anonymized URL, suitable for inclusion in a conference submission, giving the performance summary for a submitted entry.

For each year there are 12 leaderboards, corresponding to the Object Classification, Object Detection, Object Segmentation, Human Layout, and two Action Classification challenges, either for submissions trained only on PASCAL VOC data, or using additional data.

At the moment the results reported are from the VOC 2010-2012 competitions, but there is the opportunity to add your results to the leaderboard by making either a new or previous submission public. To do this, you login to the evaluation server site http://host.robots.ox.ac.uk:8080/, select an entry from your submissions page, and press the "publish scores" button. On the same page you will see the anonymous result link.

Please also note the best practice guidelines at http://pascallin.ecs.soton.ac.uk/challenges/VOC/index.html#bestpractice

In order to test the statistical significance of performance differences, we have developed methodology to assess this via bootstrapping, see http://pascallin.ecs.soton.ac.uk/challenges/VOC/pubs/bootstrap_note.pdf .
We are currently working on adding functionality to the leaderboard site to compute and display these equivalence classes.

Please address any feedback on these new features to Yusuf Aytar, yusuf(at)robots.ox.ac.uk .

Luc Van Gool, Chris Williams, John Winn, Andrew Zisserman, Ali Eslami, Yusuf Aytar PASCAL VOC team