Restricted set classification (RSC)
RSC is about assigning labels to multiple objects presented together in a set. Denote the possible classes by 1, 2, ..., c. The catch is that we know that within X, there may be at most k_i objects form class i (i = 1,...,c).
|
Who-is-who?
Have you ever wondered how a team filming a documentary about a group of wild animals recognise each individual animal in the group? The animals are so alike that it is nearly impossible to tell them apart! An automatic classifier will likely make a lot of mistakes. But if you try to recognise the animals from a single photo, you have extra information! There can only be one of each in the group. In other words, if your classifier returns two Georges, you know that it is wrong. Can we improve the accuracy by taking the extra information on board? (See [1]) |
|
Who-is-missing?
Now suppose that, this morning, only some of the animals are there. We need to know who is missing. How can we find out? (See [2]) |
|
Who-is-there?
What if we are monitoring similar species and, for a particular animal in the photo, we only need to know which species it belongs to. Say, there are fish in a tank and we would like to monitor the behaviour of each species. Our classifier will only need to recognise the species. But in this case we know more: there are only 11 guppy, 8 tetra and 9 zebra fish (not in the photo on the left). If our classifier returns 10 zebra fish from one photo of the fishtank, we would know that something is wrong. We wouldn't notice this error if each fish was classified independently. (See [3]) |
|
And here is an interesting application example. Viewers of live chess matches can take a snapshot of the board with their phone camera. A restricted set classifier will be able to recognise and position the pieces on the board. This is the perfect example of the who-is-there task. Pieces may be missing. There are 8 pawns of each colour, two bishops, two rooks, etc. Our research verified the success of the idea of RSC. (See [4]) |
Code
https://github.com/LucyKuncheva/Restricted-Set-ClassificationName that guy!
Data and code about individual animal recognition from video are available here:
https://github.com/LucyKuncheva/Animal-Identification-from-Video
References
- Kuncheva L.I., Full-Class Set Classification Using the Hungarian Algorithm, International Journal of Machine Learning and Cybernetics, 1 (1), 2010, 53-61. pdf [bib]
- Kuncheva L.I. and A.S. Jackson, Who is missing? A new pattern recognition puzzle, S+SSPR, 2014, Joensuu, Finland, Springer, LNCS 8621 243–252. pdf [bib]
- Kuncheva L. I., J. J. Rodríguez and A. S. Jackson, Restricted Set Classification: Who is there?, Pattern Recognition, 63, 2017, 158–170. pdf [bib]
- Kuncheva, L. I. and J. H. V. Constance, Restricted Set Classification with prior probabilities: A case study on chessboard recognition, Pattern Recognition Letters, 111, 2018, 36-42. pdf [bib] (data and images in GitHub)