Abstract
Identifying where a photo was taken can be achieved by matching the query ground view image to a satellite image of known location. This has been done in the past using Siamese Neural Networks by training a model that embeds images into feature vectors and com-
paring the distance between resultant vectors within the space to find a close match. Historically the number of correct recalls within top k% of matches was used as a metric when testing these models, named Re-call at k (R@k). This paper highlights an issue with prior implementations of the canonical R@k metric related to boundary cases, leading
to the miscounting of recalled images. As a result, models that provide state-of-the-art performance when measured using R@k may yield poor qualitative results in practice. Therefore, this paper proposes a novel metric, Proportional Search Space Reduction (PSSR), which measures how much the search space is reduced by a model under assessment, and
has the potential to include boundary cases that R@k may miss. Three models were trained and evaluated to show that models with high Recall at 1% do not perform as well in real world applications as the metric may suggest, and proposes the use of PSSR for future research into the problem.
paring the distance between resultant vectors within the space to find a close match. Historically the number of correct recalls within top k% of matches was used as a metric when testing these models, named Re-call at k (R@k). This paper highlights an issue with prior implementations of the canonical R@k metric related to boundary cases, leading
to the miscounting of recalled images. As a result, models that provide state-of-the-art performance when measured using R@k may yield poor qualitative results in practice. Therefore, this paper proposes a novel metric, Proportional Search Space Reduction (PSSR), which measures how much the search space is reduced by a model under assessment, and
has the potential to include boundary cases that R@k may miss. Three models were trained and evaluated to show that models with high Recall at 1% do not perform as well in real world applications as the metric may suggest, and proposes the use of PSSR for future research into the problem.
Original language | English |
---|---|
Pages | 1-13 |
Publication status | Published - 9 Nov 2023 |
Event | BNAIC/BeNeLearn 2023: Joint International Scientific Conferences on AI and Maxhine Learning - Delft, Netherlands Duration: 8 Nov 2023 → 10 Nov 2023 https://bnaic2023.tudelft.nl/ |
Conference
Conference | BNAIC/BeNeLearn 2023 |
---|---|
Country/Territory | Netherlands |
City | Delft |
Period | 8/11/23 → 10/11/23 |
Internet address |