- Spot the match – wildlife photo-identification using information theory

Whale shark photo library
The library contains 797 photos taken by researchers and tour operators during the months of March–July from 1992–2006 at Ningaloo Reef (22º 50’ S, 113º 40’ E), Western Australia. The method of image capture varied over time, so that still, video and digital images were all included in the library. A 'by-eye' comparison of 581 images in this photo library, (this total excludes several images collected in the 2001 season, as well as all photos collected between 2003 and 2006), was originally completed. During analysis, photos were sorted into quality classes on the basis of clarity, angle, distinctiveness, partial image and overall quality [39]. More details of the manual matching procedure are provided in reference [19].


Matching software and fingerprint creation
The software we used to generate potential image matches was originally designed to match natural variation in spot patterns of grey nurse sharks (Carcharias taurus – also known as the "ragged-tooth" in South Africa and the "sand tiger" shark in North America) [31]. This software – Interactive Individual Identification Software (I3S) – creates 'fingerprint' files and matches individuals by comparing particular areas demonstrating consistent spot patterns. We chose to examine the area on the flank directly behind the 5th gill slit as the most appropriate for the individual identification of whale sharks. This decision was based on spot consistency identified in previous studies and due to the ease with which photographers can view this area [19,20]. The positioning of spots in this area was also less likely to be distorted due to undulation of the caudal fin, which may affect the software's matching success.

At least three reference points are required by I3S to construct a fingerprint [31]; we chose the most easily identifiable and consistent reference points visible in flank photographs: 1) the top of the 5th gill slit, 2) the point on the flank corresponding to the posterior point of the pectoral fin and 3) the bottom of the 5th gill slit (Fig. 1a). The requirement of all three reference points to be visible in the photograph for a fingerprint to be created meant that not all 797 photos could be used. As such, we could compare 433 (54%) of the original photographs, of which 212 were of the left side (LS) and 221 were of the right side (RS) of the shark.

In this updated database, images were matched by an operator highlighting spots within the reference area on a computer screen. Three initial reference points for each image were entered (Fig. 1a), followed by the manual adding of a digital point to the centre of the most obvious spots within the reference frame. Using a search function, the software compares the new fingerprint file against all other fingerprint files in the database by using a two-dimensional linear algorithm, which is simply the sum of the distances between spot pairs divided by the square of the number of spot pairs [31]. The matched spot pairs with the minimum overall score (ranging from 0 [perfect match] to a value 3S text output into the R Package [42] for further analysis [see Additional file 1].

Information criterion algorithm
To provide a measure of match parsimony based on the philosophy of information theory and to compare possible image matches in a multi-model inferential framework [27], we modified the match score in the following manner: (1) we first back-transformed the spot-averaged sum of distances to a residual sum of distances, which was simply the spot score (SS) multiplied by the square of the number of matching spots (n); (2) we then created an information criterion (IC) analogous to the Akaike and Bayesian Information Criteria [43,44]:

equation M1

where k = an assumed number of parameters under a simple linear model (set to 1 for all models) and n' = 100/n that accounts for the fact that an increasing number of spots automatically leads to a higher SS (the 100 multiplier scales the term to be >1); (3) finally, we calculated the IC weight (w) as:

equation M2

where ΔIC = IC - ICmin for the ith image (ith 'model') from 1 through m (where m = 49). We also calculated the information-theoretic evidence ratio (ER) [27] for each matched image relative to the top-ranked image based on the w to provide a likelihood ratio of match performance. Here, ER1 is the w of the top-ranked matched photograph divided by the next most highly ranked photograph's w, ER2 is the w of the top-ranked match divided by the w of the third-best match, and so on. Therefore, ER1 provides a likelihood ratio for the match of the top-ranked photograph relative to the next most highly ranked photograph.

Match validation
To establish the ability of the wi and ER indices to assign reliable matching, we endeavoured to establish a threshold value of w1 and ER1 below which matching uncertainty was too high to match photographs reliably. We therefore validated the approach by applying our algorithms to a sample of 200 images; 25 known matched pairs (i.e., matched by eye) from both the LS and RS databases (100 images total), and 25 non-matched pairs from both LS and RS databases (100 images total). The LS and RS images were analyzed separately, using text outputs from I3S that report the candidate matching image names, I3S matching scores and the number of spot pairs matched. A match was considered successful if the corresponding image was ranked at the top of the list of potential matches (i.e., number 1 of 50).


Assessing 'by-eye' matches using I3S
Thirty-three individual sharks were re-sighted inter-annually during the manual 'by-eye' analysis of the raw photo library. Of any two by-eye matched images, one of the pair was entered into either the LS or RS database and searched. A match using I3S was considered successful if the by-eye matched images were ranked as the most likely match (as with the validation test) and confirmed using the IC algorithm.


Horizontal angle (yaw)
Footage of 10 different sharks (5 LS and 5 RS) was used to capture sequences of five images of each shark, where subjects were on varying horizontal angles (0°, 10°, 20°, 30° and 40° – Fig. 2). The angles of yaw were estimated using Screen Protractor™ software. Fingerprints were created for each image with 20 spots annotated per fingerprint. The 10° images were searched against the 0° images and 10 non-matching images. This process was repeated, substituting images where subjects were on angles of 20°, 30° and 40° for both LS and RS image sequences. Five random, non-matching pairs were also searched against 0° and 10° images, and then repeated for 20°, 30°, and 40° images. This allowed for a comparison between matching and non-matching pairs while testing for the effects of horizontal angle in images. Results were analyzed using the IC algorithm applied to the match validation and by-eye comparison tests.


Number of spot pairs
Fifty known-matching pairs were compared to one another in I3S. Of these matching pairs, only those successfully confirmed during validation of I3S matches were included in this test. I3S scores were compared against the number of spot pairs matched. The w1 for each image was also compared against the number of spot pairs matched by the I3S algorithm. A complementary log-log transformation (clog-log) was applied to normalize the distribution of I3S scores and w1, and a log10 transformation was used to normalize the distribution of spot pairs. We tested for a linear relationship between the transformed variables using least-squares regression and information-theoretic evidence ratios. Goodness-of-fit was assessed using the least-squares R2 value.
Competing interests
The author(s) declare that they have no competing interest
Authors' contributions
CWS, CJAB and MGM designed the study, CWS and CJAB did the analysis, and all authors contributed to writing the paper. CWS did most of the analysis with assistance from CJAB, and CWS took the lead in writing the manuscript.

Supplementary Material
Additional file 1

R code to calculate Information Criterion (IC) weights for match parsimony. Full instructions for use of R code are contained within the text file.


We acknowledge the support of the whale shark ecotourism industry based in Exmouth and Coral Bay (Western Australia), the Natural Heritage Trust (NHT) Marine Species Recovery Protection fund administered by the Department of Environment and Heritage (Australia), Hubbs-SeaWorld Research Institute, BHP Billiton Petroleum, Woodside Energy, the U.S. NOAA Ocean Exploration Program, the Whale Shark Research Fund administered by the Western Australia Department of Environment and Conservation (DEC), the Australian Institute of Marine Science, NOAA Fisheries and CSIRO Marine and Atmospheric Research. We particularly thank E. Wilson, C. Simpson, J. Cary, R. Mau and B. Fitzpatrick of DEC, and the logistical support and advice of C. McLean, M. Press, A. Richards, I. Field, S. Quasnichka, J. Polovina, B. Stewart, K. Wertz, T. Maxwell, J. Stevens, S. Wilson and J. Taylor, as well as assistance with I3S by Jurgen den Hartog and Renate Reijns (I3S developers). This research was reviewed and approved by the Charles Darwin University Animal Ethics Committee, the Institutional Animal Care and Use Committee of Hubbs-SeaWorld Research Institute and the animal ethics committee of DEC. We thank D. Lohman, G. Taylor, D. Bickford and J. Kirwan for supplying images.

rating: 0.00 from 0 votes | updated on: 26 Apr 2007 | views: 6164 |

Rate article:

excellent! bad…