Use of Color and Pattern Features ... Similar Coral Reef Fish Species ...


Edward D. Pavoni
Andrew Rapport
Serena W. Tsang

Prof. Joseph Wider, Prof. Gareth Russell (NJIT) ; Capstone advisor: Prof. Dario Pompili


World- wide degradation of coral reefs has raised concerns, not only about the reefs themselves, but also for the marine life they support. The long-term goal of the current project is to develop a network of underwater sensor modules to collect and analyze data over time to track reef fish populations, in order to gain understanding of the effects of environmental changes on these populations. A proof-of-concept prototype of such a module including video cameras and illumination sources has been developed (under NSF Grant # DBI-0649878) and tested here at a the Rutgers Institute of Marine and Coastal Sciences and installed for further testing and data capture at the New York Aquarium where a database of approximately 2,100 images of 16 species of coral reef fish has been collected.

Goal of the Capstone Project:

Among the challenges in identifying free-swimming fish by species are variations in illumination, pose, within-species differences in color and patterns and similarities in shape between fish of different species within the same family. Currently, image pre-processing, feature extraction and classification algorithms have been developed and tested, using fish shape and overall color content as features and the results are promising. However, the ability to distinguish species from the same families having identical shapes but differing in the location and shape of various patterns and colors found in different regions of the fish’s body are essential to the improvement of our recognition results. The Capstone project addresses that challenge It develops additional features that examine various regions of within the perimeter of the fish bodies to provide the type of fine-scale information that can significantly improve the probability of correct classification of fish by species. Using features derived from transform coding of grey-scale projections and from localized opponent color histograms, we can resolve these within-family differences between species. These new features are currently being tested on two pairs of frequently confused species from the same family. i.e. the foureye butterflyfish and banded butterflyfish, and the porkfish and bluestriped grunt from the Grunt family. When combined with the results of the higher-level classification we see evidence of significant improvements in our classification results. As the database expands to include more species, this same type of detailed analysis of internal patterns and colors should be useful in improving results across the board.

(a) Frame concept design. (b) Initial test of submersible system. Recognition Challenges: within species
(c) System operating at New York Aquarium coral reef exhibit. (top 3 rows), across species (bottom 2 rows)
(d) System instrumentation enclosure.