...SeArChInG WaYs To GeT ThErE...

Image Processing is one of the emerging areas in the field of Information Technology and people yet exploring the limits of it. It is heavily used in major areas of IT such as Computer Vision, Artificial Intelligence, Computer Security etc. Recently I got an opportunity to work with Image Processing Technologies as a part of one of my projects and my task was to find matching images from an image store when a new image is given. I started my project with googling "How to compare images using java" and I got some good articles on finding the similarity of two images. Allmost all of them were based on four basic steps, they are:

1. Locating the Region of Interest (Where the Objects appear in the given image),
2. Re-sizing the ROIs in to a common size,
3. Substracting ROIs,
4. and Calculating the Black and White Ratio of the resultant image after subtraction.

Though this sounds as a good algorithm to compare images, it took a considerable amount of time after implementing it using JAI in my project. Therefore I had to find an alternate method of doing it. After searching the Web for few hours, I came across with this amazing article by Rafael Santos on "How do I compare two images to see if they are equal?" in his famous (but not yet completed) book "Java Image Processing Cookbook" which describes a different approach to compare images for similarity using image segmentation and region manipulation. It mainly focuses on finding some regions of a given image that matches with the images in the image store rather than searching for equality of the objects in given images. He describes an algorithm to find similar regions in a set of images and provides some sample code based on image segmentation and region manipulation to compare the equality of two images. After reading the article and playing with the sample code given there, I was able to further extend it to suit my need. So I wrote this note to express my gratitude towards him :-) The algorithm proposed by him (which is also available in his book) is as follows.

1. Pre-process the image, if needed (e.g. to enhance contrast, filter noise, etc.).
2. Do an Image Segmentation, process in which the image is converted to regions which contains pixels that are similar to pixels in the same region and different from pixels to other regions. This can be done using region-growing, mathematical morphology, clustering or classification algorithms. There are many algorithms to do that, just google for "image segmentation" and other keywords to get more information.
3. With the regions, create descriptors for them. Descriptors are calculated from the region and can include shape, area, perimeter, number of holes, general color of the region, texture, orientation, position, etc.
4. If needed, do a Re-Segmentation of the image, process in which regions are merged if they can be considered as belonging to the same object. Note that this step may require some high-level knowledge of the objects and the task in general, seldom being fully automatic and often being task-dependent.
5. If needed, filter the regions that seem relevant to the task in hand, eliminating small regions or regions which are deemed unrelated to the task (again this may require some knowledge about the task).
6. Store the image's regions' descriptor for further processing. Repeat those steps for other images.
7. Use the descriptors for comparison of the contents of the images, using some of many algorithms for pattern matching, classification, clustering, artificial intelligence and data mining in general.