1. Introduction

Digital image collections are traditionally searched used textual queries (e.g. Google). Text is ideal for describing the objects present in an image, but is poor at describing the visual appearance of an image. Yet it is often desirable to search based on visual appearance, e.g. when shopping online.

Your assignment is to write a program to visually search an image collection. Your program will accept an image as a search query, and return a list of images in the collection ranked according to their similarity. You may choose to display only the top 20 images. You must demonstrate your system working using several different query images.

You must use the Microsoft Research (MSVC-v2) dataset of 591 images (20 classes) which can be downloaded from SurreyLearn  or  direct  from  MSR  here: http://download.microsoft.com/download/3/3/9/339D8A24-47D7-412F-A1E8-

1A415BC48A15/msrc_objcategimagedatabase_v2.zip

Please note the above download is 110 Mb, zipped.

2. Deliverables

You must write software in MATLAB to perform this task. The requirements specification of this software is given later; the more challenging the requirements implemented (successfully), the higher the mark you will receive.

You must document your software describing the techniques you have chosen and how you implemented them (including how you overcame any difficulties). Most importantly, your document must include evidence of testing.

50% of marks will be awarded for software (i.e. your code), and 50% for your documentation and testing. A more detailed marking scheme is included in Section 6. This coursework is worth 20% of your overall mark for EEE3032.

This year we have moved to electronic submission of the coursework on SurreyLearn . It is important that you submit both your written documentation and your code . You should use a zip file to package up your code and include a short text file called “README.TXT” that explains in a paragraph or two how to run it.

The deadline for this coursework is 4PM on Tuesday 16th  November (Week 8)

3. Resources

You are supplied with two skeleton MATLAB programs (on SurreyLearn) enabling you to start coding quickly.

1) cvpr_computedescriptors

This program will iterate through every image in the dataset and generate an image descriptor from it by calling the supplied function “extractRandom” . As supplied, that function generates a set of random numbers instead of an image descriptor.  You  should  replace  “extractRandom”  with  your  own  code  for  extracting  your  chosen  type  of  image descriptor. The descriptors are all saved into a separate folder. You can configure folder names in the code.

2) cvpr_visualsearch

This program will load in all the image descriptors computed by “cvpr_computedescriptors” . It will then pick an image at random to use as a query. It will repeatedly call “cvpr_compare” with the query descriptor and the descriptor from each image. Currently this function returns a random number. You should edit it to return the distance between the features  being  compared.  The  program  “cvpr_visualsearch”  will  visualise  your  top  10  results  in  a  basic  way. Depending on the level of sophistication of your coursework solution, you might improve this.

The purpose of the skeleton code is to get you started quickly. You should be able to create a working basic visual search system by replacing “extractRandom” with a global colour histogram (slide 4, lecture 7) and “cvpr_compare” to output Euclidean distance between the two features it is passed . This will meet Requirement 1 (see section 6. You might then go on to enhance the “cvpr_visualsearch” code to produce precision-recall statistics, and then try other descriptors or distance metrics to improve performance (you can measure the improvement using your aforementioned precision-recall code).

4. Learning Outcomes

After completing this coursework, and attending the relevant lectures, you should be able to:

1.   Select and apply appropriate computer vision techniques to extract descriptors from images.

2.   Select and apply appropriate pattern recognition techniques to match images using their descriptors.

3.    Evaluate a visual search system using standard information retrieval metrics.

4.    Conduct a scientific investigation into the relative performance of various approaches to visual search.

Please note that MATLAB is a vehicle for trying out Computer Vision ideas – we will mark you on your achievements and experimentation in visual search. We will not be assessing neatness of code or programming style (but your code must be legible for marking).

5. Plagiarism

You must complete this coursework individually. If you rip code or text from the web, or another student, and include it in your project without clear attribution, then you have committed plagiarism. Undetected plagiarism degrades the quality of your degree, as it interferes with our ability to assess you and prevents you learning through properly attempting the coursework. Consequently if we suspect plagiarism you will referred to an Academic Misconduct Panel, which for a first offence, may result in you receiving zero marks for the coursework and your mark for the entire module being capped at the pass mark (40%).

Please try not copy figures and text from lecture notes or internet sources for your documentation, even if you cite its source. Whilst not plagiarism this is still poor practice – if you wish to explain a concept please draw your own figures and use your own words. Cite any material you might have found on the internet to back up your statements. This use of citation will gain you credit. Re-using some one else’s material will not be worth credit.

6. Requirements Specification and Marking Scheme

You are provided with basic code for iterating through files in the Microsoft dataset. You must enhance this basic code to implement as many of the following functional requirements as possible.

Implement feature 1, then feature 2, and then choose any other features 3-5,8 to implement (in any order)

We do not expect you to implement all features in the time available. In particular features 6-7 require major additional work beyond expectations of this coursework; only 1 or 2 students each year successfully attempt this.

A  mark  budget  is given  for  each  requirement  –  for  example,  if you  complete  1  perfectly  (i.e.  bug  free,  and your documentation is flawless) then you gain 30 marks (15 for the implementation, 15 for documentation). Completing a few

features well is much preferred over attempting many features poorly.

You cannot earn more than 100% as the final mark for this coursework!