CS 4330: Computer Vision

Fall 2007 (FY08-1)

Introduction to the Course

Synopsis and Course Goal

This course provides an overview of Computer Vision – enabling computers to understand visual input. The course introduces the major concepts and methods, and gives you a sample of problem settings and techniques for solving them. It is intended for those who want to get an understanding of how computer vision works and how it can perform such diverse tasks as object recognition, robot navigation, and increase situational awareness. The course has strong ties between theory, examples of its application, and practical exercises in order to promote fast and persistent learning.

The course is intended for students with a basic proficiency in linear algebra and a programming language.
Example courses at NPS: MA2043 or MA3042

Course Learning Outcomes

What tasks do you need to master in order to achieve the overall course goal? Upon successful completion of this course, you will be able to:

Syllabus and Organization

Take a look at the Contents on the following pages to get an idea of the syllabus. Computer vision is a “hot topic” and rapidly evolving, so expect several modifications that reflect recent developments and advances. Every chapter is a learning module with specific learning objectives, reading and study material, exercises, and graded assignments.

Textbook, Web and Data Space

There is no required textbook for this class. We will work from a reader and several resources provided to you in class. Grades for homework assignments etc. will be on Blackboard. Software and other course material can be found here:
http://www.movesinstitute.org/~kolsch/courses/CS4330 and here:
\\comfort.ern.nps.edu\cs4330$

Hours

Lectures and lab:
(TBA) Monday 1500-1650, WA-275
(TBA) Wednesday 1500-1750, WA-275
These two time slots will be used for lectures and/or lab time. Please bring your laptop for every meeting.

Final:

Monday 12/10/07, 0800-0950 Bu-202

Holidays:

Mon 8 Oct 2007 (Columbus Day),

Mon 12 Nov 2007 (Veteran’s Day),

Thu 22 Nov 2007 (Thanksgiving)

Office Hours:

I am available for questions and help whenever I am in my office (WA-279) and have some time. If you would like a firm appointment, please don’t hesitate to set one up by email or phone (656-3402).

Grading

Grades for homework assignments etc. will be on Blackboard:
50% lab projects incl. reports
10% homework assignments, surprise quizzes (if any)
15% midterm
25% final

Homework, Late Policy, Teamwork etc.

Homework assignments will be posted on or before Mondays and are due the following Thursday at 11:59pm unless otherwise noted. No credit will be given for assignments that are more than 3 days late (Sunday 11:59pm). You can hand in at most one assignment up to 3 days late and still receive full credit. That is: one permitted late assignment for the entire quarter. Additional late assignment will be given a max of 50% credit.

Unless otherwise noted, written assignment can be handed in on a sheet of paper (preferred) or by email. If you email it to me, please make sure your name appears on top of the actual assignment (e.g., in the text file attachment), not just in the email. Programming assignment must be turned in using Blackboard’s Digital Dropbox. See the respective assignment assignment for the naming convention.

Collaborative work:

Reading Assignments

There will be weekly reading assignments from the reader and from other reading material. A “reading assignment” requires you to read the material and to write a short, one or two page summary. Demonstrate your understanding of the contents, don’t provide me with a complete recount of the entire text. You are encouraged to add a paragraph in which you critique the reading, note questions that you have or state some other related comment. Print the reports (duplex, please) and bring them to class on the respective due day. If you cannot make it to class, please email them to me prior to class.

Equipment

All special equipment and software will be provided to you:
High-performance laptop, Matlab with Image Processing Toolbox, USB or FireWire (1394) camera, OpenCV

Programming Projects

As part of the course, we will work on small programming projects with the help of a high-level computer vision library (OpenCV) to demonstrate some of the methods we have learned in class. Some sample projects are listed below, further ideas and detail can be found in Appendix C:

Resources

Textbooks

Textbooks, more or less in order of relevance (res: on reserve in the library)

  1. res: Linda G. Shapiro, George C. Stockman, Computer Vision (Prentice Hall; 1st edition, 2001), ISBN: 0130307963
  2. res: D. Ballard and C. Brown: Computer Vision (Prentice Hall, 1982)
  3. res: D. A. Forsyth, J. Ponce; Computer Vision: A Modern Approach (Prentice Hall), ISBN: 0130851981
    Since I was asked: there are no color pictures in this book.
  4. R. Hartley and A. Zisserman: Multiple View Geometry second edition, Cambridge University Proess. ISBN 0521540518
  5. Digital Image Processing (2nd Edition) by Rafael C. Gonzalez, Richard E. Woods Publisher: Prentice Hall; 2nd edition (January 15, 2002) ISBN: 0201180758
  6. Digital Image Processing Using MATLAB by Rafael C. Gonzalez, Richard E. Woods, Steven L. Eddins Hardcover: 782 pages Publisher: Prentice Hall; 1st edition (September 5, 2003) ISBN: 0130085197
  7. Robot Vision by Berthold K. P. Horn Hardcover: 480 pages Publisher: The MIT Press; Mit Press edition (March 13, 1986) ISBN: 0262081598 co-published with McGrawHill
    out of print
  8. Computer and Robot Vision by R. Haralick, and L. Shapiro Addison-Wesley, 1992
    two-volume book is tons of material, good references
  9. A Guided Tour of Computer Vision by Vishvjit S. Nalwa Addison Wesley, 1993
    great for concepts
  10. Machine Vision by R. Jain, R. Kasturi, and B. G. Schunck McGraw-Hill, 1995
  11. Machine Vision: Theory, Algorithms, Practicalities by E. R. Davies Academic Press, 1997
    chapter 2 is at
    http://www.palantir.swarthmore.edu/maxwell/papers/html/Maxwell-WUEIC-1997.htm
  12. Practical Computer Vision using C by J. R. Parker 1994
  13. J.E. W. Mayhew, J.P. Frisby; 3D Model Recognition From Stereoscopic Cues MIT Press, 1991, ISBN 0-262-13243-5 online at
    http://homepages.inf.ed.ac.uk/rbf/BOOKS/MAYHEW/
  14. M. A. Fischler, O. Firschein Intelligence: The Eye, the Brain and the Computer Addison-Wesley, 1987, ISBN: 0201120011 online at
    http://homepages.inf.ed.ac.uk/rbf/BOOKS/FISCHLER/
  15. a collection of online (and offline) computer vision/image analysis/imaging books at:
    http://homepages.inf.ed.ac.uk/rbf/CVonline/books.htm

Miscellaneous

Contents

Introduction to the Course
  Synopsis and Course Goal
  Course Learning Outcomes
  Syllabus and Organization
  Textbook, Web and Data Space
  Hours
  Grading
  Homework, Late Policy, Teamwork etc.
  Reading Assignments
  Equipment
  Programming Projects
  Resources
Contents
0 Administrativa
 0.1 Objectives
 0.2 Reading and Content
 0.3 Assignments
 0.4 Instructor Notes
1 Introduction to Computer Vision
 1.1 Objectives
 1.2 Reading and Content
 1.3 Assignments
 1.4 New York City’s Ring of Steel
 1.5 Instructor Notes
 1.6 Ballard & Brown: Computer Vision Issues (Chapter 1)
 1.7 Shapiro & Stockman: Introduction (Chapter 1)
2 Pixel Operations and Histograms
 2.1 Reading and Content
 2.2 Assignments
 2.3 Instructor Notes
3 Morphology
 3.1 Reading and Content
 3.2 Instructor Notes
 3.3 Shapiro & Stockman: Binary Image Analysis (Chapter 3)
4 Spatial Filtering
 4.1 Reading and Content
 4.2 Assignments
 4.3 Instructor Notes
 4.4 Gonzales et al.: Spatial Filtering (Chapter 3.4)
5 Color
 5.1 Reading and Content
 5.2 Assignments
 5.3 Instructor Notes
 5.4 Gonzalez & Woods: Color Image Processing (Chapter 6)
6 Segmentation
 6.1 Reading and Content
 6.2 Assignments
 6.3 Instructor Notes
 6.4 Gonzalez Matlab: Color Segmentation (Chapter 6)
 6.5 Shapiro & Stockman: Image Segmentation (Chapter 10)
 6.6 Shapiro & Stockman: Region Properties (Chapter 3.6)
 6.7 Gonzalez Matlab: Color Segmentation (regionprops)
7 Fitting
 7.1 Reading and Content
 7.2 Instructor Notes
8 Image Formation and Imaging Systems
 8.1 Reading and Content
 8.2 Instructor Notes
9 Transformations, Camera Models, Homographies
 9.1 Reading and Content
 9.2 Assignments
 9.3 Instructor Notes
 9.4 Hartley & Zisserman: Multiple View Geometry (Chapter 1)
 9.5 Hartley & Zisserman: Estimation (Chapter 4, various)
10 Classification and Learning
 10.1 Reading and Content
 10.2 Instructor Notes
11 Texture
 11.1 Reading and Content
 11.2 Assignments
 11.3 Instructor Notes
12 Capstone
 12.1 Reading and Content
 12.2 Instructor Notes
 12.3 Nayar: Computational Cameras
A Matlab
B OpenCV
C Project 1: Background Modeling
D Project 2: Chroma-Keying in OpenCV
E Project 3: Camera Location Estimation, Homographies
F Project 4: Object Classification in Underwater Video