Dept. of Systems Design Engineering
Faculty of Engineering
University of Waterloo
Canada N2L 3G1
Tel: (519) 888-4567 x84970
FAX: (519) 746-4791
I'll be putting things here occasionally, although anything important will of course be announced in class.
Because of the SYDE and MTE symposia, this week's schedule is a bit changed:
Monday, March 23 - 1:30 Lecture on clustering Wednes, March 25 - 1:30 Repeat of Monday's lecture Wednes, March 25 - 2:30 Tutorial regarding lab 3 Friday, March 27 - 1:30 Lectures continue
Mar 18: Lab 3 has been posted.
Mar 9: Midterm solutions have been posted.
Feb 27: I have put a few more pages in the Handouts section.
Feb 26: Because the midterm was on Wednesday, the schedule for this week ended up being a little ambiguous (ie, was Wednesday a double-class with a tutorial on Friday, or not?). Since the default plan, schedule-wise, is that we have class on Friday, I *will* plan on being in class tomorrow.
A few things: No calculators or any other aids Two sides of an 8.5x11 sheet (or two sheets, each with one side written) of paper permitted with anything written/typed on them The exam will have 4-5 questions, and will have a similar balance to previous exams. Important Dates: Last Class Wed April 1, 1:30-3:30 Last Tutorial Fri April 3, 1:30-2:30 Lab 3 Due Fri April 3, Email to y30liu@engmail Office Hour Wed April 8, 1:00 - 2:30 P. Fieguth DC-2615 Office Hour Thu April 9, 1:00 - 2:30 Amir DC-2628 Office Hour Fri April 10, 1:00 - 2:30 Ying DC-2620 Final Exam Mon April 13, 9:00 - 11:30 E2-1303A/B
- Question 1 a-d: Amir Shabani - Question 1 e-i: Ying Liu - Question 2 a-b: Amir Shabani - Question 2 b-g: Prof. Fieguth
Wednesdays, 3:30 - 4:30 Tutorial TA E2-1303A (If tutorial is on Wednesday, the office hour is in-class, after tutorial, TA will leave once there are no students around) Fridays, 2:30 - 3:30 Tutorial TA E2-1303A (If tutorial is on Friday, the office hour is in-class, after tutorial, TA will leave once there are no students around) Mondays 12:30 - 1:20 Fieguth DC-2643To talk with me, the easiest arrangement is to ask questions after class, at 2:20 Mon, Wed, Fri. Since there is no class after ours, I will generally hang around and avoid leaving too quickly at the end of class. I will, of course, be scheduling additional office hours before the midterm and final exams.
Students who want some basic background problems in statistics
should consult Shanmugan & Breipohl (on reserve, TK5102.5.S447):
Suggested problems from course notes:
The tutorials are meant to cover areas of potential difficulty
to students, and to allow a more informal time of question and
answer. We will try to plan the tutorials quite carefully; in
particular, there will be two types of tutorials:
Date Style Content
-------------------------------- ----- -----------------------------------------
Week 1 - Friday, January 9 Teach Statistics & Algebra Review
Week 2 - Wednesday, January 14 Lab Matlab Overview & Information (see below)
Week 3 - Friday, January 23 Teach Classifier Overview, Nonparametric Methods
Week 4 - Friday, January 30 Teach Eigendecompositions, Cov. sketching, GED sketch
Week 5 - Wednesday, February 4 Lab Lab 1 discussion and help
Week 7 - Wednesday, February 18 --- (reading week)
Week 6 - Monday, February 23 Teach GED Sketching, MAP, Pr(Error), Midterm Review
Week 8 - Wednesday, February 25 --- (midterm)
Week 9 - Wednesday, March 4 Lab Lab 2
Week 10 - Friday, March 13 Teach Parameter Estimation, Nonparametric Estimation (Parz, kNN)
Week 11 - Friday, March 20 Teach Discriminants
Week 12 - Friday, March 27 Teach Lab 3 discussion and help
Week 13 - Friday, April 3 Lab Discrim / Clustering
Weekly Sample Problems
Some of the problems at the end of the chapters in the course notes
aren't so helpful. I have tried to update the notes to include more
realistic problems, however I have also started to prepare
new problems, which I'll be posting here on a weekly basis.
These problems should be roughly similar to those
which you might expect to see on an exam, and I usually
give some sort of partial solution.
Suggested Homework Problems
Although there are no "problem sets" in this course, the ability
to solve classification problems by hand is very important,
so all students should really practice on sample problems. Some
of the problems in the course notes are somewhat advanced and
probably beyond the level of most undergraduate students, however
there are a number of more basic problems which everyone should
be able to do, and are strongly recommended!
Examples 2 - 2, 3, 12
Problems 2 - 1, 9, 12, 18, 23, 29, 33, 42
There are two other excellent references for students needing
some help with statistics and related background. Appendices
1 and 2 in Schalkoff (on reserve, Q327.S27) and Appendix A
in the book by Duda, Hart, & Stork.
Chapter 2: 1 3 5 6 7
Chapter 3: 1 2 7 8ab 12 13 14
Chapter 4: 1abc 2 3 7abc 8abcd 10 (typo in 4.8: '3/2', not '2/3')
Chapter 5: 1 4
Chapter 6: 3 4
Chapter 7: 3 4
Chapter 8: 1 2abd 3 4
Old 372 midterms and finals are available from the
Alternatively, here are PDF versions of my
2001 final, and 1997 final
exams. The 1997 final is probably a bit on the hard side, however you may
find the questions useful for studying purposes.
Students who want some basic background problems in statistics should consult Shanmugan & Breipohl (on reserve, TK5102.5.S447):
Suggested problems from course notes:
|Chapter 2:||Questions 2.1 - 2.5||Question 2.6ab||Question 2.6c|
|Chapter 3:||Questions 3.1 - 3.4||Questions 3.5 - 3.6||Questions 3.7 - 3.8|
|Questions 3.9 - 3.10||Questions 3.11||Questions 3.12|
|Questions 3.13||Questions 3.14|
|Chapter 4:||Questions 4.1 - 4.2||Questions 4.3, 4.7||Question 4.4|
|Question 4.5||Question 4.6||Question 4.8|
|Question 4.9||Question 4.10|
|Chapter 5:||Question 5.1||Question 5.4||Question 5.4|
|Chapter 6:||Question 6.1, 6.2, 6.4||Question 6.4c (i)||Question 6.4c (ii)|
|Chapter 7:||Question 7.2a||Question 7.2b||Question 7.3|
|Chapter 8:||Question 8.1||Question 8.2||Question 8.3|
Duda, Hart & Stork (Q327.D83, On Reserve) Pattern classification and scene analysis
(There are two editions; the second is quite new, and is an excellent book. Any student seriously interested in pattern recognition could consider purchasing this book.)
Schalkoff (Q327.S27, On Reserve) Pattern recognition : statistical, structural, and neural approaches
(An excellent book; not as comprehensive as Duda & Hart, but a considerably more readable book.)
Topic SD372 Schalkoff Duda & Hart Duda, Hart & Stork Course Notes (First Ed.) (Second Ed.) Introduction Ch 1 Ch 1 Ch 1 Ch 1 Statistics Background Ch 2 Appen. 1, 2 Appen A Distance Classification Ch 3 Statistical Classification Ch 4 Ch 2 Ch 2 Ch 2 Parameter Estimation Ch 5.1 Ch 3 (p.58-70) Ch 3 Ch 3 NonParametric Estimation Ch 5.2 Ch 3 (p.70-75) Ch 4 Ch 4 Linear Discriminants Ch 6 Ch 4 Ch 5 Ch 5 Unlabeled Clustering Ch 7 Ch 5 Ch 6 Ch 6 Feature Selection Ch 8 Ch 6.14 Ch 1.3
A very simple demo page, with a simple Matlab script which calls a short function.
Lab 0 Tutorial for SD372
Second Matlab Tutorial for SD372
Lab 1 is due by Monday, February 9 at 5pm. Since that is the week before reading week, with midterms, I strongly suggest that students get started early. You already know enough to do nearly the entire lab; only MAP has not yet been discussed in class. In the past students have spent an awful lot of time trying to figure out how to generate the appropriate plots in Matlab, so I am providing you with the appropriate routine here:
Ellipse plotting routine for Lab #1.
Lab Data Sets:
[p,x,y] = parzen( data, res, win ) Input Parameters: data - a two-column matrix; each row [x y] defines one point res - determines the spatial step between PDF estimates This should be a vector of five values [res lowx lowy highx highy], giving the limits along the x and y axes for which the PDF should be estimated. For example, to estimate a PDF over -1 < x < 1 and 3 < y < 7, and the estimates should be spaced 0.01 units apart, then the vector should be [0.01 -1 3 1 7] win - The code says this is optional, but the default window in the code is not the one you need to use for lab 2, so that means this is NOT optional. You should define the window as a matrix. For example, a rectangular window would be the easiest one to do. To get a 10x10 rectangular window we would pass in ones(10,10). In your case you need a Gaussian window - this takes a bit more thought: how do you create a matrix with a Gaussian shape? How big should the matrix be? Returned Parameters: x - estimated locations along x-axis; this is just [lowx:res(1):highx] y - estimated locations along y-axis; this is just [lowy:res(1):highy] p - estimated 2D PDF, a matrix of values
The aggregation part of the lab will not be based directly on material seen in-class, however everything you need to know is specified in the lab handout.
Here is the ZIP file containing the images, Matlab code, and features which you need for lab 3. There are 13 files:
1. Ten images 2. Two Matlab .m files - one to read the images, the other to plot features 3. A Matlab .mat file which contains the feature valuesFor ease in visualization, so you don't have to plot the images yourself, the ten images are shown below:
There is also the combined image, for segmentation, which looks like
(Page last updated September 13, 2016, [an error occurred while processing this directive])