About me

Contact me

My research

Courses

Humour

Fencing

Quote for the Day

Home

You've come to Doug Aberdeen's old pages. In 5 seconds you will taken to my new pages http://sml.nicta.com.au/~daa/

Title: Overview of Statistical Machine Learning

Course Director: Dr Doug Aberdeen (NICTA Post doctoral Fellow)

Formal Description (to appear on Graduate Course Award certificate): The course covers topics in basic probability and computational complexity, optimisation of parameterised functions, classification, perceptrons, non-linear regression, clustering, kernel methods, hidden Markov models, reinforcement learning and inductive logic programming.

Topics to be covered:

1. Overview of ML, basic prob and stats
2. Optimisation
3. Bayesian Methods
4. Clustering
5. Perceptron
6/7. Kernel Methods
8. Principal Components Analysis
9. Graphical Models
10. Reinforcement Learning
11. Inductive Logic Programming

Why you should do this course: The purpose is to give students a flavour of the history, concepts, techniques, and applications of statistical machine learning. The course will provide students with an understanding of how Statistical Machine Learning can aid their own research. The course could also provide extra background for students pursuing research in a field of SML. The emphasis is on learning why the algorithms work and how to apply them, not how to analyse their performance.

Schedule: Monday, Tuesday, 10:00 -- 12:00. Thursday 15:00-17:00. Friday tutes (TBA). For four weeks.

Presenters: Dr Doug Aberdeen (NICTA Post-doctoral fellow in Statistical Machine Learning). Guest presenters will include Alex Smola (SML Programme Leader)

Contact person: Dr Doug Aberdeen, doug.aberdeen@nicta.com.au 02 6125 8647

Dates:
Dates: 11 October- 4 November 2004.
Registration Date: By 27 September 2004 to Belinda Orth (belinda.orth -a-t- nicta.com.au ).
Assignments due: 12 November 2004.
Notification date: 1 December 2004.


Total number of course hours: lectures- 24 hours; 8 hours tutorials; assignments- 5 hours (at least 3 to be completed). Students are expected to complete the 60 hours with background reading and problem solving.

Assessment procedures:Students must submit at least 3 out of 4 offered assignments. Two assignments will be theory. The other two will be programming based. A pass mark must be gained on 3 of the 4 assignments offered. Only a pass or fail mark will be awarded.

Assumed background: Mathematical prerequisites will include a familiarity with linear algebra, and some elementary probability theory. The course should be understandable by anyone with some 2nd year university maths. Confidence in programming in some not too obscure computer language.

Examiners: Dr Doug Aberdeen and Dr Alex Smola.

Proposed fees for the course: There will be a $40 administration fee. For RSISE students, this will be paid by RSISE. There is no fee for those auditing the course.

Detailed Syllabus
=================

1. Overview of ML, basic prob and stats
	30 mins:
	  - Inductive vs. Deductive
	  - Representation and Optimisation
	  - Complexity Classes P, NP NP-C NP-Hard PSpace
		
	90 mins:
	  - Basic probability theory
	    - Frequentists vs Bayesians     
	    - Bayes rule
 	    - Common distributions Gaussian, Geometric...
	  - Linear algebra
            - Eigenstuff
	    - Defn of +semi-def
	    - Factorisations (SVD)
                     	    
2. Optimisation
		
	  - Random (Evolutionary Algorithms)
	  - Stoch grad
	  - EM
	
	  - Quad programing
          - Lagrange, Duality

3. Bayesian Methods
	  - Bayes learning, naive bayes (Mitchell)

	  - MAP/MLE (Duda-Hart-Stork)


4. Clustering - 
        - KNN (Jain-Flynn-Murty)

5. Perceptron - 
	- Error back propogation training 
	- Loss functions (LMS, Log loss, ...)
	- Back prop through time
	- RBFs (link to kernels)
	- Self-organising feature maps (link to clustering)
	
6/7. Kernel Methods - (Alex Smola)

	  - Linear programming	
	
	
8. PCA - 

9. Graphical Models -  
	- Bayseian Networks
	- HMMs	
	- Filtering/Prediction
	- Speech processing

10. RL - 
	- Dynamic Programming
	- Function Approximation
	- Simulation
	- Policy based methods
	- Tseauro's Backgammon

11. ILP - (Kee Siong?)
	- Association rule mining
	- Decision Trees 

12. Extra stuff depending on time/interest.
    Learning Theory
    Data mining
    Bio-informatics  



The views and opinions expressed on this web page are not necessarily those of NICTA or the Australian National University. Any HTML or image from this page may be copied and re-used freely but must not be sold.
Feedback:Doug.Aberdeen AT anu.edu.au