Historical Perspective
- Biological motivations: the McCulloch and Pitts neuron, Hebbian learning.
- Statistical motivations
Theory
- Generalisation: What is learning from data?
- The power of machine learning methods: What is a learning algorithm? What can they do?
Probability
- Probability as representation of uncertainty in models and data
- Bayes Theorem and its applications
- Law of large numbers and the Multivariate Gaussian distribution
Optimisation
- Convexity
- 1-D minimisation
- Gradient methods in higher dimensions
- Constrained optimisation
Linear Algebra
- Using matrices to find solutions of linear equations
- Properties of matrices and vector spaces
- Eigenvalues, eigenvectors and singular value decomposition
Supervised Learning
- Regression Analysis
- Classification using Bayesian principles
- Perceptron Learning
- Support Vector Machines and introduction to Kernel methods
- Neural networks/multi-layer perceptrons (MLP)
- Features and discriminant analysis
Data handling and unsupervised learning
- Principal Components Analysis (PCA)
- K-Means clustering
- Spectral clustering
Regression and Model-fitting Techniques
- Linear regression
- Polynomial Fitting
- Kernel Based Networks
Case Studies
- Example applications: Speech, Vision, Natural Language, Bioinformatics.
