Machine Learning: Unsupervised Learning

Thank you for signing up for the course! We look forward to working with you and hearing your feedback in our forums.

Need help getting started?


Course Resources

Reading Materials

Suggested Text

  1. Tom Mitchell, Machine Learning. McGraw-Hill, 1997.
    You can buy the international edition for $24-$40 from various websites.

Optional Text

  1. Larry Wasserman, All of Statistics. Springer, 2010.
  2. Richard Sutton and Andrew Barto, Reinforcement Learning: An introduction. MIT Press, 1998.
  3. Trevor Hastie, Robert Tibshirani and Jerome Friedman, The Elements of Statistical Learning. Springer, 2009.

Reading List

Coding Resources


  1. WEKA Machine learning software in JAVA that you can use for your projects
  2. Data Mining with Weka A MOOC Course
  3. ABAGAIL Machine learning software in JAVA. This is hosted on my github, so you can contribute too
  4. scikit-learn A popular python library for supervised and unsupervised learning algorithms
  5. MATLAB NN Toolbox The toolbox supports supervised learning with feedforward, radial basis, and dynamic networks and unsupervised learning with self-organizing maps and competitive layers.
  6. Murphy's MDP Toolbox for Matlab
  7. MATLAB Clustering Package By Frank Dellaert
  8. ICA Example


  1. UCI Machine Learning Repository An online repository of data sets that can be used for machine learning experiments.
  2. Stanford Large Network Dataset Dataset of large social and information networks.
  3. Vision Benchmark Suite Autonomous car dataset
  4. Other datasets

Applications of Machine Learning

Downloadable Materials

You can download Supplemental Materials, Lesson Videos and Transcripts from Downloadables (bottom right corner of the Classroom) or from the Dashboard (first option on the navigation bar on the left hand side).

Course Syllabus

Lesson 1: Randomized Optimization

  • Optimization, randomized
  • Hill climbing
  • Random restart hill climbing
  • Simulated annealing
  • Annealing algorithm
  • Properties of simulated annealing
  • Genetic algorithms
  • GA skeleton
  • Crossover example
  • What have we learned
  • MIMIC: A probability model
  • MIMIC: Pseudo code
  • MIMIC: Estimating distributions
  • Finding dependency trees
  • Probability distribution

Lesson 2: Clustering

  • Clustering and expectation maximization
  • Basic clustering problem
  • Single linkage clustering (SLC)
  • Running time of SLC
  • Issues with SLC
  • K-means clustering
  • K-means in Euclidean space
  • K-means as optimization
  • Soft clustering
  • Maximum likelihood Gaussian
  • Expectation Maximization (EM)
  • Impossibility theorem

Lesson 3: Feature Selection

  • Algorithms
  • Filtering and Wrapping
  • Speed
  • Searching
  • Relevance
  • Relevance vs. Usefulness

Lesson 4: Feature Transformation

  • Feature Transformation
  • Words like Tesla
  • Principal Components Analysis
  • Independent Components Analysis
  • Cocktail Party Problem
  • Matrix
  • Alternatives

Lesson 5: Info Theory

  • History
  • Sending a Message
  • Expected size of the message
  • Information between two variables
  • Mutual information
  • Two Independent Coins
  • Two Dependent Coins
  • Kullback Leibler Divergence

Final Project

Here you can find a description and details on the final project in Unsupervised Learning: design a system that recommends new movies for you based on your past ratings!