Machine Learning: Reinforcement Learning

Thank you for signing up for the course! We look forward to working with you and hearing your feedback in our forums.

Need help getting started?


Course Resources

Reading Materials

Suggested Text

  1. Tom Mitchell, Machine Learning. McGraw-Hill, 1997.
    You can buy the international edition for $24-$40 from various websites.

Optional Text

  1. Larry Wasserman, All of Statistics. Springer, 2010.
  2. Richard Sutton and Andrew Barto, Reinforcement Learning: An introduction. MIT Press, 1998.
  3. Trevor Hastie, Robert Tibshirani and Jerome Friedman, The Elements of Statistical Learning. Springer, 2009.

Reading List

Coding Resources


  1. WEKA Machine learning software in JAVA that you can use for your projects
  2. Data Mining with Weka A MOOC Course
  3. ABAGAIL Machine learning software in JAVA. This is hosted on my github, so you can contribute too
  4. scikit-learn A popular python library for supervised and unsupervised learning algorithms
  5. MATLAB NN Toolbox The toolbox supports supervised learning with feedforward, radial basis, and dynamic networks and unsupervised learning with self-organizing maps and competitive layers.
  6. Murphy's MDP Toolbox for Matlab
  7. MATLAB Clustering Package By Frank Dellaert
  8. ICA Example


  1. UCI Machine Learning Repository An online repository of data sets that can be used for machine learning experiments.
  2. Stanford Large Network Dataset Dataset of large social and information networks.
  3. Vision Benchmark Suite Autonomous car dataset
  4. Other datasets

Applications of Machine Learning

Downloadable Materials

You can download Supplemental Materials, Lesson Videos and Transcripts from Downloadables (bottom right corner of the Classroom) or from the Dashboard (first option on the navigation bar on the left hand side).

Course Syllabus

Lesson 1: Markov Decision Processes

  • Decision Making and Reinforcement Learning
  • Markov Decision Processes
  • Sequences of Rewards
  • Assumptions
  • Policies
  • Finding Policies

Lesson 2: Reinforcement Learning

  • Rat Dinosaurs
  • API
  • Three Approaches to RL
  • A New Kind of Value Function
  • Estimating Q from Transitions
  • Q Learning Convergence
  • Greedy Expoloration

Lesson 3: Game Theory

  • What is Game Theory
  • Minimax
  • Fundamental Result
  • Game Tree
  • Von Neumann
  • Center Game
  • Snitch
  • A Beautiful Equilibrium
  • The Two Step
  • 2Step2Furious

Lesson 4: Game Theory Continued

  • The Sequencing
  • Iterated Prisioner’s Dilemna
  • Uncertain End
  • Tit for Tat
  • Finite State Strategy
  • Folk Theorem
  • Security Level Profile
  • Grim Trigger
  • Implausible Threats
  • Pavlov
  • Computational Folk Theorem
  • Stochastic Games and Multiagent RL
  • Zero Sum Stochastic Games
  • General Sum Games


Final Project

Here you can find details on the final project in Reinforcement Learning: build a system that learns how to play and win at Pacman!