Nanodegree Program

Become a Deep Reinforcement Learning Expert

Become a reinforcement learning expert

Learn the deep reinforcement learning skills that are powering amazing advances in AI. Then start applying these to applications like video games and robotics.

  • Estimated Time
    4 Months

    At 10-15 hrs/week

  • Enroll by
    May 28, 2019

    Get access to classroom immediately on enrollment

In collaboration with

What You Will Learn

Download Syllabus
Syllabus

Deep Reinforcement Learning

Learn cutting-edge deep reinforcement learning algorithms—from Deep Q-Networks (DQN) to Deep Deterministic Policy Gradients (DDPG). Apply these concepts to train agents to walk, drive, or perform other complex tasks, and build a robust portfolio of deep reinforcement learning projects.

Write your own implementations of many cutting-edge algorithms, including DQN, DDPG, and evolutionary methods.

Hide details

4 months to complete

Prerequisite Knowledge

This program requires experience with Python, probability, machine learning, and deep learning.See detailed requirements.

  • Foundations of Reinforcement Learning

    Master the fundamentals of reinforcement learning by writing your own implementations of many classical solution methods.

  • Value-Based Methods

    Apply deep learning architectures to reinforcement learning tasks. Train your own agent that navigates a virtual world from sensory data.

    Navigation
  • Policy-Based Methods

    Learn the theory behind evolutionary algorithms and policy-gradient methods. Design your own algorithm to train a simulated robotic arm to reach target locations.

    Continuous Control
  • Multi-Agent Reinforcement Learning

    Learn how to apply reinforcement learning methods to applications that involve multiple, interacting agents. These techniques are used in a variety of applications, such as the coordination of autonomous vehicles.

    Collaboration and Competition
Icon - Dark upwards trend arrow

Apple, Facebook, and Google are investing in deep reinforcement learning.

All our Nanodegree programs include:

Real-world projects from industry experts

With real world projects and immersive content built in partnership with top tier companies, you’ll master the tech skills companies want.

1-on-1 technical mentor

Get a knowledgeable mentor who guides your learning and is focused on answering your questions, motivating you and keeping you on track.

Personal career coach and career services

You’ll have access to career coaching sessions, interview prep advice, and resume and online professional profile reviews to help you grow in your career.

Flexible learning program

Get a custom learning plan tailored to fit your busy life. Along with easy monthly payments you can learn at your own pace and reach your personal goals.
Succeed with Personalised Services
We provide services customised for your needs at every step of your learning journey to ensure your success!
Experienced Project Reviewers
Individual 1-on-1 Mentorship
Personal Career Coach
Experienced Project Reviewers
Reviews By the numbers
2000+ project reviewers
1.8M projects reviewed
4.85/5 reviewer ratings
3 hour avg project review turnaround time
Reviewer Services
  • Personalized feedback
  • Unlimited submissions and feedback loops
  • Practical tips and industry best practices
  • Additional suggested resources to improve
Succeed with Personalised Services
We provide services customised for your needs at every step of your learning journey to ensure your success!
Project Reviewers
1-on-1 Mentors
Career Coaching
Experienced Project Reviewers
Reviews By the numbers
2000+ project reviewers
1.8M projects reviewed
4.85/5 reviewer ratings
3 hour avg project review turnaround time
Reviewer Services
  • Personalized feedback
  • Unlimited submissions and feedback loops
  • Practical tips and industry best practices
  • Additional suggested resources to improve

Learn with the best

Alexis Cook
Alexis Cook

Curriculum Lead

Alexis is an applied mathematician with a Masters in Computer Science from Brown University and a Masters in Applied Mathematics from the University of Michigan. She was formerly a National Science Foundation Graduate Research Fellow.

Arpan Chakraborty
Arpan Chakraborty

Instructor

Arpan is a computer scientist with a PhD from North Carolina State University. He teaches at Georgia Tech (within the Masters in Computer Science program), and is a coauthor of the book Practical Graph Mining with R.

Mat Leonard
Mat Leonard

Instructor

Mat is a former physicist, research neuroscientist, and data scientist. He did his PhD and Postdoctoral Fellowship at the University of California, Berkeley.

Luis Serrano
Luis Serrano

Instructor

Luis was formerly a Machine Learning Engineer at Google. He holds a PhD in mathematics from the University of Michigan, and a Postdoctoral Fellowship at the University of Quebec at Montreal.

Cezanne Camacho
Cezanne Camacho

Curriculum Lead

Cezanne is a machine learning educator with a Masters in Electrical Engineering from Stanford University. As a former researcher in genomics and biomedical imaging, she’s applied machine learning to medical diagnostic applications.

Dana Sheahan
Dana Sheahan

Content Developer

Dana is an electrical engineer with a Masters in Computer Science from Georgia Tech. Her work experience includes software development for embedded systems in the Automotive Group at Motorola, where she was awarded a patent for an onboard operating system.

Chhavi Yadav
Chhavi Yadav

Content Developer

Chhavi is a Computer Science graduate student at New York University, where she researches machine learning algorithms. She is also an electronics engineer and has worked on wireless systems.

Juan Delgado
Juan Delgado

Content Developer

Juan is a computational physicist with a Masters in Astronomy. He is finishing his PhD in Biophysics. He previously worked at NASA developing space instruments and writing software to analyze large amounts of scientific data using machine learning techniques.

Miguel Morales
Miguel Morales

Content Developer

Miguel is a software engineer at Lockheed Martin. He earned a Masters in Computer Science at Georgia Tech and is an Instructional Associate for the Reinforcement Learning and Decision Making course. He’s the author of Grokking Deep Reinforcement Learning.

Student Reviews

4.6

(30)

5 stars
22
73.3%
4 stars
5
16.7%
3 stars
2
6.7%
2 stars
1
3.3%
1 stars
0
0.0%
Markus B.

State-of-the-art program with extraordinary projects. Core of moder AI. Udacity opens new doors for people who wanr to change the world. Perfect

Junjie X.

还是可以学到很多干货的,但是要真的转行或者找工作,自己平时的努力和积累也是必不可少的

Farida H.

I loved how the research spirit was obvious in this nanodegree

Muh Alif A.

The materials are presented in a clear and elaborate way.

Robson M.

The program is very didactic, the lessons are detailed and I could learn very much with the projects. These are important adjectives because we are handling with a difficult subject.

The Udacity Impact

Numbers don't lie. See what difference it makes in career searches.*

84%
Better Jobs

Career-seeking and job-ready graduates found a new, better job within six months of graduation.

$24,000
Salary Increase

Average salary increase for graduates who found a new, better job within six months of graduation.

Program Details

    PROGRAM OVERVIEW - WHY SHOULD I TAKE THIS PROGRAM?
  • Why should I enroll?

    The demand for engineers with reinforcement learning and deep learning skills far exceeds the number of engineers with these skills. This program offers a unique opportunity for you to develop these in-demand skills. You’ll implement several deep reinforcement learning algorithms using a combination of Python and deep learning libraries that will serve as portfolio pieces to demonstrate the skills you’ve acquired. As interest and investment in this space continues to increase, you’ll be ideally positioned to emerge as a leader in this groundbreaking field.

  • What jobs will this program prepare me for?

    This program is designed to build on your existing skills in machine learning and deep learning. As such, it doesn't prepare you for a specific job, but instead expands your skills in the deep reinforcement learning domain. These skills can be applied to various applications such as gaming, robotics, recommendation systems, autonomous vehicles, financial trading, and more.

  • How do I know if this program is right for me?

    This program offers an ideal path into the world of deep reinforcement learning—a transformational technology that is reshaping our future, and driving amazing new innovations in Artificial Intelligence. If you're interested in applying AI to fields such as gaming, robotics, autonomous systems, and financial trading, this is the perfect way to get started.

    ENROLLMENT AND ADMISSION
  • Do I need to apply? What are the admission criteria?

    No. This Nanodegree program accepts all applicants regardless of experience and specific background.

  • What are the prerequisites for enrollment?

    We recommend that you complete a course in Deep Learning equivalent to the Deep Learning Nanodegree program prior to entering the program. You will need to be able to communicate fluently and professionally in written and spoken English.

    Additionally, you should have the following knowledge:

    • Intermediate Python programming knowledge, including:
      • Strings, numbers, and variables
      • Statements, operators, and expressions
      • Lists, tuples, and dictionaries
      • Conditions, loops
      • Generators & comprehensions
      • Procedures, objects, modules, and libraries
      • Troubleshooting and debugging
      • Research & documentation
      • Problem solving
      • Algorithms and data structures

    Basic shell scripting:

    • Run programs from a command line
    • Debug error messages and feedback
    • Set environment variables
    • Establish remote connections

    Basic statistical knowledge, including:

    • Populations, samples
    • Mean, median, mode
    • Standard error
    • Variation, standard deviations
    • Normal distribution

    Intermediate differential calculus and linear algebra, including:

    • Derivatives & Integrals
    • Series expansions
    • Matrix operations through eigenvectors and eigenvalues
  • If I do not meet the requirements to enroll, what should I do?
    TUITION AND TERM OF PROGRAM
  • How is this Nanodegree program structured?

    The Deep Reinforcement Learning Nanodegree program is comprised of content and curriculum to support three (3) projects. Once you subscribe to a Nanodegree program, you will have access to the content and services for the length of time specified by your subscription. We estimate that students can complete the program in four (4) months working 10 hours per week.

    Each project will be reviewed by the Udacity reviewer network. Feedback will be provided and if you do not pass the project, you will be asked to resubmit the project until it passes.

  • How long is this Nanodegree program?

    Access to this Nanodegree program runs for the length of time specified in your subscription plan. See the Terms of Use and FAQ for other policies around the terms of access to our Nanodegree programs.

  • Can I get a refund?

    Please see the Udacity Nanodegree program FAQs found here for policies on enrollment in our programs.

    SOFTWARE AND HARDWARE - WHAT DO I NEED FOR THIS PROGRAM?
  • What software and versions will I need in this program?

    You will need a computer running a 64-bit operating system (most modern Windows, OS X, and Linux versions will work) with at least 8GB of RAM, along with administrator account permissions sufficient to install programs including Anaconda with Python 3.6 and supporting packages. Your network should allow secure connections to remote hosts (like SSH). We will provide you with instructions to install the required software packages.

Deep Reinforcement Learning

Become a reinforcement learning expert