Lesson 1
Self Driving Car Engineer
Nanodegree Program
Work on the future of autonomous vehicles and help make the self-driving car revolution a reality!
Work on the future of autonomous vehicles and help make the self-driving car revolution a reality!
Built in collaboration with
Waymo
Advanced
5 months
Real-world Projects
Completion Certificate
Last Updated November 22, 2023
Robot behavior planning • Object motion models • Robot motion planning • Eigen library
No experience required
Courses In This Program
Course 1 • 1 hour
Welcome to Self Driving Car Engineer Nanodegree
Welcome to the Self-Driving Car Engineer Nanodegree program! Learn about the Nanodegree experience, as well as hear from Waymo, one of Udacity's partners for the program.
Lesson 1
An Introduction to Your Nanodegree Program
Lesson 2
Getting Help
You are starting a challenging but rewarding journey! Take 5 minutes to read how to get help with projects and content.
Lesson 3
Meet Waymo
Hear from Waymo, one of the most cutting-edge autonomous vehicle companies out there! You'll learn about the company as well as about the Waymo Open Dataset, which you'll use in parts of the program.
Course 2 • 4 weeks
Computer Vision
In this course, you will develop critical Machine Learning skills that are commonly leveraged in autonomous vehicle engineering. You will learn about the life cycle of a Machine Learning project, from framing the problem and choosing metrics to training and improving models. This course will focus on the camera sensor and you will learn how to process raw digital images before feeding them into different algorithms, such as neural networks. You will build convolutional neural networks using TensorFlow and learn how to classify and detect objects in images. With this course, you will be exposed to the whole Machine Learning workflow and get a good understanding of the work of a Machine Learning Engineer and how it translates to the autonomous vehicle context.
Lesson 1
Introduction to Deep Learning for Computer Vision
Dive into Deep Learning for Computer Vision, learning about its use cases, history, and what you’ll build by the end of the course.
Lesson 2
The Machine Learning Workflow
Machine learning is more than just building a model - getting each step of the workflow right is crucial.
Lesson 3
Sensor and Camera Calibration
Learn how to calibrate your camera to remove distortions for improved perception.
Lesson 4
From Linear Regression to Feedforward Neural Networks
Build skills in linear and logistic regression before taking on feedforward neural networks, a type of deep learning.
Lesson 5
Image Classification with CNNs
Convolutional networks improve on feedforward networks for areas such as image classification - let’s get started building them!
Lesson 6
Object Detection in Images
Object detection builds on classification by finding multiple important objects within a single image - find out how!
Lesson 7 • Project
Object Detection in an Urban Environment
Use the Waymo dataset to detect objects in an urban environment.
Course 3 • 4 weeks
Sensor fusion
Besides cameras, self-driving cars rely on other sensors with complementary measurement principles to improve robustness and reliability, using sensor fusion. You will learn about the lidar sensor, different lidar types, and relevant criteria for sensor selection. Also, you will learn how to detect objects in a 3D lidar point cloud using a deep-learning approach, and then evaluate detection performance using a set of metrics. In the second half of the course, you will learn how to fuse camera and lidar detections and track objects over time with an Extended Kalman Filter. You will get hands-on experience with multi-target tracking, where you will initialize, update and delete tracks, assign measurements to tracks with data association techniques, and manage several tracks simultaneously.
Lesson 1
Introduction to Sensor Fusion and Perception
Get started with sensor fusion and perception, why they are important, and the history of their development in self-driving cars.
Lesson 2
The Lidar Sensor
Learn about the lidar sensor, capable of capturing important 3D data in point clouds.
Lesson 3
Detecting Objects in Lidar
Detect objects from the 3D data coming in from a lidar sensor.
Lesson 4 • Project
Mid-Term Project: 3D Object Detection
Use the Waymo dataset to detect 3D objects in the surrounding environment.
Lesson 5
Kalman Filters
Learn from the best! Sebastian Thrun will walk you through the usage and concepts of a Kalman Filter using Python.
Lesson 6
Extended Kalman Filters
Build an Extended Kalman Filter that's capable of handling data from multiple sources.
Lesson 7
Multi-Target Tracking
Get your tracking skills ready for the real world by learning how to track multiple targets simultaneously.
Lesson 8 • Project
Final Project: Sensor Fusion and Object Tracking
Use the Waymo dataset, along with sensor fusion, to track multiple 3D objects in the surrounding environment.
Course 4 • 4 weeks
Localization
In this course, you will learn all about robotic localization, from one-dimensional motion models up to using three-dimensional point cloud maps obtained from lidar sensors. You’ll begin by learning about the bicycle motion model, an approach to use simple motion to estimate location at the next time step, before gathering sensor data. Then, you’ll move onto using Markov localization in order to do 1D object tracking. From there, you will learn how to implement two scan matching algorithms, Iterative Closest Point (ICP) and Normal Distributions Transform (NDP), which work with 2D and 3D data. Finally, you will utilize these scan matching algorithms in the Point Cloud Library (PCL) to localize a simulated car with lidar sensing, using a 3D point cloud map obtained from the CARLA simulator.
Lesson 1
Introduction to Localization
Meet the team that will guide you through the localization lessons, and learn the intuition behind robotic localization!
Lesson 2
C++ Checkpoint
Are you ready to build Kalman Filters with C++? Take these quizzes to find out!
Lesson 3
Markov Localization
Learn the math behind localization, as well as how to implement Markov localization in C++.
Lesson 4
[Optional] Intro to PCL
Learn about the Point Cloud Library (PCL). Use a simulation highway environment to explore lidar sensing and generate point clouds.
Lesson 5
Creating Scan Matching Algorithms
Learn about and build two scan matching algorithms for localization: Iterative Closest Point (ICP) and Normal Distributions Transforms (NDT).
Lesson 6
Utilizing Scan Matching
Learn how to align point clouds with ICP and NDT before leveraging them to localize a self-driving car in a simulated environment!
Lesson 7 • Project
Scan Matching Localization
Localize a self-driving car within a point cloud from the CARLA simulator with the localization algorithms you learned in previous lessons - how accurate is your algorithm?
Taught By The Best
David Silver
Staff Software Engineer at Kodiak Robotics
David works on Self-driving trucks at Kodiak. As a Staff Software Engineer, David writes planning, control, simulation and mapping software for self-driving trucks.
Thomas Hossler
Sr Deep Learning Engineer
Thomas is originally a geophysicist but his passion for Computer Vision led him to become a Deep Learning engineer at various startups. By creating online courses, he is hoping to make education more accessible. When he is not coding, Thomas can be found in the mountains skiing or climbing.
Antje Muntzinger
Self-Driving Car Engineer
Antje Muntzinger is a technical lead for sensor fusion at Mercedes-Benz. She wrote her PhD about sensor fusion for advanced driver assistance systems and holds a diploma in mathematics. By educating more self-driving car engineers, she hopes to realize the dream of fully autonomous driving together in the future.
Andreas Haja
Instructor
Andreas Haja is an engineer, educator, and autonomous vehicle enthusiast. Andreas now works as an engineering professor in Germany. Previously, he developed computer vision algorithms and autonomous vehicle prototypes using C++.
Aaron Brown
Senior Software Engineer
Aaron has over 7 years in the autonomous vehicle field, starting as a Udacity content developer then instructor, focusing on Lidar in the Sensor Fusion and Self-Driving Car Nanodegree Program. For the last 5 years, he's been with Mercedes-Benz R&D North America, working in functional testing, Sensor Fusion, and Machine Learning driven by a passion for technological evolution and problem-solving.
Munir Jojo Verge
Lead Autonomous & AI Systems Developer at MITRE
Before MITRE, Munir was a Motion Planning & Decision-Making Manager at Amazon. He also worked for a 2 Self-driving car companies and for WaltDisney Shanghai building TronLightcycle. Munir holds a B.Eng. in Aerospace, a M.S. in Physics, and a M.S. in Space Studies.
Mathilde Badoual
Fifth year PhD student at UC Berkeley
Mathilde has a strong background in optimization and control, including reinforcement learning and has an engineering diploma from the electrical engineering school Supelec, in France. Previously she worked at Tesla in the energy and optimization team.
Ratings & Reviews
Average Rating: 4.2 Stars
34 Reviews
The Udacity Difference
Combine technology training for employees with industry experts, mentors, and projects, for critical thinking that pushes innovation. Our proven upskilling system goes after success—relentlessly.
Demonstrate proficiency with practical projects
Projects are based on real-world scenarios and challenges, allowing you to apply the skills you learn to practical situations, while giving you real hands-on experience.
Gain proven experience
Retain knowledge longer
Apply new skills immediately
Top-tier services to ensure learner success
Reviewers provide timely and constructive feedback on your project submissions, highlighting areas of improvement and offering practical tips to enhance your work.
Get help from subject matter experts
Learn industry best practices
Gain valuable insights and improve your skills
Unlock access to Self Driving Car Engineer and the rest of our best-in-class catalog
-
Unlimited access to our top-rated courses
-
Real-world projects
-
Personalized project reviews
-
Program certificates
-
Proven career outcomes
Full Catalog Access
One subscription opens up this course and our entire catalog of projects and skills.
Month-To-Month
4 Months
Average time to complete a Nanodegree program
Your subscription also includes:
Your subscription also includes:
Get Started Today
Self Driving Car Engineer
Month-To-Month
- Unlimited access to our top-rated courses
- Real-world projects
- Personalized project reviews
- Program certificates
- Proven career outcomes
4 Months
- All the same great benefits in our month-to-month plan
- Most cost-effective way to acquire a new set of skills
Related Programs
Related Programs
About Self Driving Car Engineer
Udacity's Self-Driving Car Engineer Nanodegree program is an advanced self-driving car course designed for those aspiring to become leaders in autonomous vehicle technology. The program covers critical areas such as machine learning, computer vision, and sensor fusion, providing hands-on experience in building and testing self-driving car systems. Our expert instructors, drawn from leading companies in the autonomous vehicle industry, guide students through real-world projects. This offers a unique opportunity to apply learning in practical scenarios, preparing students for exciting careers in this innovative field.