Lesson 1
Welcome to the Sensor Fusion Engineer Nanodegree Program
Welcome to the Sensor Fusion Engineer Nanodegree Program! In this lesson, you will learn more about the structure of the program and meet the team.
Sensor Fusion
Nanodegree Program
The Sensor Fusion Engineer Nanodegree program consists of four courses that teach the fundamentals of sensor fusion and perception for self-driving cars. The program covers lidar, radar, camera, and Kalman filters, and includes lessons on working with real-world data, filtering, segmentation, clustering, and object tracking. In addition, students will work on a capstone project that involves building a complete sensor fusion pipeline for autonomous vehicles. Upon completing the program, graduates will have the skills and knowledge necessary to design and implement sensor fusion systems for self-driving cars.
The Sensor Fusion Engineer Nanodegree program consists of four courses that teach the fundamentals of sensor fusion and perception for self-driving cars. The program covers lidar, radar, camera, and Kalman filters, and includes lessons on working with real-world data, filtering, segmentation, clustering, and object tracking. In addition, students will work on a capstone project that involves building a complete sensor fusion pipeline for autonomous vehicles. Upon completing the program, graduates will have the skills and knowledge necessary to design and implement sensor fusion systems for self-driving cars.
Built in collaboration with
Mercedes-Benz
Advanced
3 months
Real-world Projects
Completion Certificate
Last Updated March 21, 2023
Radar • Sensor fusion • Digital cameras • Kalman filters
Basic calculus • Basic probability • Intermediate C++
Courses In This Program
Course 1 • 1 hour
Welcome
Lesson 1
Welcome to the Sensor Fusion Engineer Nanodegree Program
Welcome to the Sensor Fusion Engineer Nanodegree Program! In this lesson, you will learn more about the structure of the program and meet the team.
Lesson 2
Introduction to Sensor Fusion
Lesson 3
Get Help with Your Account
What to do if you have questions about your account or general questions about the program.
Lesson 4
Getting Help
You are starting a challenging but rewarding journey! Take 5 minutes to read how to get help with projects and content.
Course 2 • 3 weeks
Lidar Obstacle Detection
Lesson 1
Introduction to Lidar and Point Clouds
Learn about lidar and point clouds. Use a simulation highway environment to explore lidar sensing and generate point clouds.
Lesson 2
Point Cloud Segmentation
In this lesson, you will be using Ransac with a plane model to segment point cloud data and separate it into points that are part of the road and points that are not.
Lesson 3
Clustering Obstacles
Perform Euclidean clustering, and learn how to build KD-Trees to use them to do efficient nearest neighbor search for clustering.
Lesson 4
Working with Real PCD
Take what you have learned in the previous lessons and apply it to real pcd being played back in a video.
Lesson 5 • Project
Lidar Obstacle Detection Project
In this lesson, students will submit the project that they have developed over the previous lessons.
Course 3 • 3 weeks
Camera
Lesson 1
Introduction
Introduction to the instructor, and the guest Timo Rehfeld, Mercedes-Benz R&D North America, and the course overview.
Lesson 2
Autonomous Vehicles and Computer Vision
Learn about the various levels of autonomy, some typical sensor sets, basics of camera technology, and an introduction into the OpenCV computer vision library.
Lesson 3
Engineering a Collision Detection System
Learn the collision detection basics, and estimating the TTC with Lidar and Camera.
Lesson 4
Tracking Image Features
Learn about the intensity gradient and filtering techniques; extract corners, infer features of an image, and track an object across multiple images.
Lesson 5 • Project
Camera Based 2D Feature Tracking
Lesson 6
Combining Camera and Lidar
Learn to improve the tracking process results by combining the Camera and Lidar output
Lesson 7 • Project
Track an Object in 3D Space
Lesson 8
Final Thoughts from Timo
Course 4 • 2 weeks
Radar
Lesson 1
Introduction
Lesson 2
Radar Principles
Review Radar functionality, FMCW waveform, Radar Hardware, Schematic and the Radar Equation
Lesson 3
Range-Doppler Estimation
Estimate the range and velocity of the target using the FMCW radar
Lesson 4
Clutter, CFAR, AoA, and Clustering
Discuss - Clutter formation and then its removal using CFAR technique. After that
Lesson 5 • Project
Radar Target Generation and Detection
Taught By The Best
David Silver
Staff Software Engineer at Kodiak Robotics
David works on Self-driving trucks at Kodiak. As a Staff Software Engineer, David writes planning, control, simulation and mapping software for self-driving trucks.
Stephen Welch
Instructor
Stephen is a Content Developer at Udacity and has worked on the C++ and Self-Driving Car Engineer Nanodegree programs. He started teaching and coding while completing a Ph.D. in mathematics, and has been passionate about engineering education ever since.
Abdullah Zaidi
Instructor
Abdullah holds his M.S from the University of Maryland and is an expert in the field of Radio Frequency Design and Digital Signal processing. After spending several years at Qualcomm, Abdullah joined Metawave, where he now leads Radar development for autonomous driving.
Andreas Haja
Instructor
Andreas Haja is an engineer, educator, and autonomous vehicle enthusiast. Andreas now works as an engineering professor in Germany. Previously, he developed computer vision algorithms and autonomous vehicle prototypes using C++.
Aaron Brown
Senior Software Engineer
Aaron has over 7 years in the autonomous vehicle field, starting as a Udacity content developer then instructor, focusing on Lidar in the Sensor Fusion and Self-Driving Car Nanodegree Program. For the last 5 years, he's been with Mercedes-Benz R&D North America, working in functional testing, Sensor Fusion, and Machine Learning driven by a passion for technological evolution and problem-solving.
Ratings & Reviews
Average Rating: 4.8 Stars
175 Reviews
The Udacity Difference
Combine technology training for employees with industry experts, mentors, and projects, for critical thinking that pushes innovation. Our proven upskilling system goes after success—relentlessly.
Demonstrate proficiency with practical projects
Projects are based on real-world scenarios and challenges, allowing you to apply the skills you learn to practical situations, while giving you real hands-on experience.
Gain proven experience
Retain knowledge longer
Apply new skills immediately
Top-tier services to ensure learner success
Reviewers provide timely and constructive feedback on your project submissions, highlighting areas of improvement and offering practical tips to enhance your work.
Get help from subject matter experts
Learn industry best practices
Gain valuable insights and improve your skills
Unlock access to Sensor Fusion and the rest of our best-in-class catalog
-
Unlimited access to our top-rated courses
-
Real-world projects
-
Personalized project reviews
-
Program certificates
-
Proven career outcomes
Full Catalog Access
One subscription opens up this course and our entire catalog of projects and skills.
Month-To-Month
4 Months
Average time to complete a Nanodegree program
Your subscription also includes:
Your subscription also includes:
Get Started Today
Sensor Fusion
Month-To-Month
- Unlimited access to our top-rated courses
- Real-world projects
- Personalized project reviews
- Program certificates
- Proven career outcomes
4 Months
- All the same great benefits in our month-to-month plan
- Most cost-effective way to acquire a new set of skills