Udacity part of Accenture logo
Log InJoin for Free

Sensor Fusion

Nanodegree Program

The Sensor Fusion Engineer Nanodegree program consists of four courses that teach the fundamentals of sensor fusion and perception for self-driving cars. The program covers lidar, radar, camera, and Kalman filters, and includes lessons on working with real-world data, filtering, segmentation, clustering, and object tracking. In addition, students will work on a capstone project that involves building a complete sensor fusion pipeline for autonomous vehicles. Upon completing the program, graduates will have the skills and knowledge necessary to design and implement sensor fusion systems for self-driving cars.

The Sensor Fusion Engineer Nanodegree program consists of four courses that teach the fundamentals of sensor fusion and perception for self-driving cars. The program covers lidar, radar, camera, and Kalman filters, and includes lessons on working with real-world data, filtering, segmentation, clustering, and object tracking. In addition, students will work on a capstone project that involves building a complete sensor fusion pipeline for autonomous vehicles. Upon completing the program, graduates will have the skills and knowledge necessary to design and implement sensor fusion systems for self-driving cars.

Built in collaboration with

Mercedes-Benz

Advanced

3 months

Real-world Projects

Completion Certificate

Last Updated March 21, 2023

Skills you'll learn:

Radar • Sensor fusion • Digital cameras • Kalman filters

Prerequisites:

Basic calculus • Basic probability • Intermediate C++

Courses In This Program

Course 1 1 hour

Welcome

Lesson 1

Welcome to the Sensor Fusion Engineer Nanodegree Program

Welcome to the Sensor Fusion Engineer Nanodegree Program! In this lesson, you will learn more about the structure of the program and meet the team.

Lesson 2

Introduction to Sensor Fusion

Lesson 3

Get Help with Your Account

What to do if you have questions about your account or general questions about the program.

Lesson 4

Getting Help

You are starting a challenging but rewarding journey! Take 5 minutes to read how to get help with projects and content.

Course 2 3 weeks

Lidar Obstacle Detection

Lesson 1

Introduction to Lidar and Point Clouds

Learn about lidar and point clouds. Use a simulation highway environment to explore lidar sensing and generate point clouds.

Lesson 2

Point Cloud Segmentation

In this lesson, you will be using Ransac with a plane model to segment point cloud data and separate it into points that are part of the road and points that are not.

Lesson 3

Clustering Obstacles

Perform Euclidean clustering, and learn how to build KD-Trees to use them to do efficient nearest neighbor search for clustering.

Lesson 4

Working with Real PCD

Take what you have learned in the previous lessons and apply it to real pcd being played back in a video.

Lesson 5 • Project

Lidar Obstacle Detection Project

In this lesson, students will submit the project that they have developed over the previous lessons.

Course 3 3 weeks

Camera

Lesson 1

Introduction

Introduction to the instructor, and the guest Timo Rehfeld, Mercedes-Benz R&D North America, and the course overview.

Lesson 2

Autonomous Vehicles and Computer Vision

Learn about the various levels of autonomy, some typical sensor sets, basics of camera technology, and an introduction into the OpenCV computer vision library.

Lesson 3

Engineering a Collision Detection System

Learn the collision detection basics, and estimating the TTC with Lidar and Camera.

Lesson 4

Tracking Image Features

Learn about the intensity gradient and filtering techniques; extract corners, infer features of an image, and track an object across multiple images.

Lesson 5 • Project

Camera Based 2D Feature Tracking

Lesson 6

Combining Camera and Lidar

Learn to improve the tracking process results by combining the Camera and Lidar output

Lesson 7 • Project

Track an Object in 3D Space

Lesson 8

Final Thoughts from Timo

Course 4 2 weeks

Radar

Lesson 1

Introduction

Lesson 2

Radar Principles

Review Radar functionality, FMCW waveform, Radar Hardware, Schematic and the Radar Equation

Lesson 3

Range-Doppler Estimation

Estimate the range and velocity of the target using the FMCW radar

Lesson 4

Clutter, CFAR, AoA, and Clustering

Discuss - Clutter formation and then its removal using CFAR technique. After that

Lesson 5 • Project

Radar Target Generation and Detection

Taught By The Best

Photo of David Silver

David Silver

Staff Software Engineer at Kodiak Robotics

David works on Self-driving trucks at Kodiak. As a Staff Software Engineer, David writes planning, control, simulation and mapping software for self-driving trucks.

Photo of Stephen Welch

Stephen Welch

Instructor

Stephen is a Content Developer at Udacity and has worked on the C++ and Self-Driving Car Engineer Nanodegree programs. He started teaching and coding while completing a Ph.D. in mathematics, and has been passionate about engineering education ever since.

Photo of Abdullah Zaidi

Abdullah Zaidi

Instructor

Abdullah holds his M.S from the University of Maryland and is an expert in the field of Radio Frequency Design and Digital Signal processing. After spending several years at Qualcomm, Abdullah joined Metawave, where he now leads Radar development for autonomous driving.

Photo of Andreas Haja

Andreas Haja

Instructor

Andreas Haja is an engineer, educator, and autonomous vehicle enthusiast. Andreas now works as an engineering professor in Germany. Previously, he developed computer vision algorithms and autonomous vehicle prototypes using C++.

Photo of Aaron Brown

Aaron Brown

Senior Software Engineer

Aaron has over 7 years in the autonomous vehicle field, starting as a Udacity content developer then instructor, focusing on Lidar in the Sensor Fusion and Self-Driving Car Nanodegree Program. For the last 5 years, he's been with Mercedes-Benz R&D North America, working in functional testing, Sensor Fusion, and Machine Learning driven by a passion for technological evolution and problem-solving.

Ratings & Reviews

Average Rating: 4.8 Stars

175 Reviews

Nikhil N.

March 26, 2023

The program is well thought out, the content is very close to the real world. The projects and examples provide a good balance of hands-on learning and theory.

Alejandro A.

November 22, 2022

Really Cool! The project was exciting! Doing it from scratch.

Andrei M.

October 14, 2022

The only in depth course on LiDAR, and it's in C++, so no more playing. Udacity has done a great job teaming up with Mercedes for this one.

Adan Yusseff D.

April 10, 2022

Very well structured, the tutorials are very good to learn. It's being a long time since i haven't code on C++ and this program guides you really well when you have some gaps in the knowledge like me. It is a hard program, but with a lot of interesting information from real life.

Rajnish B.

April 7, 2022

It's going great. I absolutely love the project and quizzes.

Page 1 of 35

The Udacity Difference

Combine technology training for employees with industry experts, mentors, and projects, for critical thinking that pushes innovation. Our proven upskilling system goes after success—relentlessly.

Demonstrate proficiency with practical projects

Projects are based on real-world scenarios and challenges, allowing you to apply the skills you learn to practical situations, while giving you real hands-on experience.

  • Gain proven experience

  • Retain knowledge longer

  • Apply new skills immediately

Top-tier services to ensure learner success

Reviewers provide timely and constructive feedback on your project submissions, highlighting areas of improvement and offering practical tips to enhance your work.

  • Get help from subject matter experts

  • Learn industry best practices

  • Gain valuable insights and improve your skills

Unlock access to Sensor Fusion and the rest of our best-in-class catalog

  • Unlimited access to our top-rated courses

  • Real-world projects

  • Personalized project reviews

  • Program certificates

  • Proven career outcomes

Full Catalog Access

One subscription opens up this course and our entire catalog of projects and skills.

Month-To-Month

4 Months

*

Average time to complete a Nanodegree program

*Discount applies to the first 4 months of membership, after which plans are converted to month-to-month.

Your subscription also includes:

(729)

3 months

, Intermediate

Get Started Today

Sensor Fusion

Month-To-Month


  • Unlimited access to our top-rated courses
  • Real-world projects
  • Personalized project reviews
  • Program certificates
  • Proven career outcomes

4 Months

Average time to complete a Nanodegree program

  • All the same great benefits in our month-to-month plan
  • Most cost-effective way to acquire a new set of skills
Discount applies to the first 4 months of membership, after which plans are converted to month-to-month.

Related Programs

(729)

3 months

, Intermediate

Udacity Accenture logo

Company

  • Facebook
  • Twitter
  • LinkedIn
  • Instagram

© 2011-2024 Udacity, Inc. "Nanodegree" is a registered trademark of Udacity. © 2011-2024 Udacity, Inc.
We use cookies and other data collection technologies to provide the best experience for our customers.