New!
Nanodegree Program

Become a Sensor Fusion Engineer

Learn to fuse lidar point clouds, radar signatures, and camera images using Kalman Filters to perceive the environment and detect and track vehicles and pedestrians over time.

  • Estimated Time
    4 Months

    At 10 hours/week

  • Enroll by
    August 27, 2019

    Get access to classroom immediately on enrollment

  • Prerequisites
    Intermediate C++, Calculus, and Probability

    See prerequisites in detail

Built in Collaboration With

What You Will Learn

Download Syllabus
Syllabus

Sensor Fusion Engineer

Learn to detect obstacles in lidar point clouds through clustering and segmentation, apply thresholds and filters to radar data in order to accurately track objects, and augment your perception by projecting camera images into three dimensions and fusing these projections with other sensor data. Combine this sensor data with Kalman filters to perceive the world around a vehicle and track objects over time.

Learn to fuse data from three of the primary sensors that robots use: lidar, camera, and radar.

Hide details

Estimated 4 Months

Prerequisite Knowledge

You should have intermediate C++ knowledge, and be familiar with calculus, probability, and linear algebra. See detailed requirements.

  • Lidar

    Process raw lidar data with filtering, segmentation, and clustering to detect other vehicles on the road.

    Lidar Obstacle Detection
  • Radar

    Analyze radar signatures to detect and track objects. Calculate velocity and orientation by correcting for radial velocity distortions, noise, and occlusions.

    Radar Obstacle Detection
  • Cameras

    Fuse camera images together with lidar point cloud data. You'll extract object features, classify objects, and project the camera image into three dimensions to fuse with lidar data.

    Camera and Lidar Fusion
  • Kalman Filters

    Fuse data from multiple sources using Kalman filters, and build extended and unscented Kalman filters for tracking nonlinear movement.

    Unscented Kalman Filters

All Our Nanodegree Programs Include:

Real-world projects from industry experts

With real world projects and immersive content built in partnership with top tier companies, you’ll master the tech skills companies want.

1-on-1 technical mentor

Get a knowledgeable mentor who guides your learning and is focused on answering your questions, motivating you and keeping you on track.

Personal career coach and career services

You’ll have access to career coaching sessions, interview prep advice, and resume and online professional profile reviews to help you grow in your career.

Flexible learning program

Get a custom learning plan tailored to fit your busy life. Along with easy monthly payments you can learn at your own pace and reach your personal goals.
Program OfferingsFull list of offerings included:
Subscription includes:
Class Content
Content Co-created with Mercedes-Benz
icon-checkmarkCheckmark
Real-world projects
icon-checkmarkCheckmark
Project reviews
icon-checkmarkCheckmark
Proj. feedback from experienced reviewers
icon-checkmarkCheckmark
Student Services
1-on-1 technical mentor
New
icon-checkmarkCheckmark
Check-ins with mentor
New
icon-checkmarkCheckmark
Student community
Improved
icon-checkmarkCheckmark
Career services
Personal career coaching
New
icon-checkmarkCheckmark
Interview preparations
icon-checkmarkCheckmark
Resume services
icon-checkmarkCheckmark
Github review
icon-checkmarkCheckmark
LinkedIn profile review
icon-checkmarkCheckmark
Resume shared with 500+ employers
New
icon-checkmarkCheckmark
Succeed with Personalized Services
We provide services customized for your needs at every step of your learning journey to ensure your success!
Experienced Project Reviewers
Individual 1-on-1 Mentorship
Personal Career Coach
Experienced Project Reviewers
Reviews By the numbers
2000+ project reviewers
1.8M projects reviewed
4.85/5 reviewer ratings
3 hour avg project review turnaround time
Reviewer Services
  • Personalized feedback
  • Unlimited submissions and feedback loops
  • Practical tips and industry best practices
  • Additional suggested resources to improve
Succeed with Personalized Services
We provide services customized for your needs at every step of your learning journey to ensure your success!
Project Reviewers
1-on-1 Mentors
Career Coaching
Experienced Project Reviewers
Reviews By the numbers
2000+ project reviewers
1.8M projects reviewed
4.85/5 reviewer ratings
3 hour avg project review turnaround time
Reviewer Services
  • Personalized feedback
  • Unlimited submissions and feedback loops
  • Practical tips and industry best practices
  • Additional suggested resources to improve

Learn With The Best

David Silver
David Silver

Product Lead

David Silver leads the School of Autonomous Systems at Udacity. Before Udacity, David was a research engineer on the autonomous vehicle team at Ford. He has an MBA from Stanford, and a BSE in Computer Science from Princeton.

Stephen Welch
Stephen Welch

Instructor

Stephen is a Content Developer at Udacity and has worked on the C++ and Self-Driving Car Engineer Nanodegree programs. He started teaching and coding while completing a Ph.D. in mathematics, and has been passionate about engineering education ever since.

Andreas Haja
Andreas Haja

Instructor

Andreas Haja is an engineer, educator and autonomous vehicle enthusiast with a PhD in computer science. Andreas now works as a professor, where he focuses on project-based learning in engineering. During his career with Volkswagen and Bosch he developed camera technology and autonomous vehicle prototypes.

Abdullah Zaidi
Abdullah Zaidi

Instructor

Abdullah holds his M.S from the University of Maryland and is an expert in the field of Radio Frequency Design and Digital Signal processing. After spending several years at Qualcomm, Abdullah joined Metawave, where he now leads Radar development for autonomous driving.

Aaron Brown
Aaron Brown

Instructor

Aaron Brown has a background in electrical engineering, robotics and deep learning. Aaron has worked as a Content Developer and Simulation Engineer at Udacity focusing on developing projects for self-driving cars.

Get started with

Become a Sensor Fusion Engineer

Icon-Pie-Chart
Learn
Combine and filter lidar, radar, and camera data to detect and track vehicles and pedestrians.
Icon - Clock -Blue
Average Time
On average, successful students take 4 months to complete this Nanodegree program.
Icon - Project - Blue
Benefits include
  • Real-world projects from industry experts
  • 1-on-1 technical mentor
  • Personal career coach & career services

Best Deal!

4 months access
$1436 $399 $359 per month

Pay up-front and save! Switch to the monthly price afterwards if more time is needed.

Buy with payments as low as $120 per month at 0% APR.

Learn More
Pay as you go
$399per month

Life can be unpredictable. Get maximum flexibility to learn at your own pace.

Buy with payments as low as $120 per month at 0% APR.

Learn More
The Udacity Impact

Numbers don't lie. See what difference it makes in career searches.*

84%
Better Jobs

Career-seeking and job-ready graduates found a new, better job within six months of graduation.

$24,000
Salary Increase

Average salary increase for graduates who found a new, better job within six months of graduation.

Program Details

    Program Overview - Why should I take this program?
  • Why should I enroll?

    Sensor fusion engineering is one of the most important and exciting areas of robotics.

    Sensors like cameras, radar, and lidar help self-driving cars, drones, and all types of robots perceive their environment. Analyzing and fusing this data is fundamental to building an autonomous system.

    In this Nanodegree program, you will work with camera images, radar signatures, and lidar point clouds to detect and track vehicles and pedestrians. By graduation, you will have an impressive portfolio of projects to demonstrate your skills to employers.

  • What jobs will this program prepare me for?

    As a Sensor Fusion Engineer, you'll be equipped to bring value to a wide array of industries and be eligible for many roles.

    Your opportunities might include roles such as an:

    • Imaging Engineer
    • Sensor Fusion Engineer
    • Perception Engineer
    • Automated Vehicle Engineer
    • Research Engineer
    • Self-Driving Car Engineer
    • Object Tracking Engineer
    • Sensor Engineer
    • System Integration Engineer
    • Depth Engineer
  • How do I know this program is right for me?

    If you’re interested in learning about lidar, radar, and camera data and how to fuse it together, this program is right for you.

    Sensors and sensor data are used in a wide array of applications -- from cell phones to robots and self-driving cars -- giving you a wide array of fields you could enter or advance a career in after this program.

    Enrollment and Admission
  • Do I need to apply? What are the admission criteria?

    There is no application. This Nanodegree program accepts everyone, regardless of experience and specific background.

  • What are the prerequisites for enrollment?

    To optimize your chances of success in the Sensor Fusion Engineer Nanodegree program, we’ve created a list of prerequisites and recommendations to help prepare you for the program curriculum. You should have the following knowledge:

    • Advanced knowledge in any object-oriented - programming language, preferably C++
    • Intermediate Probability
    • Intermediate Calculus
    • Intermediate Linear Algebra
    • Basic Linux Command Lines
  • If I don't meet the requirements to enroll, what should I do?

    For aspiring sensor fusion engineers who currently have a limited background in programming or math, we've created the Intro to Self-Driving Cars Nanodegree program to help you prepare. This program teaches C++, linear algebra, calculus, and statistics.

    If you have a limited background in programming, we’ve created the C++ Nanodegree program to help you prepare for the coding in this program.

    Tuition and Terms of Program
  • How is this program structured?

    The Sensor Fusion Engineer Nanodegree program is comprised of content and curriculum to support four (4) projects. Once you subscribe to a Nanodegree program, you will have access to the content and services for the length of time specified by your subscription. We estimate that students can complete the program in four (4) months working 10 hours per week.

    Each project will be reviewed by the Udacity reviewer network. Feedback will be provided and if you do not pass the project, you will be asked to resubmit the project until it passes.

  • How long is this Nanodegree program?

    Access to this Nanodegree program runs for the length of time specified in your subscription plan. See the Terms of Use and FAQ for other policies around the terms of access to our Nanodegree programs.

  • Can I get a refund?

    Please see the Udacity Nanodegree program FAQs found here for policies on enrollment in our programs.

    Hardware and Software
  • What software and versions will I need for this program?

    We have few requirements since you will be coding on our virtual environments (“Workspaces”) in the browser. This means you can complete all coursework within our platform, and do not need to install anything on your own machine.

    If you choose to complete projects on your local machine, you should install:

    • C++ Version 11
    • Point Cloud Library 1.7

    Hardware Requirements:

    • None

Get Started

Start your path towards a Sensor Fusion career today