Intermediate

Approx. 2 months

Built by
Join thousands of students

Start Free Course

Start Free Course
Free
You get
Instructor videos
Learn by doing exercises and view project instructions

Course Summary

In this course, we will explore how to wrangle data from diverse sources and shape it to enable data-driven applications. Some data scientists spend the bulk of their time doing this!

Students will learn how to gather and extract data from widely used data formats. They will learn how to assess the quality of data and explore best practices for data cleaning. We will also introduce students to MongoDB, covering the essentials of storing data and the MongoDB query language together with exploratory analysis using the MongoDB aggregation framework.

This is a great course for those interested in entry-level data science positions as well as current business/data analysts looking to add big data to their repertoire, and managers working with data professionals or looking to leverage big data.

This course is also a part of our Data Analyst Nanodegree.

Why Take This Course?

At the end of the class, students should be able to:

  • Programmatically extract data stored in common formats such as csv, Microsoft Excel, JSON, XML and scrape web sites to parse data from HTML.
  • Audit data for quality (validity, accuracy, completeness, consistency, and uniformity) and critically assess options for cleaning data in different contexts.
  • Store, retrieve, and analyze data using MongoDB.


This course concludes with a final project where students incorporate what they have learned to address a real-world data analysis problem.

Prerequisites and Requirements

The ideal student should have the following skills:

  • Programming experience in Python or a willingness to read a little documentation to understand examples and exercises throughout the course.
  • The ability to perform rudimentary system administration on Windows or Unix

At least some experience using a unix shell or Windows PowerShell will be helpful, but is not required.

No prior experience with databases is needed.

About MongoDB
This course is developed in conjunction with MongoDB, Inc., the originator and primary contributor to the open source database MongoDB. MongoDB is the leading NoSQL database. Designed for how we build and run applications today, MongoDB empowers organizations to be more agile and scalable. It enables new types of applications, better customer experience, faster time to market and lower costs.

See the Technology Requirements for using Udacity.

What Will I Learn?

Projects

P3: Wrangle OpenStreetMap Data

Choose any area of the world in https://www.openstreetmap.org and use data munging techniques, such as assessing the quality of the data for validity, accuracy, completeness, consistency and uniformity, to clean the OpenStreetMap data for a part of the world that you care about.

Syllabus

Lesson 1: Data Extraction Fundamentals

  • Assessing the Quality of Data
  • Intro to Tabular Formats
  • Parsing CSV
  • Parsing XLS with XLRD
  • Intro to JSON
  • Using Web APIs

Lesson 2: Data in More Complex Formats

  • Intro to XML
  • XML Design Principles
  • Parsing XML
  • Web Scraping
  • Parsing HTML

Lesson 3: Data Quality

  • What is Data Cleaning?
  • Sources of Dirty Data
  • Measuring Data Quality
  • A Blueprint for Cleaning
  • Auditing Validity
  • Auditing Accuracy
  • Auditing Completeness
  • Auditing Consistency
  • Auditing Uniformity

Lesson 4: Working with MongoDB

  • Data Modelling in MongoDB
  • Introduction to PyMongo
  • Field Queries
  • Projection Queries
  • Getting Data into MongoDB
  • Using mongoimport
  • Operators like $gt, $lt, $exists, $regex
  • Querying Arrays and using $in and $all Operators
  • Changing entries: $update, $set, $unset

Lesson 5: Analyzing Data

  • Examples of Aggregation Framework
  • The Aggregation Pipeline
  • Aggregation Operators: $match, $project, $unwind, $group
  • Multiple Stages Using a Given Operator

Lesson 6: Case Study - OpenStreetMap Data

  • Using iterative parsing for large datafiles
  • Open Street Map XML Overview
  • Exercises around OpenStreetMap data
  • Final Project Instructions

Instructors & Partners

instructor photo

Shannon Bradshaw

Instructor

Shannon is Director of Education at MongoDB, managing MongoDB University's in-person training and free online courses. Prior to joining MongoDB, Shannon was an Associate Professor of Computer Science at Drew University with research interests in user experience, information science, and the semantic web. For the past several years, Shannon has divided his time between academia and industry. He has trained software engineers at Goldman Sachs, designed text-retrieval systems at Morgan Stanley, and built many trading and analytics applications at boutique firms in the financial industry.

instructor photo

Gundega Dekena

Instructor

Once upon a time Gundega was a Udacity student. In a way she still is, because she is learning new things from instructors she works with and her Udacity coworkers every day.

If you occasionally want to read fun news about robotics, science and games, follow her on G+ - https://plus.google.com/+GundegaDekena.

track icon track icon

View more courses in
Software Engineering — Data Science