Important: This course is for students enrolled in a nanodegree. If you are not currently enrolled in a nanodegree, please access the full version of the Data Wrangling with Mongo DB course.

Data Wrangling with MongoDB

Need help getting started?


Course Resources

Datasets used in this course

You can download the full datasets used in this course. The ones used in the course UI are usually trimmed versions of these and may contain some differences in values:

Downloadable Materials

You can download Supplemental Materials, Lesson Videos and Transcripts from Downloadables (bottom right corner of the Classroom) or from the Dashboard (first option on the navigation bar on the left hand side).

Installing additional Modules in Python

  • Requests
  • xlutils (xlrd/xlwd)
  • BeautifulSoup
  • PyMongo
  • or pip install requests xlutils beautifulsoup4 pymongo
    (pip is a package management system used to install and manage software packages written in Python)

Installing MongoDB

Exercise and Example Code

You can download the code here as zip, or check out the repo on GitHub.

Additional MongoDB Materials

Course Syllabus

Lesson 1: Data Extraction Fundamentals

  • Assessing the Quality of Data
  • Intro to Tabular Formats
  • Parsing CSV
  • Parsing XLS with XLRD
  • Intro to JSON
  • Using Web APIs

Lesson 2: Data in More Complex Formats

  • Intro to XML
  • XML Design Principles
  • Parsing XML
  • Web Scraping
  • Parsing HTML

Lesson 3: Data Quality

  • What is Data Cleaning?
  • Sources of Dirty Data
  • Measuring Data Quality
  • A Blueprint for Cleaning
  • Auditing Validity
  • Auditing Accuracy
  • Auditing Completeness
  • Auditing Consistency
  • Auditing Uniformity

Lesson 4: Working with MongoDB

  • Data Modeling in MongoDB
  • Introduction to PyMongo
  • Field Queries
  • Projection Queries
  • Getting Data into MongoDB
  • Using mongoimport
  • Operators like $gt, $lt, $exists, $regex
  • Querying Arrays and using $in and $all Operators
  • Changing entries: $update, $set, $unset

Lesson 5: Analyzing Data

  • Examples of Aggregation Framework
  • The Aggregation Pipeline
  • Aggregation Operators: $match, $project, $unwind, $group
  • Multiple Stages Using a Given Operator

Lesson 6: Case Study - OpenStreetMap Data

  • Using iterative parsing for large data files
  • Open Street Map XML Overview
  • Exercises around OpenStreetMap data