About this Course

This course will cover the design and analysis of A/B tests, also known as split tests, which are online experiments used to test potential improvements to a website or mobile application. Two versions of the website are shown to different users - usually the existing website and a potential change. Then, the results are analyzed to determine whether the change is an improvement worth launching. This course will cover how to choose and characterize metrics to evaluate your experiments, how to design an experiment with enough statistical power, how to analyze the results and draw valid conclusions, and how to ensure that the the participants of your experiments are adequately protected.

Course Cost
Free
Timeline
Approx. 1 months
Skill Level
Intermediate
Included in Course
  • Icon course 01 3edf6b45629a2e8f1b490e1fb1516899e98b3b30db721466e83b1a1c16e237b1 Rich Learning Content

  • Icon course 04 2edd94a12ef9e5f0ebe04f6c9f6ae2c89e5efba5fd0b703c60f65837f8b54430 Interactive Quizzes

  • Icon course 02 2d90171a3a467a7d4613c7c615f15093d7402c66f2cf9a5ab4bcf11a4958aa33 Taught by Industry Pros

  • Icon course 05 237542f88ede3178ac4845d4bebf431ddd36d9c3c35aedfbd92e148c1c7361c6 Self-Paced Learning

  • Icon course 03 142f0532acf4fa030d680f5cb3babed8007e9ac853d0a3bf731fa30a7869db3a Student Support Community

Join the Path to Greatness

This free course is your first step towards a new career with the Data Analyst Nanodegree Program.

Free Course

A/B Testing

by Google

Enhance your skill set and boost your hirability through innovative, independent learning.

Icon steps 54aa753742d05d598baf005f2bb1b5bb6339a7d544b84089a1eee6acd5a8543d

Course Leads

  • Carrie Grimes
    Carrie Grimes

    Instructor

  • Caroline Buckey
    Caroline Buckey

    Instructor

  • Diane Tang
    Diane Tang

    Instructor

What You Will Learn

Lesson 1

Overview of A/B Testing

  • This lesson will cover what A/B testing is and what it can be used for.
  • How to construct a binomial confidence interval for the results.
  • How to decide whether the change is worth the launch cost.
Lesson 1

Overview of A/B Testing

  • This lesson will cover what A/B testing is and what it can be used for.
  • How to construct a binomial confidence interval for the results.
  • How to decide whether the change is worth the launch cost.
Lesson 2

Policy and Ethics for Experiments

  • How to make sure the participants of your experiments are adequately protected.
  • What questions you should be asking regarding the ethicality of experiments.
  • The four main ethics principles to consider when designing experiments.
Lesson 2

Policy and Ethics for Experiments

  • How to make sure the participants of your experiments are adequately protected.
  • What questions you should be asking regarding the ethicality of experiments.
  • The four main ethics principles to consider when designing experiments.
Lesson 3

Choosing and Characterizing Metrics

  • Learn techniques for brainstorming metrics.
  • What to do when you can't measure directly.
  • Characteristics to consider when validating metrics.
Lesson 3

Choosing and Characterizing Metrics

  • Learn techniques for brainstorming metrics.
  • What to do when you can't measure directly.
  • Characteristics to consider when validating metrics.
Lesson 4

Designing an Experiment

  • How to choose which users will be in your experiment and control group.
  • When to limit your experiment to a subset of your entire user base.
  • Design decisions affect the size of your experiment.
Lesson 4

Designing an Experiment

  • How to choose which users will be in your experiment and control group.
  • When to limit your experiment to a subset of your entire user base.
  • Design decisions affect the size of your experiment.
Lesson 5

Analyzing Results

  • How to analyze the results of your experiments.
  • Run sanity checks to catch problems with the experiment set-up.
  • Check conclusions with multiple methods, including a binomial sign test.
Lesson 5

Analyzing Results

  • How to analyze the results of your experiments.
  • Run sanity checks to catch problems with the experiment set-up.
  • Check conclusions with multiple methods, including a binomial sign test.

Prerequisites and Requirements

This course requires introductory knowledge of descriptive and inferential statistics. If you haven't learned these topics, or need a refresher, they are covered in the Udacity courses Inferential Statistics and Descriptive Statistics.

Prior experience with A/B testing is not required, and neither is programming knowledge.

See the Technology Requirements for using Udacity.

Why Take This Course

A/B testing, or split testing, is used by companies like Google, Microsoft, Amazon, Ebay/Paypal, Netflix, and numerous others to decide which changes are worth launching. By using A/B tests to make decisions, you can base your decisions on actual data, rather than relying on intuition or HiPPO's - the highest paid person's opinion! Designing good A/B tests and drawing valid conclusions can be difficult. You can almost never measure exactly what you want to know (such as whether the users are "happier" on one version of the site), so you need to find good proxies. You need sanity checks to make sure your experimental set-up isn't flawed, and you need to use a variety of statistical techniques to make sure the results you're seeing aren't due to chance. This course will walk you through the entire process. At the end, you will be ready to help businesses small or large make crucial decisions that could significantly affect their future!

What do I get?
  • Instructor videos
  • Learn by doing exercises
  • Taught by industry professionals
Icon globe e82eae5d45465aba4fbe4bb746905ce55dc3324f310b79c60e4a20089057d347

Udacity 现已提供中文版本! A Udacity tem uma página em português para você! There's a local version of Udacity for you!

前往优达学城中文网站 Ir para a página brasileira Go to Indian Site or continue to Global Site