Large Language Models (LLMs) & Text Generation

Course

Dive deeper into how computers understand and create language, and learn how to build a custom chatbot using unsupervised machine learning, prompt engineering, and retrieval augmented generation. We'll start with a high-level overview of the types of LLMs, the differences between them, and how best to account for their strengths and weaknesses. Then we'll get into the internal details, including natural language processing (NLP) techniques like tokenization, as well as modern transformer architectures and attention mechanisms. Finally, we'll build a practical LLM application that combines an LLM with a custom dataset.

Intermediate

2 weeks

Real-world Projects

Completion Certificate

Last Updated July 12, 2024

Skills you'll learn:

Together AI API • Search implementation in Python • NLP transformers • GPT

Prerequisites:

Generative AI Fluency • Intermediate Python • Deep learning

Course Lessons

Lesson 1

Introduction to LLMs

This lesson covers the types of LLMs, an intuitive understanding of their limitations and capabilities, inference and decoding hyperparameters, and strategies for effective prompt engineering.

Lesson 2

NLP Fundamentals

This lesson covers the essential Natural Language Processing topics needed to use the latest LLM technology. You will learn the basics of NLP and then dive into text encoding and text generation.

Lesson 3

Transformers and Attention Mechanism

In this lesson, you will open up the black box of transformer architectures and learn about the attention mechanisms and other components that make these powerful models possible.

Lesson 4

Retrieval Augmented Generation

In this lesson, we will learn how to create a custom Q&A bot powered by OpenAI! Along the way, you'll learn how OpenAI works and how to leverage its powerful language processing capabilities.

Lesson 5

Build Custom Datasets for LLMs

In this lesson, you will learn how to construct a relevant, quality dataset for fine-tuning large language models and performing retrieval augmented generation.

Lesson 6 • Project

Project: Build Your Own Custom Chatbot

For this project, you will use everything you learned in this course to create a custom chatbot using a dataset of your choice.

Taught By The Best

Emily McMilin

Research Scientist

Emily McMilin is a Senior Research Scientist and Independent Researcher working at the intersection of NLP and Causal Inference. She obtained her Ph.D. in Electrical Engineering from Stanford University and prior to that an M.Sc. from University of Victoria, and a B.Sc. from Stanford in Symbolic Systems.

Victor Geislinger

Machine Learning Engineer

Victor Geislinger is a machine learning engineer and is dedicated to sharing his knowledge with others. Victor recently joined Google as a software engineer focused on AI/ML but has been programming and educating others for over a decade since studying physics and math at the University of California, Santa Cruz.

Jason Lin

Chief Scientist, Reasonly AI

Jason has developed deep learning algorithms and AI applications at Lyft self-driving, Spotify and Google DeepMind. Formerly a Stanford Online and UN keynote speaker, he's earned a M.S. in Machine Learning from Georgia Tech and coauthored NLP and computer vision papers with MIT.

Erick Galinkin

Principal AI Researcher

Erick Galinkin is a hacker and computer scientist, leading research at the intersection of security and artificial intelligence at Rapid7. He has spoken at numerous industry and academic conferences on topics ranging from malware development to game theory in security.

The Udacity Difference

Combine technology training for employees with industry experts, mentors, and projects, for critical thinking that pushes innovation. Our proven upskilling system goes after success—relentlessly.

Demonstrate proficiency with practical projects

Projects are based on real-world scenarios and challenges, allowing you to apply the skills you learn to practical situations, while giving you real hands-on experience.

Gain proven experience
Retain knowledge longer
Apply new skills immediately

Top-tier services to ensure learner success

Reviewers provide timely and constructive feedback on your project submissions, highlighting areas of improvement and offering practical tips to enhance your work.