Udacity part of Accenture logo

Agent Evaluation and Observability with Claude

Make AI agents you can trust by measuring what they do and watching how they behave in production. You'll enforce structured outputs with schemas so an agent's responses stay predictable and machine-readable. Then you'll build automated evaluation frameworks that score an agent's task completion, quality, and reliability. You'll apply these skills to real document-extraction systems: a validated, routed insurance policy pipeline and a resilient mortgage extraction system that holds up under messy input. You'll also fuse disagreeing evidence sources into a single supply-chain risk assessment. A final project brings evaluation and observability together.

  • Course
  • Intermediate
  • 7 hours
  • Updated: Nov 11, 2026

Subscription · Monthly

  • Cancel Anytime
  • Unlimited access to hundreds of top-rated courses
  • Hands-on projects with expert feedback
  • Personalized career coaching and interview prep
  • Program Certificates

Skills you'll learn

3 skills

  • AI Agent Observability
  • Implementing AI Agent Evaluation
  • Claude Agent SDK

Prerequisites

4 prerequisites

Prior to enrolling, you should have the following knowledge:

  • Claude API
  • Basic Python
  • Basic typescript
  • Claude Code Platform

You will also need to be able to communicate fluently and professionally in written and spoken English.

Course Outline

  • 8 lessons
  • 1 project

Program Instructors

4 instructors

Unlike typical professors, our instructors come from Fortune 500 and Global 2000 companies and have demonstrated leadership and expertise in their professions:

Abdellah Iraamane

Senior Software Engineer at Microsoft

Valerie Scarlata

AI Curriculum Lead at Udacity

Sufian Kaki Aslam

Founder & Agentic AI Architect, SKATE

Henrique Santana

Principal Machine Learning Engineer at Dell Technologies

Abdellah Iraamane

Senior Software Engineer at Microsoft

Valerie Scarlata

AI Curriculum Lead at Udacity

Sufian Kaki Aslam

Founder & Agentic AI Architect, SKATE

Henrique Santana

Principal Machine Learning Engineer at Dell Technologies

About this program

Learn AI evaluation, structured outputs, observability, and reliability testing for production AI agents.

Subscription · Monthly

  • Cancel Anytime
  • Unlimited access to hundreds of top-rated courses
  • Hands-on projects with expert feedback
  • Personalized career coaching and interview prep
  • Program Certificates

Other programs you might like:

Company

Udacity Schools

AI Master’s

Udacity Accenture logo
© 2011-2026 Udacity, Inc. "Nanodegree" is a registered trademark of Udacity. © 2011-2026 Udacity, Inc.
We use cookies and other data collection technologies to provide the best experience for our customers.