Generative AI Data Management

Get oriented to GenAI data management: course overview, learning goals, prerequisites, Azure setup, tooling, workspaces, and essential skills for the capstone project.

Learn how generative AI systems manage data across the entire lifecycle—collection to inference—requiring continuous governance, user data ownership, and bias monitoring beyond training.

Explore enterprise RAG: its layered data architecture, chunking strategies, retrieval design, embeddings, and the governance of prompts and conversational memory.

Learn to refactor an enterprise RAG pipeline for privacy, separating user and enterprise data, supporting per-session storage, and enabling right-to-erasure on user data.

Learn to build pipelines that process and align text and images, ensuring accurate retrieval from multimodal documents using modality-specific preprocessing and structured records.

Build a multimodal data pipeline to extract, describe, and index every image from a PDF, enabling fine-grained product retrieval in an apparel catalog RAG chatbot.

Learn why and how to version data, code, and pipelines in generative AI to enable traceability, reproducibility, auditability, and responsible AI governance.

Learn to version data, prompts, and artifacts in a RAG pipeline using DVC and Git, enabling full lineage tracing, reproducible rollbacks, and defensible audit trails in generative systems.

Explore essential access control for AI systems: authentication, authorisation, pre-retrieval filtering, identity propagation, auditability, and enforcing security throughout data pipelines.

Learn to enforce access management in AI data systems by configuring role-based policies, controlling data access pre-retrieval, and maintaining auditable query logs.

Learn how data catalogs and metadata ensure generative AI uses approved, reliable, and governed data, preventing unfiltered or unauthorized information from influencing AI outputs.

Automate data catalog classification to control, filter, and trace RAG chatbot sources by metadata, enabling category-based exports and defensible, audit-ready retrieval.

Learn what synthetic data is, when to use it, key risks like bias amplification, quality validation steps, and best practices for governance and provenance in generative AI workflows.

Build a synthetic Q&A pipeline that redacts PII before LLM use, ensuring privacy and auditability while generating clean, reviewable datasets for fine-tuning.