AI Red Teaming and Adversarial Machine Learning

Get oriented: what offensive AI security covers, the seven attack families you will run yourself, and the dual-system red team engagement you deliver as the capstone.

Learn why AI red teaming targets model behavior rather than code, walk the three-stage lifecycle, and leave able to scope an engagement and write a charter developers can act on.

Use an LLM to surface attack vectors from real system documentation, then turn the ones evidence supports into a red team charter with scope, rules of engagement, and success criteria.

See how attackers fool an image classifier with changes too small to notice, with and without model access, and why a high score on clean test data proves nothing about robustness.

Build black-box and white-box attacks against a real classifier with ART, measure both with adversarial accuracy and perturbation size, and turn those numbers into a risk assessment.

Learn how flipping labels or planting a trigger corrupts a model at training time, and why a backdoored model passes every accuracy check you would normally run before shipping.

Train a clean baseline, poison it with label flips and a backdoor trigger, measure what each attack costs the model, and find which defender-side check actually catches which attack.

Learn why one sentence of user text can override an application's rules: how a model reads system versus user prompts, and the three techniques attackers reach for once filters go up.

Build a prompt injection payload suite, run it across models and system prompt configurations, score each compromise, and test whether a prompt-level defense moves the number at all.

Follow how RAG turns documents into embeddings and ranks them by meaning, then how an attacker hijacks that ranking to control what the model tells every user who asks a matching question.

Stand up a RAG pipeline with live embeddings, plant poisoned documents tuned to real questions, measure the ranking shift they cause, and test whether provenance filtering stops them.

Learn how a model's own confidence scores let an attacker rebuild the data it trained on, why overfitting makes it worse, and how much detail an inference API should ever return.

Reconstruct a face from a classifier's confidence outputs alone, then switch sides and measure how far leakage drops across full probabilities, rounded scores, and a bare label.

Treat robustness as a number you can benchmark the way you benchmark accuracy: the metrics that measure it, the frameworks that automate it, and how to gate a release on it.

Build an automated attack suite that scores a model under clean, environmental, and adversarial conditions, then compare two variants and judge which one clears a release gate.

Map the stack you inherit — model weights, frameworks, libraries, base images, cloud — and see how a flaw four layers deep in a package you never named runs with your credentials.

Scan AI container images with Trivy, parse the report and SBOM into structured findings, and build a priority score ranked by real exposure, where a MEDIUM can rightly outrank a HIGH.