ADaSci Academy

    Mastering LLM Observability with Arize Phoenix

    This course addresses the critical need for LLM observability using Arize Phoenix, an open-source platform built for monitoring and diagnostics. Participants will learn to identify issues like bias, hallucination, and dr…

    2,388+ enrolled
    4 hours
    Intermediate
    Mastering LLM Observability with Arize Phoenix
    01 — Overview

    About this course

    This course addresses the critical need for LLM observability using Arize Phoenix, an open-source platform built for monitoring and diagnostics. Participants will learn to identify issues like bias, hallucination, and drift, giving them mastery over the tool, empowering them to build trustworthy, cost-effective AI systems.

    02 — Outcomes

    What you'll learn

    ✓
    Understand the critical need for observability in large language model (LLM) applications and the challenges associated with unmonitored LLM behavior.
    ✓
    Gain hands-on experience using Arize Phoenix for real-time monitoring, traceability, and evaluation of LLM outputs.
    ✓
    Learn to identify and measure key performance indicators (KPIs) like coherence, hallucination rate, and drift in LLM-based systems.
    ✓
    Master the ingestion and structuring of LLM trace data (spans, traces, and events) to facilitate transparent and meaningful analysis.
    ✓
    Explore advanced LLMOps workflows and observability techniques using Arize Phoenix’s architecture, integrations, and MCP Server.
    03 — Curriculum

    Course content

    5 sections · 18 lectures

    LLM Observability and its Importance
    0
    Identifying KPIs relevant to LLMs
    49m
    Key Performance Indicators for Measuring LLM Performance
    0
    Deep Dive into Arize Phoenix Ecosystem
    38m
    Arize Phoenix and Its Utility in LLM Observability
    0
    Implementation of Tracing using Arize Phoenix
    0
    Structuring Data for Effective Analysis within Phoenix
    0
    Exploring Integrations and LLM Frameworks
    13m
    Observability Integration with Phoenix, OpenAI, LangChain, and LlamaIndex
    0
    Vector Database Integration in Arize Phoenix
    9m
    RAG Pipeline with Phoenix Observability + Weaviate + OpenAI
    0
    Semantic Search and Similarity Analysis for LLMs
    0
    Drift Detection and Monitoring in LLMs
    0
    Performance Metrics and Evaluation for LLMs
    0
    RAG Evaluation using Arize Phoenix
    15m
    Summarization Quality Comparison using Arize Phoenix
    0
    Implementation of Phoenix MCP Server with Claude Desktop & Cursor
    9m
    Course Assessment
    0
    04 — Prerequisites

    Requirements

    • This course addresses the critical need for LLM observability using Arize Phoenix, an open-source platform built for monitoring and diagnostics. Participants will learn to identify issues like bias, hallucination, and drift, giving them mastery over the tool, empowering them to build trustworthy, cost-effective AI systems.
    05 — Audience

    Who should take this course

    • Beginners to Intermediate

    Save 50% with Membership

    Get unlimited access to all courses and exclusive member discounts on certifications.

    50% off all courses30% off certificationsExclusive content
    Mastering LLM Observability with Arize Phoenix

    $49.99

    Member Pricing

    FREE with Premium

    50% off with General

    Duration4h
    LevelIntermediate
    CertificateYes

    $49.99