Generative AI Engineer

DATB-GAIEG : Generative AI Application Evaluation and Governance

Course Duration

4 Hours

Delivery

ILT/VILT

Overview

Overview Course Curriculum Training Schedule Exam & Certification FAQs

This is your introduction to evaluating and governing generative AI systems. First, you’ll explore the meaning behind and motivation for building evaluation and governance/security systems. Next, we’ll connect evaluation and governance systems to the Databricks Data Intelligence Platform. Third, we’ll teach you about a variety of evaluation techniques for specific components and types of applications. Finally, the course will conclude with an analysis of evaluating entire AI systems with respect to performance and cost.

What You'll Learn

Importance of Evaluating GenAI Applications
Securing and Governing GenAI Applications
GenAI Evaluation Techniques
End-to-end Application Evaluation

Who Should Attend

Data scientists, ML engineers and AI practitioners responsible for evaluating the performance, safety, reliability and quality of generative-AI applications.
Professionals implementing governance frameworks for LLMs, including evaluation metrics, bias detection, hallucination monitoring, safety checks, compliance workflows and human-in-the-loop review.
Individuals overseeing model behaviour, prompt-response quality, cost optimisation, drift detection and adherence to organisational or regulatory AI policies.
Practitioners with experience in Python, MLflow, Databricks tools or LLM development who want to advance their skills in responsible-AI practices and structured evaluation of generative-AI systems.
Teams moving generative-AI solutions into production and needing consistent, transparent, and governable evaluation processes that ensure trustworthy and aligned AI outputs.

Prerequisites

Familiarity with natural language processing concepts
Familiarity with prompt engineering/prompt engineering best practices
Familiarity with the Databricks Data Intelligence Platform
Familiarity with RAG (preparing data, building a RAG architecture, concepts like embedding, vectors, vector databases, etc.)
Experience with building LLM applications using multi-stage reasoning LLM chains and agents

Learning Journey

Coming Soon...

Module 1. Importance of Evaluating GenAI Applications

Why to Evaluate GenAI Applications
Exploring Licensing of Datasets
Prompts and Guardrails Basics
Implementing and Testing Guardrails for LLMs

Module 2. Securing and Governing GenAI Applications

AI System Security
Implementing AI Guardrails

Module 3. GenAI Evaluation Techniques

Evaluation Techniques
Benchmark Evaluation
LLM-as-a-Judge
Domain-Specific Evaluation

Module 4. End-to-end Application Evaluation

AI System Architecture
Custom Metrics
Offline vs. Online Evaluation

Frequently Asked Questions (FAQs)

None

Keep Exploring

Course Curriculum

Training Schedule

Exam & Certification

FAQs

Frequently Asked Questions

Improve yourself and your career by taking this course.

Enroll Now

More Courses By Databricks

Browse All Courses

IBM Analytics3_8g102g-ibm-security-guardium-data-protection-foundations-training

Data Engineer

Ready to Take Your Business from Great to Awesome?

Level-up by partnering with Trainocate. Get in touch today.

Name

Phone

I'm inquiring for

Myself

Company

Inquiry Details

By submitting this form, you consent to Trainocate processing your data to respond to your inquiry and provide you with relevant information about our training programs, including occasional emails with the latest news, exclusive events, and special offers.

You can unsubscribe from our marketing emails at any time. Our data handling practices are in accordance with our Privacy Policy.