Data Engineer

DATB-PO : Databricks Performance Optimization

Course Duration

4 Hours

Delivery

ILT/VILT

Overview

Overview Course Curriculum Training Schedule Exam & Certification FAQs

In this course, you’ll learn how to optimize workloads and physical layout with Spark and Delta Lake and and analyze the Spark UI to assess performance and debug applications. We’ll cover topics like streaming, liquid clustering, data skipping, caching, photons, and more.

Note: This course is part of the 'Advanced Data Engineering with Databricks' course series.

What You'll Learn

Spark Architecture
Designing the Foundation File Explosion
Code Optimization
Fine-Tuning - Choosing the Right Cluster

Who Should Attend

Prerequisites

The content was developed for participants with these skills/knowledge/abilities:

Ability to perform basic code development tasks using Databricks (create clusters, run code in notebooks, use basic notebook operations, import repos from git, etc.)
Intermediate programming experience with PySpark
Extract data from a variety of file formats and data sources
Apply a number of common transformations to clean data
Reshape and manipulate complex data using advanced built-in functions
Intermediate programming experience with Delta Lake (create tables, perform complete and incremental updates, compact files, restore previous versions, etc.)

Learning Journey

Coming Soon...

Spark Architecture

Spark UI Introduction

Designing the Foundation File Explosion

Introduction to Designing Foundation
File Explosion
Data Skipping and Liquid Clustering
Lab: Data Skipping and Liquid Clustering

Code Optimization

Skew
Shuffle
Spill
Lab: Exploding Join
Serialization
User-Defined Functions

Fine-Tuning - Choosing the Right Cluster

Fine-Tuning: Choosing the Right Cluster
Pick the Best Instance Types

Frequently Asked Questions (FAQs)

None

Keep Exploring

Course Curriculum

Training Schedule

Exam & Certification

FAQs

Frequently Asked Questions

Improve yourself and your career by taking this course.

Enroll Now

More Courses By Databricks

Browse All Courses

Data Engineer

Ready to Take Your Business from Great to Awesome?

Level-up by partnering with Trainocate. Get in touch today.

Name

Phone

I'm inquiring for

Myself

Company

Inquiry Details

By submitting this form, you consent to Trainocate processing your data to respond to your inquiry and provide you with relevant information about our training programs, including occasional emails with the latest news, exclusive events, and special offers.

You can unsubscribe from our marketing emails at any time. Our data handling practices are in accordance with our Privacy Policy.