Alex Soto

Practical Change Data Streaming Use Cases With Debezium And Quarkus

Stop using risky dual writes. Learn how Debezium uses your database's transaction log to reliably stream every change and guarantee data consistency across your services.

Practical Change Data Streaming Use Cases With Debezium And Quarkus
#1about 3 minutes

Introduction to change data capture with Debezium

An overview of how change data capture (CDC) with Debezium and Quarkus can solve the problem of dual writes in microservices.

#2about 4 minutes

The challenge of data consistency with dual writes

Dual writes to multiple databases or services can lead to data inconsistencies when one of the writes fails.

#3about 6 minutes

Core concepts of Apache Kafka for event streaming

Apache Kafka is a fault-tolerant, scalable, publish-subscribe system designed for real-time event stream processing.

#4about 4 minutes

How change data capture (CDC) works

Change data capture automatically captures database changes like inserts, updates, and deletes and streams them as events.

#5about 5 minutes

Using Debezium for transaction log-based CDC

Debezium is a Kafka connector that taps into database transaction logs to reliably capture and propagate data changes.

#6about 2 minutes

The structure of a Debezium change event message

Debezium change events are JSON messages containing before and after states of the data, plus metadata about the operation.

#7about 5 minutes

Solving dual writes with the transactional outbox pattern

The outbox pattern ensures data consistency by writing business data and an event to an outbox table within a single database transaction.

#8about 5 minutes

Migrating monoliths with the strangler fig pattern

The strangler fig pattern uses CDC to replicate data from a monolith to a new microservice, enabling a gradual and safe migration.

#9about 3 minutes

Implementing the outbox pattern with Quarkus and Kubernetes

Use Quarkus to implement the outbox pattern and deploy the entire system, including Kafka managed by Strimzi, on Kubernetes.

#10about 6 minutes

Live demo of Debezium capturing database changes

A practical demonstration shows how inserting data into a database table automatically triggers Debezium to publish a change event to a Kafka topic.

#11about 10 minutes

Q&A on CDC implementation and operational challenges

Discussion covers the challenges of building a custom CDC solution, Debezium's fault tolerance, and handling lost transaction logs.

Related jobs
Jobs that call for the skills explored in this talk.

Featured Partners

Related Videos

See all videos
Event Messaging and Streaming with Apache Pulsar57:19

Event Messaging and Streaming with Apache Pulsar

Mary Grygleski

Quarkus. A Bliss for developers44:40

Quarkus. A Bliss for developers

Alex Soto

Let's Get Started With Apache Kafka® for Python Developers38:50

Let's Get Started With Apache Kafka® for Python Developers

Lucia Cerchie

Development of reactive applications with Quarkus39:10

Development of reactive applications with Quarkus

Niklas Heidloff

Application Modernization Leveraging Gen-AI for Automated Code Transformation41:17

Application Modernization Leveraging Gen-AI for Automated Code Transformation

Syed M Shaaf

Application Modernization and Rabbits33:40

Application Modernization and Rabbits

Niklas Heidloff

Cloud Chaos and Microservices Mayhem39:16

Cloud Chaos and Microservices Mayhem

Holly Cummins

2021: Familiar APIs on Kickass Runtimes #slideless37:55

2021: Familiar APIs on Kickass Runtimes #slideless

Adam Bien

From learning to earning

Jobs that call for the skills explored in this talk.

Kafka DevOps

REWE digital
Municipality of Madrid, Spain

Kafka
DevOps