Python-Based Data Streaming Pipelines Within Minutes
Bobur Umurzokov - 3 months ago
Implementing production-ready streaming data pipelines can be lengthy and complex, often taking months to set up and manage. Infrastructure challenges make it difficult for Data Engineers to simulate production environments, while Data Scientists face challenges with data integration, building offline AI/ML pipelines, and reproducing models for debugging. In this talk, we will explore a modern approach in practice to empower your data team to independently create, deploy, and manage data pipelines using Python. Learn how to make your Python code run in streaming with serverless infrastructure from day 1 without heavily relying on external teams. You will learn about tools and technologies for zero-infrastructure transformations that every streaming needs. We will leverage free-to-start tools like GlassFlow, Debezium, NATS message broker for processing real-time events, Docker, and Kubernetes to build a sample AI-powered data pipeline.
Jobs with related skills

Senior Frontend Developer (m/f/x)
ALDI DX
·
1 month ago
Mülheim an der Ruhr, Germany
Hybrid

Field Application Engineer (FAE) (m/w/d)
Lauterbach GmbH
·
16 days ago
Höhenkirchen-Siegertsbrunn, Germany

Senior Backend Developer (m/w/x)
ÖBB-Konzern
·
1 month ago
Vienna, Austria
Hybrid

Virtuoso EDA Software Engineer and Administrator
Fraunhofer-Institut für Integrierte Schaltungen IIS
·
1 month ago
Erlangen, Germany
Hybrid
Related Videos