Data Engineer

• Top Must-Haves:
• **-30% cleaning and handle, 30% building the pipeline, 30 % analytical skills
• 7+ years (around that) of experience in Data Engineering or Data Platform development.
• Finance or payroll domain experience is ideal- understands payroll data structures, pay period logic, compensation/deduction relationships, or has worked in financial services data environments.
• Experience handling large-scale datasets (billions of records, multi-TB, time-series data).
• Aws cloud experience (or azure cloud if databricks exp is there)
• Strong Databricks proficiency with deep understanding of:
• Unity Catalog (governance, access patterns, catalog/schema design)
• Delta Lake internals (optimization, clustering, change data feed, versioning)
• Databricks Workflows (orchestration, dependencies, monitoring)
• Databricks Asset Bundles (Declarative automation bundles) or equivalent deployment frameworks
• Exposure to architecture-level design decisions (can participate in architecture discussions, understands tradeoffs).

• Strong proficiency in:

• Python
• SQL
• PySpark
• Data Modeling
• ETL/ELT Development

Analytical fluency: exploratory data analysis, basic statistical concepts (correlation, distributions, time-series patterns), feature engineering support.

Proficiency with pandas/numpy for ad-hoc analytical work alongside production PySpark.

CI/CD implementation experience (Bitbucket Pipelines, Jenkins, automated deployment).

Proficiency with AI-assisted development tools (GitHub Copilot, Amazon Q, Kiro, or equivalent).

Data quality and validation frameworks