Requirements: English
Company: emagine Polska
Region: Warsaw , Masovian Voivodeship
Project information:
Industry: Insurance
Rate: up to 185 pln/h net + vat, B2B
Location: Warsaw first 2-3 months hybrid, then remote
Project language: Polish, English
About the Role
As aSenior Data Engineer, you will design and build scalableData Hubsthat supportanalytical, reporting, operational, and Generative AI use cases. This role involves definingstandards, best practices, and architecture patterns, mentoring engineers, and ensuring high-quality data solutions.
Responsibilities:
Data Hub ArchitectureDesign and implement efficient, scalableData Hubsintegrating multiple data sources.
Technical LeadershipDefinebest practices, patterns, and standards, ensuring consistency across implementations.
Mentoring & StrategyGuide engineers, review code, and providetechnical direction.
Data PipelinesDevelopbatch and real-time data processingworkflows.
Quality & MonitoringImplementdata validation, anomaly detection, and automated monitoring, taking action when needed.
Automation & CI/CDEnable seamlessdeployment and infrastructure automation.
Security & ComplianceEnsuredata governance, security, and regulatory compliance.
DocumentationMaintain clear, structuredtechnical documentation.
Requirements:
Python, SQLStrong data engineering and automation expertise.
Databricks & Spark Deep knowledge ofDatabricks (primary tool) and Apache Spark.
Azure Data StackExperience withData Factory, ADLS, Synapse, Azure SQL, Event Hubs.
Real-Time ProcessingExpertise instreaming data and real-time analytics.
Automation & DevOpsProficiency inCI/CD, Terraform, Kubernetes/AKS, Docker.
Leadership & MentoringAbility toset direction, mentor engineers, and drive best practices.
Documentation & CommunicationClear, structuredtechnical writing and knowledge sharing.
Language SkillsEnglish proficiency, minimum B2 level.
Technology Stack
Primary ToolsDatabricks, Apache Spark, Delta Lake.
Azure ServicesData Factory, ADLS, Azure SQL, Synapse, Azure DevOps, Event Hubs.
Development & AutomationPython, SQL, GitHub, Terraform, Docker, Kubernetes/AKS.