Requirements: English
Company: NextChallenge
Region: Krakw , Lesser Poland Voivodeship
We are looking for an experienced and driven Lead Data Engineer with a deep understanding of the iGaming industry.
About WhizdomAI:
WhizdomAI is revolutionising the iGaming industry with data-driven recommendations and personalisation services. We are seeking an exceptional Lead Data Engineer to spearhead the strategic initiative to establish a standardised, scalable, and efficient platform that reduces integration complexity, optimises infrastructure, streamlines client onboarding, and enhances the delivery of actionable insights for iGaming operators.
Role Overview:
As the Lead Data Engineer, you will be responsible for architecting, implementing, and leading the development of the companys next-generation data platform. You will work across multiple workstreams to establish the standardised data integration framework, implement the core iGaming data model, and build efficient transformation processes that will form the foundation of the companys advanced analytics and recommendation engine.
Key Responsibilities:
- Lead the design and implementation of the companys unified data integration framework and standardised iGaming data model;
- Architect the data pipeline for batch, real-time, and clickstream data processing;
- Design and implement data transformation frameworks using dbt and other modern data tools;
- Develop secure multi-tenant data architecture with appropriate isolation controls;
- Establish comprehensive data quality frameworks with automated validation rules, monitoring, and alerting;
- Design and implement data governance policies, including metadata management, data lineage tracking, and data catalogue solutions.
Team Leadership & Execution:
- Lead a team of data engineers, providing technical guidance, mentorship, and code reviews;
- Collaborate with cross-functional teams including data scientists, analytics engineers, and client implementation specialists;
- Establish engineering best practices, coding standards, and documentation requirements;
- Drive implementation of CI/CD pipelines for data infrastructure and transformation code;
- Manage technical dependencies across workstreams and coordinate development priorities;
- Lead data governance initiatives, working with stakeholders to define data ownership, stewardship roles, and governance processes;
- Foster a culture of data quality and data-driven decision making across the organisation.
Client Implementation & Optimization:
- Design client-specific data connectors and transformation processes for major iGaming platforms;
- Implement performance optimisation for query execution and resource utilisation;
- Develop scalable solutions that can accommodate growing data volumes and client base;
- Work with client implementation teams to ensure successful deployment and data validation;
- Establish monitoring and observability for all data pipeline components;
- Implement comprehensive data quality validation at all stages of the data lifecycle;
- Design and enforce data quality SLAs with proper measurement and reporting;
- Create data quality scorecards and dashboards for continuous improvement.
Required Qualifications:
- 7+ years of experience in data engineering, with at least 3 years in a lead or senior role;
- Expert knowledge of data modelling techniques and dimensional modelling for analytics;
- Extensive experience with AWS data services (Redshift, S3, Glue, Lambda, etc.);
- Strong proficiency with dbt and SQL for data transformation;
- Experience with streaming data technologies (Kafka, Kinesis) and real-time processing;
- Hands-on experience with Python, Airflow (or similar orchestration tools), and infrastructure-as-code;
- Understanding of data security, encryption, and multi-tenant architecture principles;
- Experience implementing CI/CD practices for data infrastructure and pipelines;
- Strong background in implementing data quality frameworks and validation processes;
- Experience with data governance tools, metadata management, and data catalogues;
- Knowledge of data lineage tracking and impact analysis methodologies.
Domain Knowledge:
- Experience working with high-volume, multi-tenant SaaS platforms;
- Understanding of real-time analytics and recommendation engines;
- Familiarity with iGaming or similar transactional platforms is a plus;
- Experience with clickstream data processing and user behaviour analytics;
- Knowledge of regulatory compliance requirements related to data governance in regulated ind