About:
The Data Engineer extracts, combines, transforms, loads, manipulates and cleanses data all whilst effectively handling interfaces to large amounts of data and creates/integrates data structures suitable for analysis.
The Data Engineer must understand data architecture and data management. Develop and deploys algorithms in performance-efficient code in analytical and data platforms. Liaise with enterprise architects, data governors, and subject matter experts to fill any gaps in the requirements or design.
Core Responsibilities:
Works and experiments with state-of-the-art data storage and processing technologies.
Builds large-scale batch and real-time data pipelines with data processing frameworks in cloud platforms or on-premise
Leverages best practices in continuous integration and delivery
Pre-process, clean and structures data to facilitate data exploration and advanced analytics/Machine Learning activities
Learns and uses modern data preparation, integration and AI-enabled metadata management tools and techniques.
Tracks data consumption patterns, working with data governance and information stewards within these teams, and participates in vetting and promoting content created in the business and by data scientists to the curated data catalogue for governed reuse
Performs intelligent sampling and caching.
Recommends and implements automation in existing and future integration flows.
Helps improve data quality by driving advancements in data acquisition, testing and tooling
Works in cross-functional agile teams to continuously experiment, iterate and deliver on new product objectives
Analyzes requested changes to fill in the details fitting in the complete picture and while doing that aligns with all parties involved (Architects, Data Governance, ...), either as preparation for yourself or for data engineer colleagues, before starting the implementation.
Education & Experience:
Bachelors or Masters Degree in Computer Science preferred or equivalent combination of education and experience
Minimum 4 years of experience as a data engineer
Experience in (captive) financial services and understands the processes and execution models in financial services is an asset
Experience in Azure SQL, Azure Data Factory, Azure DevOps, Databricks, Delta Lake is a must, certification is recommended
Experience in working with high-volume heterogeneous data sets 2
Experience in agile software processes and data-driven development
Experience with data lakes and dimensional models
Required Competencies and Skills:
Azure Data Engineer Associate certification is recommended
Databricks Data Engineer Professional certification is recommended
Strive for clean, reusable and performant code
Skilled ETL/ELT developer
Strong understanding of data and information
Business savvy, understand information from a business perspective and use that to more effectively develop enabling technologies
Strong problem-solving and analytical skills
Goal-oriented and positive attitude to deliver against targets
Able to work independently and proactively
Global, open, innovative, holistic and goal-oriented mindset
Ability to set and meet deadlines
Ability to work in high-pressure situations
Excellent attention to detail
Advanced level of English, oral and written
If you are interested in this position or know someone who would be perfect for this role please contact us to find out all details of the position!