Requirements: English
Company: TN Poland
Region: Wieliczka , Lesser Poland Voivodeship
We''re seeking a Senior Data Engineer to architect, build, and scale our data warehouse solutions and data products. You''ll work with a modern data stack and have the opportunity to shape our data infrastructure while solving complex data engineering challenges.
Your main responsibilities:
- Design, implement, and optimize data pipelines using Apache Airflow and Python
- Build real-time data streaming solutions using Apache Kafka
- Architect and maintain our data warehouse infrastructure
- Develop RESTful APIs using FastAPI to expose data products
- Create and maintain database migrations using Alembic
- Develop and optimize complex SQL queries across different database environments
- Create robust data quality monitoring and testing frameworks
- Collaborate with data scientists and analysts to enable efficient data access
- Mentor junior engineers and contribute to technical decision-making
- Design and implement data models that balance performance and maintainability
You''re ideal for the role if you have:
- 5+ years of experience in data engineering roles
- Strong expertise in Python development and SQL optimization
- Hands-on experience with:
- Apache Airflow for workflow orchestration
- Apache Kafka for stream processing
- PostgreSQL or similar RDBMS
- Analytics databases (StarRocks, DuckDB, or similar MPP databases)
- FastAPI for building data APIs
- DBT for data transformation and documentation
- Alembic for database migrations and schema management
- Understanding of data modeling concepts (dimensional modeling)
- Understanding of data transformation best practices and testing methodologies
- Experience with data quality management and testing frameworks
- Familiarity with cloud platforms (AWS/GCP/Azure)
- Version control (Git) and CI/CD practices
#J-18808-Ljbffr