Requirements: English
Company: OneSource Consulting
Job Title:- Kafka Expert
Location:- Brussels, Belgium
Language:- English, Dutch or French
Duration:- Longterm
Expected responsibilities
- Design, develop, and maintain data pipelines using Apache Kafka and AWS services
- Implement and manage Kafka Connect connectors for data integration
- Configure and maintain Kafka Schema Registry for data governance
- Monitoring and optimizing performance of Kafka clusters on MSK
- Develop or assist with developing Java applications that interact with Kafka ecosystem
- Write Infrastructure as Code using Terraform for AWS resources (optional, but nice to have)
- Collaborate with team members on data architecture decisions
- Implement data security and compliance measures
- Troubleshoot and resolve production issues
Required Technical Skills
- Strong experience with Apache Kafka ecosystem:
- Kafka message brokers and topics
- Kafka Connect framework
- Schema Registry and Avro serialization
- Proficient in Java development or able to translate Kafka topics towards Java implementation
- Experience with AWS services:
- Amazon MSK (Managed Streaming for Kafka)
- S3, RDS, and DynamoDB
- Lambda, SNS, and SQS
- IAM and security best practices
- Infrastructure as Code using Terraform
- Version control with Git
- Optional Additional Skills
Experience with:
- AWS Glue for ETL processes
- Databricks platform
- Snowflake data warehouse
- AWS Kinesis Firehose
- Knowledge of data modeling and design patterns
- Understanding of event-driven architectures
- Experience with CI/CD pipelines