AWS Data Engineer – (EMR, ETL, Hudi) – Finance – London
Day rate: £550 – £750 (inside IR35)
Duration: 3 – 6 months
Start: ASAP
Hybrid: In office 2 days per week.
Job Description:
My new client is looking for a Data Engineer with expertise in AWS data engineering services, particularly EMR, Glue, Hudi, Lake Formation, Athena, and Kinesis. The ideal candidate will also have experience in using Terraform to provision these services. As a Data Engineer, you will play a critical role in shaping our data infrastructure, ensuring data is collected, processed, and made accessible to various stakeholders efficiently and securely.
Key Responsibilities:
AWS Data Engineering: Leverage your expertise in AWS data engineering services to design, build, and maintain data pipelines and ETL processes using services such as EMR, Glue, Hudi, Lake Formation, Athena, and Kinesis.
Terraform Infrastructure: Utilize Terraform to provision and manage AWS resources, ensuring the scalability and reliability of the data infrastructure.
Data Transformation: Develop and optimize ETL processes to transform raw data into usable formats, ensuring data quality, consistency, and performance.
Data Warehousing: Work with data warehousing solutions to enable efficient storage and retrieval of data for analytical purposes.
Data Modeling: Collaborate with data scientists and analysts to design and implement data models that support business requirements.
Data Security: Implement security best practices to protect sensitive data and ensure compliance with relevant regulations and policies.
Monitoring and Optimization: Set up monitoring tools and processes to identify and resolve performance and reliability issues. Continuously optimize data pipelines for efficiency.
Documentation: Maintain comprehensive documentation of data engineering processes, infrastructure, and configurations.
Qualifications:
- Bachelor’s degree in computer science, engineering, or a related field (or equivalent work experience).
- Proven experience in AWS data engineering services, including EMR, Glue, Hudi, Lake Formation, Athena, and Kinesis.
- Proficiency in using Terraform to provision and manage AWS resources.
- Strong SQL and database management skills.
- Experience with data modeling and ETL processes.
- Familiarity with data warehousing technologies.
- Knowledge of data security and compliance best practices.
- Strong problem-solving skills and attention to detail.
- Excellent communication and collaboration skills.
- Ability to work in a fast-paced, dynamic environment.
Preferred Qualifications:
- AWS certification in data-related services.
- Experience with real-time data streaming technologies.
- Knowledge of big data technologies like Apache Spark and Hadoop.
- Familiarity with data orchestration and workflow management tools.
Job Information
Job Reference: JO-2310-338823
Salary: £500 - £750 per day
Salary per: zero
Job Duration: 3 - 6 months
Job Start Date: ASAP
Job Industries: Data
Job Locations: Greater London
Job Types: Contract