Oh no! This role has already been filled.

Don't worry, we have lots of other exciting jobs for you!

See available jobs in Data

That job has expired. Here are some similar roles:

Social Insight Quality Control Analyst

SALT is supporting a global FMCG company, who are looking for a meticulous and insights-savvy "Social Insight Quality Control Analyst" to oversee the accuracy and consistency of deliverables across a…

Data Analyst

New Job Alert - Senior Data Analyst - Local Government Transformation A UK local authority is undertaking a significant, multi-year transformation programme across core service areas including Adult Social Care,…

Data Analyst – Power BI

ob Title: Data Analyst (Power BI) Location: Remote Contract Length: 6 months (with potential extension) Start Date: ASAP Rate: £325 (Inside IR35) Overview: We are currently seeking an experienced Data…

£300 - £325.00 per day

Social Measurement Director

I'm working with a well-known social-first creative agency based in London, where every campaign is built around measurable social impact. They are looking for a senior Social Measurement Director to…

£375 - £425.00 per day

Senior Data Engineer (AWS,DevOps,Big Data) – Finance – London

Data

Greater London Contract / 6 months £500 - £650 per day

Data Engineer (AWS,DevOps,Big Data) – Finance – London

Day rate: £550 – £650 inside IR35

Duration: 6 months

Start: ASAP

Hybrid 2 – 3 days in office

My new client is looking for a highly skilled Data Engineer with expertise in cloud DevOps, big data administration, and data engineering on AWS. The ideal candidate will have a deep understanding of AWS Lake Formation, data product concepts, and Spark job management, including auditing, monitoring, and performance tuning. This role involves creating and managing services and tools to support a multi-tenant environment, ensuring optimal performance and scalability.

Key Responsibilities:

  • Cloud DevOps & Big Data Administration:
    • Manage and optimize big data environments on AWS, with a focus on efficient administration and maintenance.
    • Leverage AWS Lake Formation to design and implement data lakehouses versus data fabric architectures, ensuring data integrity and accessibility.
  • Data Engineering & Spark Management:
    • Develop and maintain Spark jobs, with a focus on auditing, monitoring, and instrumentation to ensure reliability and performance.
    • Perform Spark performance tuning, including understanding and applying Spark 3+ features, such as Adaptive Query Execution (AQE) and job-level resource management.
    • Create services and tools to manage a multi-tenant environment, ensuring seamless data operations across tenants.
  • Infrastructure as Code (IaC):
    • Utilize Terraform for infrastructure provisioning and management, ensuring scalable and secure environments.
    • Integrate with AWS Glue and HIVE for data processing and management, optimizing workflows for large-scale data operations.
  • Data Storage & Management:
    • Work with data storage formats like Parquet and Hudi to optimize data storage and retrieval.
    • Implement and manage IAM policies for secure data access and management across AWS services.
  • Collaboration & Continuous Improvement:
    • Collaborate with cross-functional teams, including data scientists, analysts, and other engineers, to develop and deploy data solutions.
    • Continuously improve data engineering practices, leveraging new tools and techniques to enhance performance and efficiency.

Required Skills:

  • Cloud DevOps & Big Data:
    • Extensive experience in cloud DevOps and big data administration on AWS.
    • Proficiency in AWS Lake Formation, with a strong understanding of data lakehouse versus data fabric concepts.
  • Programming & Data Engineering:
    • Expertise in Python for data processing and automation.
    • Deep knowledge of Spark internals (Spark 3+ features), including architecture, events, system metrics, AQE, and job-level resource management.
  • Tech Stack:
    • Strong hands-on experience with Terraform for infrastructure management.
    • Experience with data formats such as Hudi and Parquet.
    • Familiarity with AWS Glue, HIVE, and IAM for data management and security.

Good to Have:

  • Additional Tools & Technologies:
    • Knowledge of Iceberg and its application in data storage and management.
    • Experience with Airflow for workflow automation.
    • Familiarity with Terragrunt for managing Terraform configurations.
    • Understanding of DynamoDB and its integration within data environments.

Job Information

Job Reference: JO-2408-346896
Salary: £500 - £650 per day
Salary per: day
Job Duration: 6 months
Job Start Date: ASAP
Job Industries: Data
Job Locations: Greater London
Job Types: Contract

Here are some related jobs

Senior Data Scientist

Senior Data Scientist - Marketing FocusAre you a highly skilled Data Scientist with a passion for marketing and a knack for driving impact across diverse business lines? We're partnering with…

£60000.00 - £70000.00 per annum

Senior Insight Analyst

SALT is looking for an experienced social-listening and digital-insights specialist who can manage a global team of 8 freelance analysts and deliver insight projects end-to-end at pace. You'll be the…

£0.00 - £200.00 per day

Senior IT Systems Administrator (Windows)

I'm working with a leading University in the South of England who are looking for a Senior IT Systems Administrator (Windows). The role is focused on managing and developing core…

×

Upload your CV

Upload your CV to our database.

  • Max. file size: 49 MB.
  • Hidden
  • This field is for validation purposes and should be left unchanged.
Senior Data Engineer (AWS,DevOps,Big Data) - Finance - London

Please let us know where you are, or where you would like to be in the world so we can point you in the right direction.