Oh no! This role has already been filled.

Don't worry, we have lots of other exciting jobs for you!

See available jobs in Data

That job has expired. Here are some similar roles:

Azure Data Engineer – Local Government / Public Sector

Job Title: Data Engineer - Azure | Local Government | Inside IR35 Location: Central London (Hybrid - 1-2 days on-site per week) Rate: £500/day (Inside IR35) Contract: 6 months (initial)…

inside IR35

Senior Data Scientist

Senior Data Scientist - Marketing FocusAre you a highly skilled Data Scientist with a passion for marketing and a knack for driving impact across diverse business lines? We're partnering with…

£60000.00 - £70000.00 per annum

Data Analyst – Power BI

Job Title: Data Analyst (Power BI) Location: Remote Contract Length: 6 months (with potential extension) Start Date: ASAP Rate: £325 (Inside IR35) Overview: We are currently seeking an experienced Data…

Children’s Services Data Analyst – Local Government

Data Analyst - Children's Services £35 per hour | Inside IR35 | London Borough | Hybrid (1 day onsite per week) 3-Month Rolling Contract | Start Date: 2nd June 2025…

£0.00 - £35.00 per hour + Inside IR35

Senior Data Engineer (AWS,DevOps,Big Data) – Finance – London

Data

Greater London Contract / 6 months £500 - £650 per day

Data Engineer (AWS,DevOps,Big Data) – Finance – London

Day rate: £550 – £650 inside IR35

Duration: 6 months

Start: ASAP

Hybrid 2 – 3 days in office

My new client is looking for a highly skilled Data Engineer with expertise in cloud DevOps, big data administration, and data engineering on AWS. The ideal candidate will have a deep understanding of AWS Lake Formation, data product concepts, and Spark job management, including auditing, monitoring, and performance tuning. This role involves creating and managing services and tools to support a multi-tenant environment, ensuring optimal performance and scalability.

Key Responsibilities:

  • Cloud DevOps & Big Data Administration:
    • Manage and optimize big data environments on AWS, with a focus on efficient administration and maintenance.
    • Leverage AWS Lake Formation to design and implement data lakehouses versus data fabric architectures, ensuring data integrity and accessibility.
  • Data Engineering & Spark Management:
    • Develop and maintain Spark jobs, with a focus on auditing, monitoring, and instrumentation to ensure reliability and performance.
    • Perform Spark performance tuning, including understanding and applying Spark 3+ features, such as Adaptive Query Execution (AQE) and job-level resource management.
    • Create services and tools to manage a multi-tenant environment, ensuring seamless data operations across tenants.
  • Infrastructure as Code (IaC):
    • Utilize Terraform for infrastructure provisioning and management, ensuring scalable and secure environments.
    • Integrate with AWS Glue and HIVE for data processing and management, optimizing workflows for large-scale data operations.
  • Data Storage & Management:
    • Work with data storage formats like Parquet and Hudi to optimize data storage and retrieval.
    • Implement and manage IAM policies for secure data access and management across AWS services.
  • Collaboration & Continuous Improvement:
    • Collaborate with cross-functional teams, including data scientists, analysts, and other engineers, to develop and deploy data solutions.
    • Continuously improve data engineering practices, leveraging new tools and techniques to enhance performance and efficiency.

Required Skills:

  • Cloud DevOps & Big Data:
    • Extensive experience in cloud DevOps and big data administration on AWS.
    • Proficiency in AWS Lake Formation, with a strong understanding of data lakehouse versus data fabric concepts.
  • Programming & Data Engineering:
    • Expertise in Python for data processing and automation.
    • Deep knowledge of Spark internals (Spark 3+ features), including architecture, events, system metrics, AQE, and job-level resource management.
  • Tech Stack:
    • Strong hands-on experience with Terraform for infrastructure management.
    • Experience with data formats such as Hudi and Parquet.
    • Familiarity with AWS Glue, HIVE, and IAM for data management and security.

Good to Have:

  • Additional Tools & Technologies:
    • Knowledge of Iceberg and its application in data storage and management.
    • Experience with Airflow for workflow automation.
    • Familiarity with Terragrunt for managing Terraform configurations.
    • Understanding of DynamoDB and its integration within data environments.

Job Information

Job Reference: JO-2408-346896
Salary: £500 - £650 per day
Salary per: day
Job Duration: 6 months
Job Start Date: ASAP
Job Industries: Data
Job Locations: Greater London
Job Types: Contract

Here are some related jobs

Business Operations Team Leader

📣 New Opportunity - Business Operations Team Leader | Remote | 6 Months FTC Secondment Must have HE Background I'm working with a well-established, mission-driven education provider currently seeking a…

£0.00 - £30627.00 per annum

Digital Analyst – 3-month temp to perm contract

Salt is looking for an experienced Digital Analyst (3-month FTC) to support campaign tracking, reporting and insights across digital, web, social, and email activity. The role sits within a multi-disciplinary…

Merton Borough Council

NEW JOB ALERT - CONTRACT - HEAD OF COMMISSIONING & MARKET DEVELOPMENT FOR COMMUNITY AND HOUSING A local authority in London is seeking an experienced strategic leader to take on…

×

Upload your CV

Upload your CV to our database.

  • Max. file size: 49 MB.
  • This field is hidden when viewing the form
  • This field is for validation purposes and should be left unchanged.
Senior Data Engineer (AWS,DevOps,Big Data) - Finance - London

Please let us know where you are, or where you would like to be in the world so we can point you in the right direction.