Data Engineer

Job Title: Data Engineer

Location: Hybrid (2 days in office, Glasgow)

Day Rate: £390/day (Inside IR35)

About the Role:

We are seeking an experienced Data Engineer to join our blue-chip client’s dynamic team. The ideal candidate will have extensive hands-on experience in data engineering, particularly with Spark, Python, and Java. A background in banking or financial services is highly desirable, as you will work on projects that require domain-specific knowledge. If you have a deep understanding of big data ecosystems, enjoy solving complex problems, and thrive in a hybrid work environment, this role is for you!

Key Responsibilities:

• Develop and optimize scalable ETL pipelines using Spark and other big data technologies.
• Work on large datasets using HDFS, Hive, and similar tools in the big data ecosystem.
• Implement and maintain CI/CD pipelines to automate data workflows.
• Collaborate with cross-functional teams to design data solutions and support ongoing analytics efforts.
• Ensure data quality, availability, and performance in production environments.

Core Skills & Experience Required:

• Minimum of 8+ years of experience in data engineering.
• Hands-on experience with Spark 2.x/3.x.
• Proficiency in Python (or Scala/Java), with a preference for Python.
• Strong knowledge of HDFS, SQL, and experience in building ETL pipelines.
• Familiarity with big data ecosystems including data nodes/edge nodes, Hive, and similar technologies.
• Experience in data wrangling and working with large datasets.
• Competency with CI/CD pipelines and experience in deploying scalable data solutions.

Desirable:

• Previous experience in banking or financial services industries.
• Familiarity with financial data structures and regulatory requirements.