Our investment banking client are seeking an very experienced Data Lakehouse Lead with a proven track record of building and maintaining data lakehouses based on Databricks within the banking sector, preferably within the derivatives space.

Please note you MUST have investment banking experience in middle/Front Office roles, ideally in the risk and derivatives space. Applicants without this experience cannot be considered.

The ideal candidate will have a strong background in Scala, with specific experience working in Databricks. The successful candidate will join our team of data professionals and be responsible for designing, implementing, and maintaining the platform and the data pipelines to support our financial reporting requirements.

The platform is used by Front Office technical staff, so this is an exciting opportunity for the candidate to work directly with the Front Office traders and quants.

Responsibilities:

• Design and implement data pipelines to support financial reporting and Front Office analytics requirements.
• Design and implement MLOps pipelines (MLFlow).
• Develop and maintain data ingestion and processing systems in Databricks.
• Collaborate with business stakeholders to understand their data requirements and provide appropriate solutions.
• Build scalable and efficient data processing pipelines to handle large volumes of data.
• Ensure data quality and consistency throughout the data pipeline.
• Troubleshoot issues and optimize performance of the data pipeline.
• Document data pipelines and processes to ensure knowledge transfer and maintainability.

Requirements:

• Bachelor's or Master's degree in Computer Science, Information Systems, or related field.
• At least 5 years of experience as a data engineer.
• Must have good middle and Front Office business knowledge ideally covering risk and derivatives.
• Proven experience building and maintaining data pipelines in the derivatives space.
• Experience working with Databricks.
• Strong programming skills in Scala and Python.
• Solid understanding of database design principles and Datamodelling.
• Familiarity with data integration tools and technologies.
• Experience with Agile methodologies and SDLC practices.
• Strong problem-solving and analytical skills.
• Excellent written and verbal communication skills.

My client is very open to remote working with occasional travel to their central London offices.

Please send your CV for immediate consideration.

Lead Data Engineer (Databricks), Banking, London, Outside IR35, Remote