Lead Data Engineer

Holborn and Covent Garden
1 month ago
Applications closed

Related Jobs

View all jobs

BI Developer

Data Engineer (MS Fabric) - Perm - Hybrid London

Senior Data Engineer - Databricks

Data Ops Engineer

Software Engineer

CDD Platform Lead

Lead Data Engineer ( Databricks )
London - Hybrid - Remote
Permanent
£100,000 - £130,000 plus up to 20% bonus based on performance and commercial contribution

About the Role

We’re looking for a Lead Data Engineer to spearhead some of our clients most strategic Databricks engagements.

This is a senior client-facing leadership role, blending hands-on technical delivery with architectural design and pre-sales influence.

You'll be leading high-performing squads, guiding complex transformations, and working directly with senior stakeholders to bridge business needs and engineering excellence — particularly in industries like manufacturing, utilities, and aviation.

This is a key hire to support our clients expanding Databricks practice, to build capacity for future growth.

What You’ll Be Doing

  • Act as the technical lead on client engagements, owning design and delivery of data solutions in Databricks.

  • Architect robust, scalable data platforms using the medallion architecture.

  • Translate business requirements into scalable workflows, advising on data governance, quality, and security.

  • Design and implement complex data pipelines using tools like Delta Live Tables (DLT) and Unity Catalog.

  • Guide teams in implementing best practices across engineering, DevOps, and model deployment.

  • Support pre-sales activity, including shaping proposals, estimates, and technical roadmaps.

  • Provide technical leadership, mentorship, and oversight to squads of Senior and Associate Engineers.

  • Collaborate closely with Platform Engineers and Platform Architects to align infrastructure with data needs.

  • Contribute to growing the Databricks capability – from delivery frameworks to internal tooling and capability development.

  • Lead a team of data engineers, fostering a collaborative and growth-oriented environment.

  • Evaluate new data engineering technologies and strategies, assessing their relevance and fit for the organisation’s strategic goals.

  • Work closely with the commercial team to scope projects and develop proposals that align technical capabilities with client requirements.

    Essential Skills & Experience

  • 8+ years in data engineering, with at least 2+ in a technical leadership role

  • Proven experience designing and leading Databricks-based data platforms

  • Deep understanding of the medallion architecture, data lakehouse design, and transformation workflows

  • Hands-on expertise with DLT, Unity Catalog, and model deployment frameworks

  • Strong communication and consulting skills – able to lead client conversations and manage stakeholders

  • Experience in agile delivery environments and cross-functional teams

  • Commercial awareness – comfortable contributing to pre-sales, growing accounts, and engaging with commercial targets

    Desirable Skills

  • Experience in physical asset-heavy industries (e.g. utilities, manufacturing, aviation)

  • Familiarity with platform and DevOps collaboration, especially on AWS or Azure

  • Certifications in Databricks or cloud platforms (AWS/Azure)

  • Background in consulting or client delivery environments

    Why Join?

  • Join a consultancy that’s doubling down on Databricks with enterprise-grade delivery

  • Be the go-to technical leader on projects with real-world business impact

  • Shape the future of our Databricks workforce strategy and delivery model

  • Career progression into Delivery Lead, Practice Lead, or Pre-Sales Specialist

  • Competitive compensation and strong bonus structure, aligned with delivery and commercial impact

    To find out more about this high profile Lead Data Engineering position, click apply

Get the latest insights and jobs direct. Sign up for our newsletter.

By subscribing you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Quantum-Enhanced AI in the Cloud: A New Dawn for Distributed Computing

Cloud computing has reshaped the digital landscape, freeing organisations from on-premises hardware constraints and paving the way for services that scale on demand. From hosting business-critical databases to enabling global content delivery networks, the cloud has become the backbone of modern IT. Parallel to this evolution, Artificial Intelligence (AI) has rapidly matured, offering transformative capabilities in data analytics, recommendation systems, and automation. Yet for all its power, conventional computing infrastructure sometimes struggles to handle the complex, high-dimensional problems AI can generate—especially in areas like drug discovery, advanced cryptography, or intricate optimisation. Quantum computing has burst onto the scene as a new paradigm that might address these computational bottlenecks. By leveraging quantum bits (qubits) and phenomena like superposition and entanglement, quantum computers could, in principle, explore vast solution spaces at unprecedented speeds. What happens when these three forces—cloud computing, AI, and quantum technology—converge? We enter a new era of quantum-enhanced AI in the cloud, where distributed systems seamlessly integrate classical and quantum resources to deliver powerful, scalable, and efficient solutions. This article will explore: How cloud computing has set the stage for next-generation AI. The basics of quantum computing and why it’s poised to disrupt traditional workloads. Real-world applications of quantum-enhanced AI that leverage cloud infrastructure. Emerging job roles and skill requirements in this fast-evolving landscape. Ethical, technical, and logistical considerations that must guide future developments. Whether you’re a seasoned cloud professional looking to stay ahead of the curve, a data scientist curious about the promise of quantum, or a tech enthusiast eager to see what’s next, read on. The future of computing may be closer than you think.

Cloud Jobs at Newly Funded UK Start-ups: Q3 2025 Investment Tracker

Cloud computing continues to be a pivotal force in the UK tech scene. From serverless architectures and edge computing solutions to AI-driven data analytics platforms, start-ups across the country are leveraging cloud technologies to innovate at lightning speed. This trend is far from a passing phase; in fact, cloud adoption has become a strategic imperative for businesses of all sizes, enhancing their scalability, flexibility, and competitive edge in a global marketplace. In Q3 2025, UK-based cloud computing start-ups attracted impressive funding rounds, signalling robust investor confidence in the sector. Whether you’re a cloud engineer, DevOps practitioner, project manager, or aspiring to break into cloud-based roles, these newly funded ventures are on the lookout for talent. This Q3 2025 Investment Tracker will guide you through the most notable funding news, the job opportunities that these investments have created, and how you can position yourself to land a coveted role at these cutting-edge companies. Lastly, we’ll introduce you to Cloudcomputingjobs.co.uk—a dedicated platform that connects cloud professionals with emerging start-ups and established enterprises alike. If you’re eager to harness the momentum of this booming sector, read on to discover the latest hiring trends, in-demand skill sets, and practical steps to accelerate your cloud career.

Portfolio Projects That Get You Hired for Cloud Jobs (With Real GitHub Examples)

As cloud computing becomes the de facto choice for businesses of all sizes, cloud professionals are in high demand. From infrastructure automation to container orchestration and serverless architecture, cloud roles require a blend of technical expertise and practical, hands-on skills. Whether you’re aiming to be a Cloud Engineer, Solutions Architect, DevOps Professional, or any role involving AWS, Azure, or Google Cloud Platform (GCP), a well-curated portfolio of projects is often the deciding factor for hiring managers. In this comprehensive guide, we’ll explore: Why a portfolio matters more than ever in the cloud computing job market. Which types of projects align with different cloud roles. Real GitHub examples that exemplify best practices in cloud development and deployment. Actionable project ideas to help you stand out from the crowd. Best practices for structuring and presenting your repositories for maximum impact. By the end, you’ll have a clear blueprint to build an impressive portfolio that resonates with potential employers. And when you’re ready to make your move, don’t forget to upload your CV on Cloud-Jobs.co.uk so that top companies seeking cloud talent can see your experience firsthand.