Pay · estimate~$120k – $160kMarket rate · not set by employer
PostedNov 19, 2026
LevelMid level
In short
We are seeking an experienced Data Engineering Tech Lead to lead the design, development, optimization, and maintenance of scalable enterprise data engineering and Lakehouse…
SQLPYTHONAZUREDATABRICKSSPARK
📋 About the job
📍
LocationIndianapolis, IN (Onsite)
🗂️
Job typeLong Term Contract
💼Job Title: Lead Azure Data Engineer
Required Experience: 12+ Years
🏢Position Overview
We are seeking an experienced Data Engineering Tech Lead to lead the design, development, optimization, and maintenance of scalable enterprise data engineering and Lakehouse solutions on the Azure cloud platform. In this role, you will drive both batch and near-real-time data processing across complex enterprise data sources, enforce robust data governance, and mentor engineering teams to enable data transformation in the Healthcare sector.
🔹12+ overall years of experience in Data Engineering / IT industry roles.
🔹5+ years of hands-on experience across the Azure Cloud platform ecosystem.
🔹4+ years of core experience with Azure Databricks and PySpark .
🔹3+ years of experience with Azure Data Factory (ADF) .
🔹3+ years of experience with ADLS Gen2 .
🔹3+ years of experience with Azure SQL .
🔹2+ years of experience in Databricks Workflows and Unity Catalog governance.
🔹2+ years of experience in modular Python programming , package builds, and testing (e.g., PyTest/unittest).
Key Responsibilities
🔹Data Engineering & Lakehouse Architecture
🔹Design, build, and optimize scalable Lakehouse solutions ingesting structured, semi-structured, and unstructured data from ERP platforms, APIs, relational databases, NoSQL stores, and on-premises sources.
🔹Optimize Spark jobs and handle large-scale data processing using DataFrame and RDD APIs.
🔹Implement performance tuning, partitioning strategies, file format optimizations (Parquet/Delta), and Spark SQL tuning.
🔹Governance, Storage & Pipeline Orchestration
🔹Implement data access controls, object lineage, audit trails, and security policies using Unity Catalog integrated with Azure AD.
🔹Architect multi-layered storage patterns (Bronze-Silver-Gold) on ADLS Gen2 using lifecycle policies, RBAC/ACLs, and folder-level security.
🔹Build modular, parameterized orchestration workflows using ADF and Databricks Workflows (custom activities, triggers, dynamic expressions, error handling).