Posted 2 days ago
Lead Data Engineer
- Work arrangement not specified
- Azure
- Data Engineering
- Python
- Apache Spark
$130,000 - $140,000 year
Job description
Robert Half is seeking a Lead Cloud Data Engineer to provide technical leadership in the design, development, and optimization of scalable cloud-based data solutions supporting enterprise reporting, analytics, and business operations. This position will focus heavily on Azure Databricks, leading complex data integration initiatives, establishing engineering standards, and helping shape the organization's cloud data architecture.The ideal candidate combines strong hands-on data engineering experience with the ability to lead technical design, solve complex data challenges, and guide other engineers. This individual will work closely with the Data Architect, business stakeholders, and technical teams to develop reliable, high-performing data solutions. Experience with manufacturing environments or ERP data is highly valued.Residency Requirement: Candidates must currently reside in Wisconsin to be considered for this position.Flexible hybrid/remote work schedule, with occasional onsite for meetings.Key ResponsibilitiesLead the technical design, development, and optimization of scalable ETL/ELT pipelines using Azure Databricks, PySpark, Python, and SQL.Partner closely with the Data Architect to design cloud data solutions, establish engineering standards, and contribute to long-term data architecture decisions.Lead complex data integration initiatives involving ERP, manufacturing, supply chain, inventory, finance, and other enterprise systems.Design and implement cloud-based lakehouse solutions using Delta Lake and Medallion Architecture.Provide technical direction on data modeling, ingestion frameworks, pipeline orchestration, and data transformation strategies.Develop reusable engineering frameworks, coding standards, and best practices to improve scalability, reliability, and maintainability.Integrate Azure Databricks with Azure Data Factory, Azure Data Lake Storage, and other cloud services.Lead troubleshooting and root cause analysis for complex production issues, optimizing pipelines and Databricks environments for performance and cost efficiency.Establish and promote best practices for data quality, validation, governance, security, and monitoring.Collaborate directly with business stakeholders, analysts, and IT teams to translate complex business requirements into scalable technical solutions.Mentor other data engineers, provide technical guidance, and lead code reviews and design discussions.Support CI/CD practices, automated deployments, and continuous improvement initiatives across the data engineering environment.
More jobs at Robert Half
Senior Data Engineer
Robert Half Recruiter Green Bay, Wisconsin, United States🇺🇸
Other jobs in United States
Injury Claims Settlement Negotiator - Scottsdale, AZ
Sivad Consulting Scottsdale, Arizona, United States🇺🇸
Chapel Hill Full Time Team Member
Chick-fil-A United States 🇺🇸
Chapel Hill Part Time Team Member
Chick-fil-A United States 🇺🇸
Hospital Housekeeper (Gracewood, GA)
The Employee Connect Georgia, United States🇺🇸
Sports Medicine Physician (Norfolk, VA)
The Employee Connect Norfolk, Virginia, United States🇺🇸
Healthcare Contract Paralegal
Veritehealthcollective Remote · Ohio, United States🇺🇸
Project Manager (Construction/MEP)
Titanlink Recruiting LLC New York City, New York, United States🇺🇸
Product Manager, Two-Sided Marketplace
Titanlink Recruiting LLC New York City, New York, United States🇺🇸
Patient Financial Representative Senior,
Sigma Systems, Inc. Irving, Texas, United States🇺🇸
Sr. Data Entry Operator
Sigma Systems, Inc. Boston, Massachusetts, United States🇺🇸
Accounting Manager
Beacon Communications Littleton, Colorado, United States🇺🇸
Injury Claims Settlement Negotiator - Seattle, WA
Sivad Consulting Seattle, Washington, United States🇺🇸