Data Engineering Technical Supervisor (Medior) – Databricks & Python

Work Schedule

Standard (Mon-Fri)

Environmental Conditions

Office

Job Description

Thermo Fisher Scientific 

Data Engineering Technical Supervisor

Position Summary 

We are seeking a Data Engineering Technical Supervisor to provide hands-on technical leadership and day-to-day guidance to a team of data engineers building scalable, reliable, and high-quality data solutions. This role does not include people management responsibilities. 

The role combines hands-on data engineering, technical supervision, mentoring, and project leadership, with a strong focus on Databricks, Python/PySpark, SQL, Delta Lake, and modern data engineering practices. 

As part of the Data & Analytics team, the Technical Supervisor will collaborate with data engineers, BI developers, analysts, product owners, and business stakeholders while supporting the reliability, maintainability, and operational continuity of production ETL/ELT processes in Databricks. 

Key Responsibilities 

• Provide technical leadership and mentoring to data engineers and technical oversight for Databricks solutions. 

• Lead and contribute hands-on to the design and implementation of scalable ETL/ELT pipelines using Databricks, Python/PySpark, SQL, and Delta Lake. 

• Guide engineers on architecture, design patterns, coding standards, testing, and performance optimization. 

• Review technical designs and code to ensure solutions are scalable, maintainable, secure, and aligned with engineering standards. 

• Troubleshoot complex production issues, support root-cause analysis, reduce technical debt, and improve platform reliability. 

• Promote engineering best practices including Git-based source control, automated testing, CI/CD, peer review, and documentation. 

• Develop reusable Python/PySpark components and standardized patterns for data ingestion, transformation, validation, and processing. 

• Work with Delta Lake, Unity Catalog, and lakehouse architecture patterns, and support Databricks workflow orchestration and automation. 

• Implement appropriate data quality, monitoring, observability, and error-handling mechanisms. 

• Enable BI developers to build ETL/ELT pipelines within governed engineering patterns, including approved templates, standards, guardrails, and code reviews. 

• Facilitate technical discussions, design reviews, knowledge sharing, and technical onboarding. 

• Foster a collaborative engineering culture focused on ownership, continuous improvement, and high-quality delivery.

Delivery & Stakeholder Collaboration 

• Partner with BI developers, analysts, product owners, and business stakeholders to translate data requirements into scalable technical solutions. 

• Provide technical estimates, identify dependencies and risks, and support prioritization and delivery of engineering initiatives. 

• Communicate technical concepts, trade-offs, risks, and recommendations clearly to technical and non-technical stakeholders. 

• Ensure solutions meet agreed functional, performance, security, quality, and operational requirements. 

Required Qualifications 

• Bachelor's degree in Computer Science, Data Engineering, Software Engineering, Information Technology, or a related discipline, or equivalent professional experience. 

• 4+ years of professional experience in data engineering, software engineering, or a related technical field. 

• Strong hands-on experience with Python/PySpark, Databricks, Apache Spark, and SQL in production data engineering. 

• Experience designing and operating production-grade ETL/ELT pipelines using Delta Lake and lakehouse architecture concepts. 

• Solid understanding of data modeling, data quality, partitioning, performance optimization, and distributed data processing. 

• Experience with Git-based development, automated testing, CI/CD, and code review. 

• Demonstrated ability to troubleshoot complex technical issues and provide constructive engineering guidance. 

• Experience mentoring engineers or providing technical leadership. 

• Strong analytical, problem-solving, communication, and stakeholder-management skills. 

Preferred Qualifications 

• Experience with Databricks on AWS/GCP/Azure

• Experience with Unity Catalog and Databricks governance capabilities. 

• Experience with data observability, monitoring, and pipeline automation. 

• Experience optimizing Spark/Databricks workloads for performance and cost.