Tech Lead Data - Databricks

This position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for a Tech Lead Data - Databricks based in Brazil. We are seeking an experienced, hands-on Data Tech Lead to drive data engineering initiatives using Databricks on Azure. In this role, you will shape technical standards, guide architectural decisions, and help engineering teams deliver scalable, secure, and reliable data solutions. Working closely with the Data Architect, you will establish best practices across Lakehouse development and the Medallion architecture. You will play a key role in optimizing data pipelines, strengthening DataOps practices, and ensuring performance, governance, and quality. This opportunity combines technical leadership with practical engineering in a collaborative, delivery-focused environment. You will also mentor engineers and help build reusable standards that accelerate delivery and create lasting business impact.

Provide technical leadership to Data Engineering teams, supporting solution design, code reviews, technical decisions, and the resolution of delivery blockers. Collaborate with the Data Architect to define the technical vision, coding standards, data modeling conventions, and best practices for the Lakehouse environment using Delta Lake and the Medallion architecture. Design, develop, and oversee data pipelines in Databricks using PySpark and Spark SQL across Bronze, Silver, and Gold layers, ensuring scalability, reliability, security, and governance. Establish and reinforce DataOps practices, including Git version control, CI/CD, automated unit and integration testing, data quality validation, and technical documentation. Optimize pipeline performance, including partitioning strategies, Spark tuning, cloud cost management, and operational reliability through monitoring, observability, and alerting. Ensure non-functional requirements are met, particularly regarding access control, security, and data policies across Azure and Databricks. Partner with business and analytics stakeholders to translate requirements into scalable technical solutions and realistic delivery plans. Mentor engineers, promote engineering excellence, and develop reusable templates, playbooks, and technical references to improve team consistency and productivity. Requirements: Strong experience in Data Engineering projects, including previous experience providing technical and operational leadership as a Tech Lead or Senior Engineer. Extensive hands-on experience with Databricks, PySpark, and Spark SQL, particularly in designing and implementing batch data pipelines, with working knowledge of streaming when required. Deep knowledge of Delta Lake and practical experience implementing the Medallion architecture across Bronze, Silver, and Gold layers. Solid experience with Azure data services, including Azure Databricks, Azure Data Lake Storage (ADLS), and integration with identity and security services such as Azure Active Directory. Strong SQL and Python skills, along with a solid understanding of dimensional data modeling, consumption-oriented models, and schema and data contract best practices. Experience implementing DataOps practices, including Git, CI/CD for data pipelines, automated testing, and clear technical documentation. Proven ability to review architecture and code, troubleshoot performance and scalability issues, and guide engineering teams toward effective solutions. Advanced English proficiency for collaboration with stakeholders and the preparation of technical documentation. Nice to have: Experience with Unity Catalog, Databricks Workflows and Repos, Delta Live Tables, data quality frameworks such as Great Expectations or Soda, and dbt on the Lakehouse. Additional advantages: Familiarity with orchestration tools such as Airflow, observability practices involving metrics, logs, and alerts, and change data capture or streaming technologies such as Event Hubs and Kafka. Consulting experience, exposure to international clients, and Databricks or Azure certifications are also valued. Benefits: Health and dental insurance. Meal and food allowances. Childcare assistance. Extended paternity leave. Access to gym memberships and health and wellness professionals through Wellhub (Gympass) and TotalPass partnerships. Profit Sharing and Results Participation (PLR). Life insurance. Access to a continuous learning platform and professional development resources. Discount club with partner offers. Free online resources dedicated to physical, mental, and overall well-being. Pregnancy and responsible parenting courses. Partnerships with online learning platforms. Access to a language-learning platform. Additional benefits and well-being initiatives.