Principal Data Engineer

Full Job Description

Lead data-platform architecture: Design and evolve ETL workflows, data pipelines, and data models using Spark, Python, and SQL. Drive hands-on, end-to-end implementation: Build data solutions using Spark (or similar distributed systems), open table formats such as Delta Lake, and Parquet files; develop scalable, secure APIs and data access controls, including RBAC; provision resources through Infrastructure as Code; and deliver production-grade deployments through CI/CD with observability and operational runbooks. Think beyond immediate requirements: Own complex system-design decisions, evaluate alternatives, and account for cross-team dependencies, scalability, resiliency, disaster recovery, and cost. Make evidence-based decisions: Proactively gather facts, validate assumptions, test hypotheses, and use telemetry and investigation to resolve complex problems rather than relying on unverified assumptions. Advance AI-native engineering: Apply AI across design, coding, testing, and troubleshooting; validate AI-generated outputs and help the team adopt effective, responsible practices. Own production outcomes: Establish monitoring and operational readiness, drive incident resolution and root-cause analysis, and implement preventive improvements. Automate safe deployments, targeting zero-touch deployment where possible. Raise engineering quality: Lead design and code reviews, strengthen testing, and embed security, privacy, and compliance into solutions throughout their lifecycle. Collaborate and mentor: Communicate trade-offs clearly, align partner teams on dependencies and ownership, mentor engineers, and share knowledge to improve team capabilities. Bachelor's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python Master's Degree in Computer Science or related technical field AND 12+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR Bachelor's Degree in Computer Science or related technical field AND 15+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience. Experience building data platforms for experimentation, metrics, and scorecards, including understanding success metrics, guardrail metrics, and data quality. Experience in Databricks like platforms.