Overview
Overview Manages and evolves the enterprise data lake and data warehouse while ensuring the reliable, secure, and efficient flow of high-quality data. Implements data processes, managing data architecture, designing ETL processes, and analyzing data for business insights. The base salary range for this position is $99,937-$157,044. Actual pay will be determined based upon a candidate’s job-related knowledge, skills, education, experience, geographic location, and may include other job-related factors such as certification(s), professional licensure, or internal equity considerations. Responsibilities Design, implement and maintain scalable data pipnes on WS using S3, DMS, Glue, lambda, step function/MWAA & Redshift. Develop robust batch and near-real-time ETL/ELT workflow to ingest, cleanse, transform and load data from databases, legacy applications and event streams using Python & Pysp
Full job description
Full Job Description
Overview
Manages and evolves the enterprise data lake and data warehouse while ensuring the reliable, secure, and efficient flow of high-quality data. Implements data processes, managing data architecture, designing ETL processes, and analyzing data for business insights.
The base salary range for this position is $99,937-$157,044.
Actual pay will be determined based upon a candidate’s job-related knowledge, skills, education, experience, geographic location, and may include other job-related factors such as certification(s), professional licensure, or internal equity considerations.
Responsibilities
-
Design, implement and maintain scalable data pipnes on WS using S3, DMS, Glue, lambda, step function/MWAA & Redshift.
-
Develop robust batch and near-real-time ETL/ELT workflow to ingest, cleanse, transform and load data from databases, legacy applications and event streams using Python & Pyspark.
-
Design incremental/CDC mechanism, including restart ability, idempotency, duplicate handling and recovery.
-
Implement automated controls for completeness, accuracy, reconciliation, schema changes & lineage.
-
Optimize Glue/Spark, Athena, Redshift & S3 workload through partitioning, columnar formats, query tuning and appropriate storage/compute design.
-
Design near real time/event-driven pipelines using Kinesis/Kafka where required, covering ordering, retry, idempotency and failure recovery.
-
Implement AWS data security, least privilege access, data classification and governance controls.
-
Monitor pipelines such as CloudWatch, troubleshoot failure and resolving production data incidents.
-
Enforce Git/version control, code review, automated testing and CI/CD practices.
-
Work with product owners, architect, reporting and business stakeholders to translate requirements into scalable data solutions.
-
Document pipelines and operational procedures.
Qualifications
Qualifications Required
-
Bachelor's degree in a computer-related field from an accredited college or university and five (5) or more years of experience in data engineering, building scalable and distributed ETL data pipelines in enterprise environments.
-
Experience building and operating scalable AWS-based data platforms and pipelines using services including Lambda, Glue, Athena, S3, Redshift, DMS, MWAA (Airflow), and Step Functions, supporting batch, CDC, and near real-time data processing.
-
Advanced proficiency in Python, SQL, and PySpark with hands-on experience developing reusable ETL/ELT frameworks, data warehouses, data marts, and integrations across databases, APIs, event streams, and analytics environments.
-
Experience implementing data quality, governance, and optimization best practices, including automated validation frameworks, Lake Formation and Glue Data Catalog, performance tuning, and cost optimization across AWS data services.
-
Strong communication skills with the ability to translate complex data concepts for business stakeholders; experience in healthcare, life sciences, and other highly regulated environments with HIPAA, GDPR, FDA, or similar compliance requirements preferred.
-
Experience with metadata management, data lineage, data observability, master data management, or enterprise data catalog solutions.
-
Knowledge with data modeling & analytical data model, schema design, schema evolution, and data structure optimized for reporting and analytics.
-
Knowledge of data lake and data warehouse architecture include data partitioning and columnar storage format such as Parquet.
-
Relevant AWS certification, such as AWS Certified Data Engineer – Associate, or an equivalent cloud or data engineering certification.
WORKING CONDITIONS
-
Flexible work hours in fun collaborative environment
-
Working remote requires a reliable internet connection
-
Must have the ability to travel, as needed for company meetings
Tips for this job
Practical Job and Scholarship guidance. These tips do not replace official rules or create new eligibility requirements.
- Tailor the CV and application to the responsibilities and required skills stated on the official employer page.
- Use concrete evidence of relevant work, projects and measurable results rather than generic claims.
- Confirm location, work authorization, remote restrictions and sponsorship terms before applying.
- Apply through the original employer or official recruitment destination shown on this page.
Verification notes
Verified from public schema.org JobPosting structured data on the official source page. The complete published description, responsibilities, requirements and benefits were normalized when present; unstated facts were not inferred.
Job and Scholarship is the discovery and verification layer. Confirm eligibility, dates, salary/funding and application instructions on the original source before submitting anything.
Arc Onesolutions Careers ↗Browse current Job and Scholarship listings from Arc Onesolutions Careers →
