Posted:
Job Description
Job Summary
We are seeking a Data QA / Validation Engineer (Contractor) to ensure data quality across Databricks/Spark/Delta Lake pipelines.
The role focuses on validating clean, accurate datasets that power downstream analytics, machine learning, and real-time decision-making systems.
You will design and implement automated data quality tests for ingestion and transformation layers (Bronze ? Silver ? Gold), enforce schema and data contracts, detect drift, validate feature pipelines, and monitor data anomalies.
Key Responsibilities
- Develop and execute automated data quality tests for ingestion and transformation layers.
- Enforce schema and data contracts across pipelines.
- Detect and address data drift and anomalies.
- Validate feature pipelines to ensure accuracy and reliability.
- Monitor data quality and maintain issue logs for critical pipelines.
- Deliver production-grade validation suites and provide QA sign-off for key data processes.
Required Qualifications
- 8+ years of experience in SQL.
- 5+ years of experience with Databricks and Spark.
- 3+ years of experience in data quality frameworks (e.g., Great Expectations, Deequ, Soda, dbt tests).
- 3+ years of experience using GenAI tools (Cursor, Windsurf, GitHub Copilot, Databricks Assistant) for test generation and validation workflows.
- 5+ years of experience delivering production-grade validation suites and QA sign-off for critical pipelines.