Big Data/Hadoop/Splunk Tester -- KUMDC

Job Description

<p><b>Job Summary</b></p><p>We are seeking a Data QA / Validation Engineer (Contractor) to ensure data quality across Databricks/Spark/Delta Lake pipelines.</p><p>The role focuses on validating clean, accurate datasets that power downstream analytics, machine learning, and real-time decision-making systems.</p><p>You will design and implement automated data quality tests for ingestion and transformation layers (Bronze ? Silver ? Gold), enforce schema and data contracts, detect drift, validate feature pipelines, and monitor data anomalies.</p><p><br></p><p><b>Key Responsibilities</b></p><ul><li>Develop and execute automated data quality tests for ingestion and transformation layers.</li><li>Enforce schema and data contracts across pipelines.</li><li>Detect and address data drift and anomalies.</li><li>Validate feature pipelines to ensure accuracy and reliability.</li><li>Monitor data quality and maintain issue logs for critical pipelines.</li><li>Deliver production-grade validation suites and provide QA sign-off for key data processes.</li></ul><p><br></p><p><b>Required Qualifications</b></p><ul><li>8+ years of experience in SQL.</li><li>5+ years of experience with Databricks and Spark.</li><li>3+ years of experience in data quality frameworks (e.g., Great Expectations, Deequ, Soda, dbt tests).</li><li>3+ years of experience using GenAI tools (Cursor, Windsurf, GitHub Copilot, Databricks Assistant) for test generation and validation workflows.</li><li>5+ years of experience delivering production-grade validation suites and QA sign-off for critical pipelines.</li></ul>