AI QE Engineer

Job Description

<p>We are a digital-first technology services firm backed by Goldman Sachs Asset Management and Everstone Capital. We specialize in accelerating business transformation and delivering human-centric digital experiences. For over 28 years, Apexon has been meeting customers wherever they are in the digital lifecycle and helping them outperform their competition through speed and innovation.</p><p><br></p><p>AI QE Engineer - Job Description:</p><p>Role Summary We are looking for a AI QA Engineer with strong automation expertise and hands-on experience validating LLMs, GenAI workflows, and AI-driven applications.</p><p>This role involves building automated test suites, validating AI outputs, ensuring system reliability, and contributing to the quality strategy for next-generation AI features</p><p><br></p><p>Core Experience</p><p>• 4-10 years as a Software Engineer, SDET, or Automation Engineer.</p><p>• Strong coding skills in Python, TypeScript, or Java.</p><p>• Hands-on experience developing automation scripts, tools, or frameworks.</p><p>• Practical experience using LLMs, prompt engineering, and evaluating AI generated outputs.</p><p>• Familiarity with agentic AI systems and exposure to tools like LangGraph, AutoGen, or CrewAI.</p><p>• Basic understanding of Model Context Protocol (MCP) or context-aware workflow automation (nice to have). </p><p>AI / ML Technologies</p><p>• Practical experience with AI/ML frameworks: LangChain, Hugging Face, GPT models, vector databases, RAG pipelines.</p><p>• Experience with ML/DL libraries such as: Scikit-learn, PyTorch, TensorFlow, Keras, Transformers, OpenCV.</p><p>• Ability to work with embeddings, similarity search, and content evaluation metrics. GenAI & AI Agent Development</p><p>• Ability to integrate or build GenAI components, including RAG pipelines or agent based workflows.</p><p>• Support model evaluation tasks such as: Output quality checks, Hallucination detection, Prompt validation, Regression checks for model updates</p><p><br></p><p>Automation & Quality Engineering</p><p>• Experience building automation using Python, PyTest, Selenium, Playwright, or API testing libraries.</p><p>• Ability to design and execute automated tests for: Functional Integration, API ,Basic performance & reliability testing</p><p>• Hands-on experience testing RESTful APIs and building automated API suites. Cloud, DevOps & CI/CD</p><p>• Exposure to deploying AI or automation solutions on AWS.</p><p>• Working knowledge of CI/CD pipelines, including: Automated testing ,Model validation steps Versioning & artifact management SDLC & Collaboration</p><p>• Strong understanding of the software development lifecycle, including requirements, development, testing, and defect analysis.</p><p>• Ability to collaborate with Developers, Data Scientist, and QE teams, clearly communicating progress, risks, and results.</p><p>• Skilled in documenting and tracking defects and participating in defect triage.</p>