Role: AI QE engineering
Location: Chicago, IL or Irving, TX (Onsite)
Experience: 10+ Years
Engagement: Contract
Job Description:
We are looking for an experienced Sr. AI Quality & Reliability Engineer to drive the design, development, and execution of AI Quality Engineering initiatives supporting enterprise AI transformation. The candidate will be responsible for AI validation, AI-assisted testing, runtime quality controls, and scalable testing frameworks to ensure the reliability and quality of AI-powered applications.
Key Responsibilities:
AI Quality Engineering & Technical Execution
Design and execute quality engineering activities for AI-powered applications, LLM-enabled workflows, intelligent automation solutions, agentic systems, and enterprise AI platforms.
Develop AI-native testing approaches, validation processes, runtime quality controls, reusable testing accelerators, and scalable testing workflows.
Modernize traditional QA practices to support AI-enabled applications and healthcare workflows.
Establish AI quality standards, validation approaches, release readiness processes, and governance-aligned testing practices.
Manage AI validation activities, defect management, test execution, release support, and quality improvement initiatives.
AI Validation, Testing & Runtime Assurance
Perform functional testing, prompt testing, workflow testing, regression testing, and runtime quality assurance.
Collaborate with AI Engineering, AIOps, LLMOps, Security, Governance, Clinical, and Data teams.
Support runtime monitoring, distributed tracing, observability, incident management, and production reliability.
Develop AI evaluation frameworks, validation datasets, quality scoring approaches, and automated testing workflows.
Support drift detection, human-in-the-loop validation, and AI monitoring initiatives.
Drive AI-assisted testing, intelligent automation, and reusable test automation frameworks.
Validate healthcare workflows, payer operations, and AI-enabled business processes.
Delivery Coordination & Operational Support
Coordinate testing activities, sprint execution, issue tracking, risk identification, and release support.
Evaluate testing readiness, implementation dependencies, and quality risks.
Support tooling evaluations, automation initiatives, and testing modernization.
Establish scalable testing processes and reusable quality engineering frameworks.
Collaboration, Mentorship & Continuous Improvement
Mentor quality engineers and analysts and provide technical guidance to delivery teams.
Communicate testing risks, implementation issues, and recommendations to technical and business stakeholders.
Promote engineering best practices, continuous improvement, and responsible AI adoption.
Evaluate emerging AI testing, observability, automation, and runtime assurance technologies.
Required Qualifications:
10+ years of overall IT experience.
5+ years of experience in Quality Engineering, Quality Assurance, Software Testing, or Enterprise Application Delivery.
2+ years of technical leadership experience in testing initiatives, automation, or enterprise delivery projects.
Strong experience in Quality Engineering / QA across enterprise platforms, APIs, operational workflows, healthcare applications, or integrated business systems.
Experience with test automation frameworks, SDLC, Agile methodologies, defect management, regression testing, release validation, and production support.
Understanding of AI/LLM validation, prompt testing, AI-assisted testing, and runtime quality practices.
Strong analytical, communication, collaboration, and leadership skills.
Preferred Skills:
GenAI and LLM Testing.
AI Model Evaluation and Validation.
Prompt Testing and Engineering.
Agentic AI Testing.
AI Observability and Runtime Monitoring.
AIOps / LLMOps.
Healthcare / Payer Domain Experience.
Test Automation and API Testing.
