Quality Assurance Engineer

Job Description Roles & Responsibilities Key Responsibilities 1. AI & LLM Validation Non-Deterministic Testing: Architect automated frameworks to evaluate Generative AI outputs for hallucination, consistency, and factual accuracy against "Gold Standard" datasets. RAG Evaluation: Implement automated metrics (e.g., RAGAS, faithfulness, answer relevance) to verify that Retrieval-Augmented Generation pipelines accurately cite technical and regulatory documentation. Prompt Regression: Design regression suites to monitor "prompt drift," ensuring model updates do not degrade the quality of AI-generated engineering documents. 2. Integration & System Verification Enterprise Integration: Build robust tests to validate data consistency between AI agents and critical systems (e.g., SAP S/4HANA, Ariba), ensuring the integrity of Bill of Materials (BOM) and financial data. Performance Benchmarking: Design tests to validate latency and throughput for forecasting models and risk-scoring engines using tools like Locust, JMeter, or K6. API & Security Validation: Automate testing of secure API gateways, verifying Role-Based Access Control (RBAC) and PII redaction logic before data reaches AI models. 3. Governance & Traceability V-Model Alignment: Map automated test cases to "System Requirements" to create digital evidence for formal Verification and Validation (V&V) reports. Stage Gate Compliance: Prepare "Test Readiness" packages for formal reviews, providing quantitative evidence that systems are stable enough to move from MVP to Production. Defect Lifecycle Management: Manage the feedback loop between Requirements Quality Assistants and development teams, tracing AI logic defects back to specific model versions. What You ll Bring Technical Requirements Core Automation: Expert proficiency in Python (Pytest) and standard libraries (Selenium/Playwright, Requests). AI Evaluation: Hands-on experience with LLM evaluation frameworks (e.g., DeepEval, TruLens) and "Ground Truth" dataset management. Performance Engineering: Proficiency in crafting Performance Test Plans and implementations (Locust, K6, etc.). Data Validation: Expertise in SQL and data quality tools (e.g., Great Expectations) for Data Lakehouses and Vector Databases. CI/CD & DevOps: Strong experience integrating quality gates into GitLab CI/CD pipelines. Engineering Practices: Deep understanding of modern QE practices, including Shift Left, Test Pyramid, and Mono-repo architectures. Professional Qualifications Experience: 5+ years in QA Automation, with 2+ years focused on complex data-driven applications, ML models, or AI agents. Domain Expertise: Background in Defense, Aerospace, or highly regulated industries is a strong plus. Familiarity with IV&V processes is highly desirable. Analytical Mindset: Ability to define pass/fail criteria for probabilistic systems and communicate "Confidence Levels" to engineering leadership. Why AI71? Final Line of Defense: You define whether an AI agent is "trusted" to negotiate a contract or design a critical system component in a safety-critical environment. Tax-Free Compensation: A market-leading, 100% tax-free salary and benefits package. Full Relocation Support: Comprehensive assistance including flights, housing support, and visa sponsorship for you and your family to Abu Dhabi. Pioneering Work: Set the global standard for how defense organizations validate intelligent systems using world-leading models. Company Industry IT - Software Services Department / Functional Area Engineering Keywords Quality Assurance Engineer Get real-time job updates only on our App