For our client from pharmaceutical industry we are looking for an AI Test Engineer specializing in testing Generative (and non-Generative) AI applications. The ideal candidate will have expertise in both traditional software testing (test automation) and specialized testing methodologies for Large Language Models (LLMs)-based and AI-based systems. This role requires deep understanding of LLM behaviors, evaluation metrics, and testing frameworks to ensure the reliability, fairness, and effectiveness of our AI-powered solutions.
For our client from pharmaceutical industry we are looking for an AI Test Engineer specializing in testing Generative (and non-Generative) AI applications. The ideal candidate will have expertise in both traditional software testing (test automation) and specialized testing methodologies for Large Language Models (LLMs)-based and AI-based systems. This role requires deep understanding of LLM behaviors, evaluation metrics, and testing frameworks to ensure the reliability, fairness, and effectiveness of our AI-powered solutions.
,[Generative AI Testing Strategy: Design and implement comprehensive testing strategies specifically tailored for LLM-based applications, including evaluation of model responses, RAG pipelines accuracy, and overall system reliability, Quality Assurance Framework Development: Utilize client''s testing frameworks that address both traditional software quality aspects and AI-specific concerns such as output consistency, contextual accuracy, and ethical compliance; co-create and maintain such frameworks when required, Test Automation Development: Design and implement automated testing solutions for continuous evaluation of LLM applications, including integration tests, performance tests, and specialized AI behavior tests, Test Documentation: Create and maintain comprehensive test documentation testing strategy, test cases, and testing guidelines specific to AI applications and compliant with Roche practices; document the analysis of requirements and risks and develop tests accordingly, Performance Testing: Collaborate with other engineers to conduct thorough performance testing of GenAI applications, including response time analysis, load testing, and resource utilization monitoring] Requirements: Python, AI, Git, Jira, pytest, GenAI, LLM, CI/CD Pipelines, AWS, Azure Additionally: Sport subscription, Private healthcare, International projects.