Skill needed: Python, Machine Learning, LLMs.
Employer: Dynatrace
Dynatrace makes it easy and simple to monitor and run the most complex, hyper-scale multicloudsystems. Dynatrace is a full stack and completely automated monitoring solution that can trackevery user, every transaction, across every application.Our team is looking for a Lead Data Scientist specialized in Large Language Models (LLMs) to design, build, and scale generative AI capabilities for real-world, enterprise-grade use cases. In this hands-on technical leadership role, you’ll own the end-to-end LLM stack, from data/knowledge, Ingestion and retrieval to prompt and tool-use architecture, evaluation frameworks,safety/guardrails, and cost/latency optimization.Your Tasks• Own the LLM system architecture: Retrieval pipelines, prompt/tool design,routing/fallbacks, safety layers, and telemetry, optimized for quality, latency, and cost.• Establish technical standards for RAG: content ingestion, chunking/windowing, hybridretrieval, reranking, query understanding, and structured output contracts.• Define evaluation strategy: Create a rigorous eval suite covering answer correctness,attribution/grounding, toxicity/safety, privacy leakage, determinism, latency, and cost.• Formalize LLMOps: Versioning for prompts/datasets/models, experiment governance,prompt and dataset registries, and promotion criteria from dev - staging - prod.• Drive tool/agent design: API schema design for function calling, error handling, recoverystrategies, self-correction, and guardrail integration.• Make build-vs-buy calls: Weigh managed providers vs. open-source/self-hosted,considering performance, cost, IP, privacy, and compliance.• Mentoring: Provide deep technical mentorship on prompting, retrieval design, evals, andsafe deployment; lead reviews of prompts, pipelines, and evaluation reports.Hands-on Data Science• Implement end-to-end RAG systems: ingestion - chunking - embeddings - hybrid search -rerank - prompt assembly - tool calls - post-processing.• Engineer robust prompts/tools: reusable templates, multi-turn strategies, structuredoutputs via JSON Schema/Pydantic.• Select/tune models: foundation models, embeddings, rerankers; apply LoRA/PEFT ordistillation when justified.• Build eval corpora: golden sets, KPIs for accuracy, groundedness, deflection, toolsuccess.• Implement guardrails: PII/PHI detection, policy prompts, jailbreak resistance, filters,safety scorecards.• Productionize: ship resilient services with analytics, alerts (drift, quality, cost), SLOs, etc.• Optimize for scale: token, latency, cost; caching, context packing, batching, speculativedecoding, routing by intentWhat will help you succeedMinimum requirements:• Advanced CS/AI/ML degree or equivalent, strong ML background.• 7+ years DS/ML, 3+ years NLP /LLMs, shipped production systems.• Python and core ML stack: 5+ years of professional Python.• Data engineering for unstructured data (3+ years): text processing, parsing, embedding-friendly preprocessing.• Proven RAG expertise (1+ years): embeddings, retrieval, reranking, chunking.• Evaluation depth (1+ years): offline/online evals for accuracy, grounding, safety.• Safety/privacy (1+ years): moderation, PII/PHI redaction, policy enforcement.• LLMOps (1+ years): prompt/version management, experiment tracking, monitoring.• Excellent communication: explain trade-offs, drive data decisions.Desirable experiance:• Serving/scaling: vLLM/TGI, Ray Serve, Triton; GPU/CPU trade-offs.• Tuning/distillation: LoRA/PEFT, safety alignment, synthetic data.• Domain: observability, support systems, multilingual, regulated environments.• Cloud/security: Snowflake/AWS, managed vs self-hosted.• Experience with graph-based knowledge bases (e.g., GraphDB, Neo4j) and knowledgegraphs to complement RAG systems with entity modeling and relationship-aware retrieval.Why you will love being a Dynatracer• Working models that offer you the flexibility you need, ranging from full remote options tohybrid ones combining home and in-office work• A team that thinks outside the box, welcomes unconventional ideas, and pushesboundaries• An environment that fosters innovation enables creative collaboration and allows you togrow• A globally unique and tailor-made career development program recognizing yourpotential, promoting your strengths, and supporting you in achieving your career goals• A truly international mindset with Dynatracers from different countries and cultures allover the world, and English as the corporate language that connects us all• A culture that is being shaped by our global team’s diverse personalities, expertise, andbackgrounds• A relocation team that is eager to help you start your journey to a new country, alwaysthere to support and by your side. If you need to relocate for a position you’re applying for,we offer you a relocation allowance and support with your visa, work permit,accommodation