Applied Scientist / ML Engineer

GOSST T&L - AI/ML Role for LLM Accuracy & Model DevelopmentJob Description GOSST Turnover & Launch is looking for a hands-on ML/AI engineer (contractor, L5 equivalent) to join our team and drive LLM accuracy improvements across our product portfolio. This role sits at the intersection of applied machine learning and operational technology - you will fine-tune models, build evaluation frameworks, and improve AI-driven automation for our Turnover & Launch tools (OrderPad, UTP, Field Installation, and supporting agentic systems).

Key Responsibilities

• Fine-tune and optimize Large Language Models (LLMs) for domain-specific tasks including code estimation, sprint planning, and operational automation.• Design and implement evaluation frameworks to measure model quality — including automated metrics, LLM-asjudge, and human evaluation workflows.• Build and maintain RAG (Retrieval-Augmented Generation) pipelines that ground model outputs in codebase and operational data.• Develop prompt engineering strategies and optimize prompt chains for multi-step agentic workflows.• Collaborate with SDEs and SDMs to integrate model improvements into production services (Amazon Bedrock, SageMaker, or equivalent).• Analyze model failure modes, identify accuracy gaps, and propose targeted improvements.• Establish baseline metrics and track accuracy improvements over time with dashboards and reporting.• Stay current on LLM advancements and recommend adoption of new techniques (LoRA, DPO, constitutional AI, etc.) where applicable.