Together AI Introduces Together Compiler: Agent-Driven System Accelerates Inference Performance Up to 2.6x
Together AI unveiled Together Compiler, an agent-powered optimization system that continuously re-tunes model inference as production traffic patterns shift. The system employs coding agents to rewrite model forward passes and uses a persistent engine with hot reload capabilities
Together AI unveiled Together Compiler, an agent-powered optimization system that continuously re-tunes model inference as production traffic patterns shift. The system employs coding agents to rewrite model forward passes and uses a persistent engine with hot reload capabilities, achieving up to 2.6x higher throughput compared to published InferenceX configurations within 24 hours. Testing across multiple models including Kimi K3, GLM 5.3, and DeepSeek v4.1 Flash demonstrated significant performance gains, with the system running on NVIDIA and AMD hardware. The approach prioritizes naive agent strategies over expert-curated knowledge bases and measures real traffic patterns rather than random data to ensure practical optimization gains.