Together AI has published a blog post outlining research focused on improving the efficiency, reliability, and scalability of AI inference. The post discusses the challenges AI-native teams face as they transition from building models to deploying them in production environments.
Why it matters: Improving inference efficiency can help AI-native teams deploy models more effectively at scale.
Together AI compared Kimi K2.7 Code and Claude Fable 5 by generating 12 landing pages. Kimi K2.7 Code cost 94% less and achieved scores within a few points of Claude Fable 5 on every page. The blog discusses the factors that influenced these results.
Why it matters: This comparison demonstrates a substantial cost advantage for Kimi K2.7 Code while maintaining similar quality, which could impact developer tool selection.
Together AI released ParallelKernelBench, a benchmark that tests LLMs on writing fast multi-GPU CUDA kernels across 87 real workloads. The best-performing model solves under a third of the tasks, though some generated kernels outperform any public implementation.
Why it matters: This benchmark highlights both the current limitations and emerging potential of LLMs in high-performance computing code generation.
Together AI has developed what it claims is the world’s fastest speech-to-text stack, according to benchmarks by Artificial Analysis. The company attributes this achievement to optimizing the entire system path for automatic speech recognition, rather than focusing solely on GPU inference.
Why it matters: Faster speech-to-text systems could reduce latency in real-time transcription and voice applications, improving the practicality of AI-powered speech recognition.
Together AI has released Violin, an open-source AI video translation tool that integrates speech recognition, LLM translation, and text-to-speech. The tool aims to make video content accessible across languages.
Why it matters: Violin democratizes video translation by providing an open-source alternative to proprietary services, potentially lowering barriers for content creators and educators worldwide.
Together AI has introduced Provisioned Throughput, a service that offers reserved inference capacity for open models such as MiniMax M3 and GLM-5.2. The offering features token-based pricing, a 99% uptime SLA, and claims up to 90% lower costs compared to proprietary APIs, while removing the need for GPU-hour calculations and infrastructure management.
Why it matters: This gives developers a predictable and cost-effective way to run open models at scale without managing infrastructure.
Together AI has partnered with Pearl Research Labs to launch a discounted inference endpoint for the Gemma-4-31B-it-pearl model. The service leverages a 'Proof of Useful Work' mechanism, converting AI workloads into cryptocurrency emissions to help reduce inference costs.
Why it matters: This partnership introduces a novel economic model that could lower the cost of running large language models by integrating cryptocurrency incentives.
Together AI has introduced Voice Finder, a tool that enables developers to search, filter, and audition over 600 voices using natural-language prompts or uploaded audio samples. The tool supports multiple Together AI TTS models and is designed to simplify the process of selecting synthetic voices for applications.
Why it matters: This tool streamlines the process of finding the right synthetic voice, reducing development time for voice-enabled apps.
Together AI introduces Parcae, a stable looped language model that matches the quality of a Transformer twice its size, with a 770M model achieving 1.3B-level performance. The company also presents the first scaling laws for looping, showing that increasing recurrence is a compute-efficient way to improve performance.
Why it matters: This approach could enable more efficient AI models that deliver higher performance with fewer parameters, potentially reducing computational costs and energy consumption.
Together AI has launched EinsteinArena, a platform where AI agents collaborate and compete to solve open math problems. The agents have already achieved 11 new state-of-the-art results, including raising the kissing number lower bound in dimension 11 from 593 to 604.
Why it matters: This shows how collective AI agent intelligence can contribute to scientific progress in mathematics.
Together AI defines an AI Native Cloud as infrastructure designed specifically for AI models rather than traditional legacy workloads. The company positions this concept as important for the next major shift in AI development.
Why it matters: This highlights the increasing emphasis on specialized cloud infrastructure tailored for AI, which could influence how AI models are deployed and scaled.
Together AI research shows that large language models (LLMs) can optimize database query execution plans by correcting cardinality estimation errors, resulting in up to 4.78x speedups. This method leverages LLMs' understanding of query semantics and outperforms traditional statistical heuristics.
Why it matters: This approach could lead to significant improvements in database performance by using AI-driven optimization instead of conventional statistical methods.
Together AI has released the Wan 2.7 video model suite, featuring four models designed for video generation, continuation, reference-driven workflows, and editing. The rollout begins with text-to-video capabilities.
Why it matters: This release broadens the range of accessible video AI tools for developers, supporting multiple workflows on a single platform.
Together AI has integrated Deepgram's production-grade speech-to-text and text-to-speech models into its Dedicated Model Inference platform. This allows developers to build real-time voice agents using Deepgram's Nova-2 and other voice models on Together AI's infrastructure.
Why it matters: The integration streamlines the development of real-time voice AI agents by combining advanced speech models with scalable inference infrastructure.
Together AI's kernel research team develops high-performance GPU kernels such as FlashAttention and ThunderKittens to bridge the gap between hardware and production AI. The team focuses on optimizing low-level operations to improve efficiency and speed for AI workloads.
Why it matters: This work directly impacts the performance and cost of running large-scale AI models by making GPU utilization more efficient.
Together AI has introduced Aurora, an open-source reinforcement learning framework that transforms speculative decoding into a self-improving system. Aurora learns from every request it serves and achieves a 1.25x speedup over a well-trained static speculator.
Why it matters: Aurora enables LLM inference to continuously improve without manual retuning, potentially reducing latency and cost in production.
Together AI introduces a 'Divide & Conquer' framework that breaks long documents into parallel chunks processed by a planner, workers, and manager. This method enables smaller models like Llama-3-70B and Qwen-72B to outperform GPT-4o single-shot on long context tasks.
Why it matters: This approach shows that smaller models can surpass larger ones on long context tasks through orchestration, potentially reducing reliance on massive models.
Together AI has announced a new platform for building real-time voice agents, featuring co-located speech-to-text, large language model, and text-to-speech infrastructure. The system achieves end-to-end latency under 500ms and natively supports Deepgram and Cartesia.
Why it matters: This enables developers to build responsive voice agents with low latency, improving user experience in conversational AI applications.
Together AI has expanded its fine-tuning service to include support for tool calling, reasoning, and vision-language models. The update also enables training of models with over 100 billion parameters, offers up to 6× higher throughput, and provides job cost and ETA estimates.
Why it matters: This update broadens the capabilities of Together AI's fine-tuning platform, enabling developers to customize advanced models for complex tasks involving function calling, reasoning, and multimodal inputs.
Together AI has announced Mamba-3, a state space model (SSM) designed for efficient inference. According to the company, Mamba-3 is faster than Transformers at decode, stronger than its predecessor Mamba-2, and is available as open-source.
Why it matters: Mamba-3 highlights ongoing advancements in state space models as potential alternatives to Transformers for language model inference.