Nvidia Groq 3 LPX Hits Production: 3,400 Tokens Per Second

Nvidia's Groq 3 LPX Reaches Full Production for Agentic AI

On 24 August 2026, Nvidia announced that the Groq 3 LPX — a dedicated AI inference accelerator developed through its approximately 20 billion dollar acquisition of Groq — has entered full production. The chip is an extension of Nvidia's Vera Rubin AI computing platform and is designed specifically for the latency-sensitive, high-throughput workloads that agentic AI systems demand. The announcement marks the commercial arrival of a chip category that Nvidia has been building toward since the Groq acquisition: purpose-built inference silicon that separates the token generation workload from the GPU clusters used for training.

In independent benchmarking by Artificial Analysis, the Groq 3 LPX achieved 3,400 output tokens per second running the Gemma 4 31B model on a 100,000-token context window. At that rate — roughly 2,550 words per second — a single LPX system can generate a detailed 2,000-word analysis document in under a second, or serve dozens of simultaneous conversational AI sessions without the latency spikes that degrade real-time agent interactions. That benchmark is not a theoretical ceiling but a production measurement taken under a large-context load representative of the agentic workloads the chip was designed for.

What Makes the Groq 3 LPX Architecturally Different

The chip's defining design decision is the placement of 500 megabytes of SRAM directly on the die. Most inference accelerators, including general-purpose GPUs, rely on high-bandwidth memory connected externally to the processor. Moving data between the compute cores and external memory repeatedly — for every attention head, every weight lookup, every key-value cache access — creates a bandwidth bottleneck that caps how quickly tokens can be generated regardless of how fast the compute cores run. By integrating 500 megabytes of on-die SRAM, the Groq 3 LPX eliminates that bottleneck for the models it is optimised to serve.

Samsung fabricates the chip. It slots into Nvidia's Vera Rubin platform as a dedicated inference module, and racks can be configured with up to 256 LPX accelerators, allowing data centre operators to provision purpose-built inference capacity at scale. The design allows enterprises to separate inference workloads — which prioritise low-latency token generation — from training workloads, which need different hardware profiles and run on different duty cycles. A facility that already operates Vera Rubin racks for training can add LPX modules for inference without replacing its existing infrastructure.

Who Deploys First and How the Market Opens

Nebius, an AI cloud provider that operates the Nebius Token Factory platform, will be the first to deploy the Groq 3 LPX in production. Groq, the inference cloud company whose chip architecture forms the basis of the LPX product, is scheduled to follow as a second major adopter. Both deployments target enterprise and developer customers building LLM-powered applications who need consistent, scalable, low-latency inference at a competitive cost per token.

The sequence matters because it signals that the LPX will reach developers through cloud APIs before it becomes available as on-premise hardware for most organisations. Teams building on inference cloud services from Nebius or Groq's own platform will experience the chip's performance gains through lower latency and higher throughput in the APIs they already call — without needing to manage silicon procurement or rack configuration themselves.

Why Inference Hardware Has Become a Standalone Market

Training and inference are now treated as structurally separate hardware problems. Training runs intermittently on large clusters, tolerates batch optimisations that add latency, and is economically manageable because it runs for a bounded period. Inference runs continuously at production scale, must respond to individual requests in tens of milliseconds, and over the operational life of an AI application consumes more compute than training ever did. A model trained once may serve billions of inference calls over two or three years.

That asymmetry — continuous high-volume inference versus episodic training — has created demand for purpose-built inference chips that sacrifice GPU-level programmability to maximise token generation speed on a defined set of model architectures. Nvidia's entry into this category with the Groq 3 LPX, via the Groq acquisition, signals that the inference hardware market is now large enough to justify dedicated silicon rather than repurposed general compute. This is a structural shift, not an incremental product update.

What the Groq 3 LPX Means for AI Teams Building in India

Indian AI development teams deploying production agents in customer service, document processing, workflow automation, and regional language interfaces depend on the hardware layer underneath the cloud APIs they call. When inference accelerators reduce the cost and latency of token generation at cloud providers, the economics of building production AI applications improve for every team that operates through those clouds.

Teams evaluating whether to build AI agents for real-time use cases — interactive chatbots, document analysis pipelines, voice interfaces — have historically had to budget for inference latency and cost as limiting constraints. Each hardware generation that pushes throughput higher and cost lower expands the set of use cases where an AI agent is economically viable. The Groq 3 LPX, reaching cloud availability through Nebius and Groq's platform in 2026, is the most recent step in that structural direction. For most real-time use cases in India's enterprise and consumer markets, the economics of AI agents are improving faster than development timelines — making the decision to build now more defensible than waiting.

The Bottom Line

On 24 August 2026, Nvidia confirmed that the Groq 3 LPX inference accelerator has entered full production. Built from Nvidia's approximately 20 billion dollar acquisition of Groq, the chip delivers 3,400 output tokens per second on the Gemma 4 31B model at a 100,000-token context, places 500 megabytes of SRAM on die to eliminate memory bandwidth constraints, is fabricated by Samsung, and slots into Vera Rubin racks at up to 256 units per configuration. Nebius Token Factory deploys first, with Groq's inference cloud to follow. For Indian AI teams building production applications, the arrival of purpose-built inference silicon at cloud scale means faster and more economical agent deployments are becoming the baseline expectation — and each generation makes the business case for agentic AI stronger than the last.