AI & ML5 min read

Cognition SWE-2 Scores 92.8 on Terminal-Bench Using Kimi K3

Cognition launched SWE-2 on 10 September 2026 — post-trained from Kimi K3's 2.8T parameters using RL, scoring 92.8 on Terminal-Bench 2.1 and 64% cheaper than Fable 5.1.

Cognition SWE-2 Scores 92.8 on Terminal-Bench Using Kimi K3

Cognition Releases SWE-2 on 10 September 2026

On 10 September 2026, Cognition released SWE-2, the new model underlying Devin, the company's AI software engineer product. SWE-2 is post-trained from Kimi K3, the flagship model from Moonshot AI with 2.8 trillion parameters, using a reinforcement-learning process that Cognition developed to train multiple effort levels simultaneously while directly penalising the cost of each agent rollout. The model is available immediately in Devin Desktop and Devin CLI, with a staged rollout to Devin Web and Devin Fusion planned. On Terminal-Bench 2.1, the benchmark measuring performance on command-line software engineering tasks, SWE-2 scores 92.8 — ahead of Fable 5.1 at 91.4 and GPT-6 Astra at 89.9 in Cognition's published comparison table. Across comparable task quality, Cognition reports SWE-2 is 64 percent cheaper than Fable 5.1.

Kimi K3 as the Base and the Reinforcement-Learning Training Method

SWE-2 uses Kimi K3, Moonshot AI's 2.8-trillion-parameter model, as its post-training foundation. Cognition's contribution is the training methodology. The reinforcement-learning process trains three named effort levels — medium, high, and max — in a single run, rather than training separate models or applying simple inference-time scaling. Each effort level is trained with a cost penalty: the RL objective rewards correct task completion but penalises unnecessary agent steps and token use, directly embedding cost efficiency into the model rather than applying it as a post-hoc constraint at inference time. The result is that SWE-2 medium — the default effort level for most interactive tasks — uses 58 percent fewer agent steps and costs 81 percent less on average per run compared with SWE-1.7, the prior model, on equivalent tasks from Cognition's internal evaluation set.

Terminal-Bench 2.1 and DeepSWE 1.1 Results

SWE-2's headline result is 92.8 on Terminal-Bench 2.1, the benchmark that evaluates models on real command-line software engineering tasks including package management, environment configuration, script execution, and debugging terminal errors. At 92.8, SWE-2 leads Fable 5.1's 91.4 and GPT-6 Astra's 89.9 in Cognition's published table. On DeepSWE 1.1, which assesses models on real pull-request tasks drawn from open-source repositories, SWE-2 scores 73.0 — slightly below Sakana AI's Fugu Ultra v2.0 at 74.3 and DeepSeek V4.1 Flash at 74.2. On FrontierCode 1.1 Main, which tests sustained code reasoning across complex software problems, SWE-2 reaches 50.0, a fraction below Fable 5.1's 50.9 and 3.3 points below GPT-6 Astra's 53.3.

The Benchmark Where SWE-2 Falls Behind

The clearest limitation in SWE-2's published results is Terminal-Bench 4, a harder evaluation testing more complex multi-step tasks, longer-horizon reasoning, and deeper environment understanding. SWE-2 scores 27.3 on Terminal-Bench 4, compared with 55.8 for Fable 5.1 and 57.9 for GPT-6 Astra — a gap of more than 28 percentage points against both frontier models. The contrast between SWE-2's 92.8 on Terminal-Bench 2.1 and its 27.3 on Terminal-Bench 4 indicates the model performs well on common practical development tasks but has not yet matched frontier models on the most demanding, novel autonomous engineering challenges. Engineering teams evaluating SWE-2 for autonomous operation on complex repositories should benchmark their specific task types before committing to a production deployment.

Devin Availability and Effort Levels in Practice

SWE-2 is live in Devin Desktop and Devin CLI as of the 10 September 2026 launch. Devin Web and Devin Fusion will receive SWE-2 through a staged rollout. The three effort levels map to distinct deployment contexts: medium suits interactive developer workflows where quick iterations and low per-task cost matter; high is appropriate for moderately complex tasks involving multi-file changes and code research; max targets extended agentic operation where Devin runs autonomously on a well-specified task over many steps. All three effort levels share the same underlying model weights — only the inference-time budget differs, not the parameters.

What SWE-2 Means for Engineering Teams in India

For Indian product companies and development teams running AI-assisted coding at scale, SWE-2's 81 percent cost reduction on medium effort tasks against SWE-1.7 changes which workloads are economically viable. Tasks that previously justified Devin only in limited high-value situations — automated pull-request review on smaller internal services, test generation for legacy codebases, first-pass bug investigation on older code — become viable for continuous use at lower cost per task. Indian software companies serving regulated sectors such as banking, insurance, and healthcare maintain large volumes of legacy code where improving test coverage is a persistent operational priority; SWE-2 medium is well-positioned for that category of work. The availability in Devin Desktop and CLI also means engineering teams can integrate Devin into local development workflows without routing work through a browser, relevant for teams with stricter data-handling requirements.

The Bottom Line

On 10 September 2026, Cognition launched SWE-2, the new model powering Devin, post-trained from Moonshot AI's 2.8T-parameter Kimi K3 with an RL process that trains three effort levels simultaneously while penalising rollout cost. SWE-2 scores 92.8 on Terminal-Bench 2.1, ahead of Fable 5.1 and GPT-6 Astra, and 73.0 on DeepSWE 1.1. SWE-2 medium uses 58 percent fewer agent steps and costs 81 percent less than SWE-1.7; Cognition reports overall costs 64 percent below Fable 5.1. The model's published weakness is Terminal-Bench 4, where it scores 27.3 against Fable 5.1's 55.8. SWE-2 is live in Devin Desktop and CLI, with Web and Fusion to follow. For Indian engineering teams, the cost reduction makes AI-assisted coding and code review economically viable for a broader range of ongoing tasks than was practical with prior models.

Frequently Asked Questions

What is Cognition SWE-2 and when was it released?+

SWE-2 is the new model underlying Devin, Cognition's AI software engineer, released on 10 September 2026. It is post-trained from Moonshot AI's Kimi K3, a 2.8-trillion-parameter model, using a reinforcement-learning process that trains three effort levels — medium, high, and max — in a single run while penalising agent rollout cost. The model is available in Devin Desktop and Devin CLI at launch, with Devin Web and Devin Fusion receiving it in a staged rollout. SWE-2 scores 92.8 on Terminal-Bench 2.1 — ahead of Fable 5.1 at 91.4 and GPT-6 Astra at 89.9 — 73.0 on DeepSWE 1.1, and 50.0 on FrontierCode 1.1 Main. Cognition reports SWE-2 is 64 percent cheaper than Fable 5.1 and that SWE-2 medium uses 58 percent fewer agent steps and costs 81 percent less per run than the prior SWE-1.7 model.

What are SWE-2's benchmark scores and how does it compare to frontier models?+

SWE-2's strongest benchmark result is 92.8 on Terminal-Bench 2.1, which Cognition's published comparison table places ahead of Fable 5.1 at 91.4 and GPT-6 Astra at 89.9. On DeepSWE 1.1, which evaluates models on real pull-request tasks from open-source repositories, SWE-2 scores 73.0 — below Sakana AI's Fugu Ultra v2.0 at 74.3 and DeepSeek V4.1 Flash at 74.2. On FrontierCode 1.1 Main, SWE-2 reaches 50.0, slightly below Fable 5.1's 50.9 and 3.3 points below GPT-6 Astra's 53.3. The model's clearest weakness is Terminal-Bench 4, where it scores 27.3 against Fable 5.1's 55.8 and GPT-6 Astra's 57.9 — a gap of more than 28 percentage points. SWE-2 leads on common practical terminal engineering tasks but trails significantly on the more demanding Terminal-Bench 4 evaluation.

How does SWE-2's reinforcement-learning training method work?+

SWE-2's RL training process trains three effort levels — medium, high, and max — in a single reinforcement-learning run, rather than training separate model versions or applying inference-time scaling on a fixed model. Each effort level is trained with a reward that penalises both incorrect task outputs and unnecessary agent steps or token expenditure, embedding cost efficiency directly into the training objective. This contrasts with approaches that train a model to maximise task accuracy and then apply cost constraints at inference time. The result is a model whose medium effort level preserves or exceeds the task performance of SWE-1.7 while completing tasks with 58 percent fewer agent steps and at 81 percent lower cost on average per run. High and max effort levels allow the model to take more planning and verification steps on tasks that warrant a larger computational budget.

How does SWE-2 benefit Indian engineering teams using Devin?+

For Indian engineering teams using Devin, SWE-2's cost reduction — 64 percent cheaper than Fable 5.1 and 81 percent less per run than SWE-1.7 at medium effort — changes which tasks are economically viable for AI assistance. Tasks previously justified only for high-value, high-complexity work become practical for routine use: automated code review on smaller internal services, test generation for legacy codebases, and first-pass bug investigation on older applications. Indian software companies in banking, insurance, and government contracting maintain large volumes of legacy code where AI-assisted test coverage expansion is a persistent priority; SWE-2 medium is well-suited for that type of sustained, moderate-complexity work. The availability in Devin Desktop and CLI means teams can integrate Devin into local development workflows on AWS India or Azure India without routing tasks through a browser, which matters for teams with data-handling requirements around sensitive code.

Work with us

TechPillow builds ai & machine learning for teams across India and beyond.

Explore
TT

Written by

TechPillow Team

Sharing insights on technology, product development, and the Indian tech ecosystem.

Ready to Build Something Extraordinary?

From ideation to launch, we're your end-to-end technology partner.

Book a Free Strategy Call