Skip to content

People Papers Models Technologies Organizations Events Datasets Benchmarks

Atlas/Benchmarks

SWE-bench

October 2023

Fix real GitHub issues in real repositories. The coding-agent benchmark. From 2% in 2023 to above 70% on Verified in 2025.

Connections

Introduced in 1

SWE-bench: Can Language Models Resolve Real-World GitHub Issues?2023

Used by 1

SWE-bench Verified2024

Succeeded by 1

SWE-bench Verified2024

Models evaluated 1

Claude Opus 42025

Facts

Date: October 2023
Era: Generative AI and agents

Sources: AI4U editorial.

AI4U Atlas. Built from Wikidata, arXiv, OpenAlex, the Epoch AI notable models dataset (CC BY), the Hugging Face Hub and OpenRouter, plus our own writing.

Timeline · Chat · aheadintofun