A.X K2

Abstract

We introduce A.X K2, a 688B-parameter Mixture-of-Experts (MoE) language model trained from scratch as a high-performance foundation for agentic applications. A.X K2 is trained on approximately 8.5T tokens in total—fewer than its predecessor, A.X K1—by prioritizing a smaller but higher-quality data mixture; it nonetheless improves substantially over A.X K1 across the board—by over 30 percentage points on some benchmarks—reflecting substantial gains in token efficiency from an improved multi-stage data-processing pipeline. To support long contexts efficiently, we introduce Sparse Gated Attention (SGA), which combines sparse attention with gated attention, and adopt Gated Norm to stabilize large-scale training. To strengthen agentic capability, we substantially expand agentic and software-engineering data and extend the usable context length, enabling multi-step tool use and long-horizon task execution. A.X K2 further supports explicitly controllable reasoning through a simple yet effective Think-Fusion training recipe, enabling user-controlled switching between thinking and non-thinking modes within a single unified model. Extensive evaluations demonstrate that A.X K2 performs competitively against strong open-weight baselines, matching or exceeding them on math and Korean-language benchmarks.