RUC-NLPIR/Awesome-Long-Horizon-Agents - AI 开源项目

Toward Long-Horizon AI Agents — Foundations, Evolution, Harnesses, Optimization, Applications, and Frontiers

News

[2026/07] 📣 The brief introduction of our survey can be found on X and Xiaohongshu.
[2026/07] 📄 Our paper Towards Long-Horizon Agents: A Survey is available on OpenReview.
[2026/07] 🚀 We released the paper list for Towards Long-Horizon Agents: A Survey, restructured to mirror the paper chapter-by-chapter.
[2026/07] 🙌 Open PR Request: add a missing work in PR ([Venue Year] Title. [paper] [code]).

Time horizon growth of frontier AI agents

Figure 1. The time horizon of frontier AI agents is growing exponentially, roughly doubling every few months.

Introduction

Large language models have evolved from single-turn chatbots into the decision-making core of autonomous agents. As Figure 1 shows, the time horizon of tasks they can complete unaided is growing exponentially. This surfaces one decisive requirement we call long horizon: persistent iteration across reasoning, tool use, observation, and revision over many interdependent steps — from tasks within a single context window to those spanning windows, sessions, or open-ended task streams.

Our survey frames long-horizon agency as a system-level capability jointly shaped by two forces:

Externalized harness engineering: loops and workflows, context and memory, tools and skills, orchestration, hooks, and verification.
Internalized model optimization: architecture, data and environment synthesis, pre-/mid-training, fine-tuning, agentic reinforcement learning, on-policy distillation, and self-evolution.

The two sides co-evolve through experience and feedback: capabilities first implemented explicitly in the harness may later be internalized into the model policy, while stronger policies in turn enable more capable harnesses. Figure 2 lays out this co-evolutionary landscape end to end.

Figure 2. The landscape of long-horizon agent research, organized around externalized harness engineering and internalized model optimization.

Citation

If you find this survey and repository useful for your research, please consider citing:

@article{dong2026longhorizon,
    doi = {10.20944/preprints202607.1328.v1},
    url = {https://doi.org/10.20944/preprints202607.1328.v1},
    year = 2026,
    month = {July},
    publisher = {Preprints},
    author = {Guanting Dong and Xiaoshuai Song and Yuyang Hu and Jiajie Jin and Chenghao Zhang and Yifei Chen and Xiaoxi Li and Huaying Yuan and Xinyu Yang and Tongyu Wen and Jiejun Tan and Hongjin Qian and Shijue Huang and Junting Lu and Zhenyu Li and Wanjun Zhong and Yutao Zhu and Tat-Seng Chua and Zhicheng Dou and Ji-Rong Wen},
    title = {Towards Long-Horizon Agents: A Survey},
    journal = {Preprints}
}

Foundations: Formalizing Long-Horizon Agents
Evolution: From Prompting to Runtime
Stage I — Prompt Engineering (2020–2023)
Stage II — Context Engineering (2023–2025)
Stage III — Runtime Harnesses (2025–Present)
Harnesses: Externalizing Long-Horizon Capability (Pillar I)
Loops and Workflows
Context and Memory
Tools, MCP, and Skills
Orchestration
Hooks and Middleware
Verification
Optimization: Internalizing Long-Horizon Capability (Pillar II)
Architectural Substrate
Data and Environment Synthesis
Pre-training and Mid-training
Fine-tuning
Agentic Reinforcement Learning
On-Policy Distillation
Self-Evolution
Applications: Long-Horizon Agents in Practice
Software Engineering
Information Seeking
Computer Use
Multimodal Agents
General-Purpose Agents
Benchmarks and Resources
Frontiers: Open Problems
Citation
Contributing

Foundations: Formalizing Long-Horizon Agents

Three levels of long-horizon tasks and capabilities

Section figure. Three levels of long-horizon tasks (H1 ⊂ H2 ⊂ H3) and their required capabilities (C1 ⊂ C2 ⊂ C3).

We formalize a long-horizon agent as a base policy coupled to a surrounding harness, $\mathrm{Agent}=\pi_\theta\oplus\mathcal{H}$, and organize long-horizon difficulty into three nested levels (H1 ⊂ H2 ⊂ H3), each paired with the capability it demands (C1 ⊂ C2 ⊂ C3):

Level	Task horizon	Demanded capability
H1	Intra-context, within one window (~minutes)	C1 — Intra-context interactive reasoning
H2	Cross-context, across windows/sessions (~hours–days)	C2 — Cross-context state & memory
H3	Cross-task, open-ended task stream	C3 — Cross-task experience accumulation

To make the notion of "horizon" concrete, METR measures capability as the length of tasks an agent can complete at a fixed success rate (e.g., the 50%-task-completion time horizon), giving an empirical yardstick that separates long-horizon agency from adjacent notions such as long-running execution, autonomy, and self-evolution.

Evolution: From Prompting to Runtime

Section figure. Three stages of co-evolution: from the language of a prompt, to the information per call, to the whole trajectory sustained by a runtime harness.

Stage I — Prompt Engineering (2020–2023)

NeurIPS 2020 Language Models are Few-Shot Learners. [paper]
NeurIPS 2022 Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. [paper]
NeurIPS 2022 Large Language Models are Zero-Shot Reasoners. [paper]
ICLR 2023 Self-Consistency Improves Chain of Thought Reasoning in Language Models. [paper]
ICLR 2023 Least-to-Most Prompting Enables Complex Reasoning in Large Language Models. [paper]
ICLR 2023 ReAct: Synergizing Reasoning and Acting in Language Models. [paper] [code]
ICML 2023 PAL: Program-aided Language Models. [paper] [code]
TMLR 2023 Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks. [paper] [code]
NeurIPS 2023 Tree of Thoughts: Deliberate Problem Solving with Large Language Models. [paper] [code]
CoRL 2022 Do As I Can, Not As I Say: Grounding Language in Robotic Affordances. [paper] [code]
NeurIPS 2022 Training Language Models to Follow Instructions with Human Feedback. [paper]
ICLR 2023 Large Language Models Are Human-Level Prompt Engineers. [paper] [code]
EMNLP 2023 Automatic Prompt Optimization with “Gradient Descent” and Beam Search. [paper]
arXiv 2021 Show Your Work: Scratchpads for Intermediate Computation with Language Models. [paper]
EMNLP-IJCNLP 2019 Language Models as Knowledge Bases?. [paper]
NeurIPS 2023 Describe, Explain, Plan and Select: Interactive Planning with LLMs Enables Open-World Multi-Task Agents. [paper] [code]

Stage II — Context Engineering (2023–2025)

NeurIPS 2020 Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. [paper]
ACL 2023 Precise Zero-Shot Dense Retrieval without Relevance Labels. [paper]
ICLR 2024 RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval. [paper] [code]
ICLR 2024 Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. [paper] [code]
NeurIPS 2023 Toolformer: Language Models Can Teach Themselves to Use Tools. [paper]
NeurIPS 2024 Gorilla: Large Language Model Connected with Massive APIs. [paper] [code]
ICLR 2024 ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs. [paper] [code]
arXiv 2021 WebGPT: Browser-assisted Question-answering with Human Feedback. [paper]
NeurIPS 2023 HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face. [paper] [code]
NeurIPS 2022 FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness. [paper] [code]
TACL 2024 Lost in the Middle: How Language Models Use Long Contexts. [paper]
arXiv 2023 MemGPT: Towards LLMs as Operating Systems. [paper] [code]
UIST 2023 Generative Agents: Interactive Simulacra of Human Behavior. [paper] [code]
EMNLP 2023 LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models. [paper] [code]
arXiv 2023 Retrieval-Augmented Generation for Large Language Models: A Survey. [paper]
ICML 2020 REALM: Retrieval-Augmented Language Model Pre-Training. [paper]
arXiv 2024 RULER: What's the Real Context Size of Your Long-Context Language Models?. [paper]
arXiv 2025 ACON: Optimizing Context Compression for Long-horizon LLM Agents. [paper]
arXiv 2025 A Survey of Context Engineering for Large Language Models. [paper]
arXiv 2023 ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases. [paper]
TACL 2022 ♫ MuSiQue: Multihop Questions via Single-hop Question Composition. [paper]
ACL 2024 ∞Bench: Extending Long Context Evaluation Beyond 100K Tokens. [paper]

Stage III — Runtime Harnesses (2025–Present)

NeurIPS 2023 Reflexion: Language Agents with Verbal Reinforcement Learning. [paper] [code]
NeurIPS 2023 Self-Refine: Iterative Refinement with Self-Feedback. [paper] [code]
ICML 2024 Executable Code Actions Elicit Better LLM Agents. [paper] [code]
ICLR 2024 MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework. [paper] [code]
COLM 2024 AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation. [paper] [code]
ACL 2024 ChatDev: Communicative Agents for Software Development. [paper] [code]
arXiv 2024 Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks. [paper] [code]
Open standard 2024 Model Context Protocol. [paper] [code]
ICLR 2025 OpenHands: An Open Platform for AI Software Developers as Generalist Agents. [paper] [code]
NeurIPS 2024 SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering. [paper] [code]
ICLR 2026 Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents. [paper]
COLM 2025 Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning. [paper] [code]
Web specification 2025 AGENTS.md. [paper]
arXiv 2024 AutoFlow: Automated Workflow Generation for Large Language Model Agents. [paper]
arXiv 2026 Agents' Last Exam. [paper]
arXiv 2026 Before the Tool Call: Deterministic Pre-Action Authorization for Autonomous AI Agents. [paper]
arXiv 2026 Claw-R1: A Step-Level Data Middleware System for Agentic Reinforcement Learning. [paper]
arXiv 2025 SEW: Self-Evolving Agentic Workflows for Automated Code Generation. [paper]
arXiv 2026 RLAnything: Forge Environment, Policy, and Reward Model in Completely Dynamic RL System. [paper]
FSE 2026 AgentBound: Securing Execution Boundaries of AI Agents. [paper]
arXiv 2025 UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning. [paper]
Preprints 2026 Harness Engineering for Language Agents: The Harness Layer as Control, Agency, and Runtime. [paper]
arXiv 2026 LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks. [paper] [code]

Harnesses: Externalizing Long-Horizon Capability (Pillar I)

Section figure. An agent harness in action: six components sustain a single goal across many dependent steps.

Loops and Workflows

Linear Workflows - ICLR 2023 ReAct: Synergizing Reasoning and Acting in Language Models. [paper] [code] - NeurIPS 2023 Reflexion: Language Agents with Verbal Reinforcement Learning. [paper] [code] - NeurIPS 2023 Self-Refine: Iterative Refinement with Self-Feedback. [paper] [code] - ICLR 2024 Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. [paper] [code] - TMLR 2024 Cognitive Architectures for Language Agents. [paper] [code] - IEEE ICAIBD 2025 A Survey on Agent Workflow - Status and Future. [paper]

Plan-Execute Workflows - ACL 2023 Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models. [paper] [code] - arXiv 2023 ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models. [paper] [code] - Findings of NAACL 2024 ADaPT: As-Needed Decomposition and Planning with Language Models. [paper] [code] - arXiv 2026 O-Researcher: An Open Ended Deep Research Model via Multi-Agent Distillation and Agentic RL. [paper] - arXiv 2026 Toward Generalist Autonomous Research via Hypothesis-Tree Refinement. [paper] - arXiv 2026 Verified Multi-Agent Orchestration: A Plan-Execute-Verify-Replan Framework for Complex Query Resolution. [paper]

Branching Workflows - NeurIPS 2023 Tree of Thoughts: Deliberate Problem Solving with Large Language Models. [paper] [code] - ICLR 2023 Self-Consistency Improves Chain of Thought Reasoning in Language Models. [paper] - ICML 2024 Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models. [paper] [code] - NAACL 2025 CodeTree: Agent-guided Tree Search for Code Generation with Large Language Models. [paper] - AAMAS 2026 ReAcTree: Hierarchical LLM Agent Trees with Control Flow for Long-Horizon Task Planning. [paper] - AAAI 2024 Graph of Thoughts: Solving Elaborate Problems with Large Language Models. [paper] - TMLR 2025 Tree Search for Language Model Agents. [paper] - Findings of ACL 2026 Chain-in-Tree: Back to Sequential Reasoning in LLM Tree Search. [paper] - arXiv 2025 Sherlock: Reliable and Efficient Agentic Workflow Execution. [paper]

Context and Memory

Working Context (discard / compress / select) - Anthropic Blog 2025 Effective Context Engineering for AI Agents. [paper] - arXiv 2025 ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarization. [paper] [code] - ICLR 2026 MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent. [paper] [code] - ICLR 2026 MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents. [paper] [code] - ACL 2025 HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Model. [paper] [code] - ICLR 2026 Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models. [paper] - ACL Findings 2026 Memory-as-Action: Autonomous Context Curation for Long-Horizon Agentic Tasks. [paper] [code] - arXiv 2025 DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models. [paper] - arXiv 2026 MiroThinker-1.7 & H1: Towards Heavy-Duty Research Agents via Verification. [paper] - arXiv 2025 IterResearch: Rethinking Long-Horizon Agents with Interaction Scaling. [paper] - arXiv 2026 AgentFugue: Agent Scaling for Long-Horizon Tasks through Collective Reasoning. [paper] - arXiv 2026 ContextBudget: Budget-Aware Context Management for Long-Horizon Search Agents. [paper] - AAMAS 2026 LEGOMem: Modular Procedural Memory for Multi-agent LLM Systems for Workflow Automation. [paper] - arXiv 2026 Memex(RL): Scaling Long-Horizon LLM Agents via Indexed Experience Memory. [paper] - arXiv 2026 SAM: State-Adaptive Memory for Long-Horizon Reasoning Agent. [paper] - arXiv 2025 Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solving. [paper]

Persistent Memory (factual / experiential) - ECAI 2025 Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory. [paper] [code] - NeurIPS 2024 HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models. [paper] [code] - NeurIPS 2025 A-Mem: Agentic Memory for LLM Agents. [paper] [code] - EMNLP 2025 Memory OS of AI Agent. [paper] [code] - AAAI 2024 ExpeL: LLM Agents Are Experiential Learners. [paper] [code] - ICML 2025 Agent Workflow Memory. [paper] [code] - ICLR 2026 ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory. [paper] - TMLR 2024 Voyager: An Open-Ended Embodied Agent with Large Language Models. [paper] [code] - arXiv 2025 Zep: A Temporal Knowledge Graph Architecture for Agent Memory. [paper] - Findings of ACL 2026 From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanisms. [paper] - NeurIPS 2025 G-Memory: Tracing Hierarchical Memory for Multi-Agent Systems. [paper] - arXiv 2026 Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills. [paper] - arXiv 2025 MIRIX: Multi-Agent Memory System for LLM-Based Agents. [paper] - AAAI 2026 PRIME: Planning and Retrieval-Integrated Memory for Enhanced Reasoning. [paper] - EACL 2026 H-MEM: Hierarchical Memory for High-Efficiency Long-Term Reasoning in LLM Agents. [paper] - WWW 2025 MemoRAG: Boosting Long Context Processing with Global Memory-Enhanced Retrieval Augmentation. [paper] - ACL 2026 EverMemOS: A Self-Organizing Memory Operating System for Structured Long-Horizon Reasoning. [paper] - arXiv 2026 MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks. [paper] - arXiv 2025 Memory in the Age of AI Agents. [paper] - arXiv 2026 Anatomy of Agentic Memory: Taxonomy and Empirical Analysis of Evaluation and System Limitations. [paper] - arXiv 2026 From Player to Master: Enhancing Test-Time Learning of LLM Agents via Reinforcement Learning over Memory. [paper] - arXiv 2026 Rethinking Memory Mechanisms of Foundation Agents in the Second Half: A Survey. [paper] - arXiv 2026 MemSifter: Offloading LLM Memory Retrieval via Outcome-Driven Proxy Reasoning. [paper] - arXiv 2025 From Human Memory to AI Memory: A Survey on Memory Mechanisms in the Era of LLMs. [paper] - arXiv 2026 Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering. [paper] - arXiv 2026 Unified Context Evolution for LLM Agents. [paper] - arXiv 2026 Inference-Time Budget Control for LLM Search Agents. [paper]

Tools, MCP, and Skills

Tool interfaces & protocols - NeurIPS 2023 Toolformer: Language Models Can Teach Themselves to Use Tools. [paper] - ICLR 2024 ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs. [paper] [code] - 2024 Model Context Protocol (MCP) Specification. [paper] [code] - ICML 2025 The Berkeley Function Calling Leaderboard (BFCL): From Tool Use to Agentic Evaluation of Large Language Models. [paper] [code] - ICLR 2025 τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. [paper] [code] - arXiv 2025 MCP-Universe: Benchmarking Large Language Models with Real-World Model Context Protocol Servers. [paper] [code] - ICML 2026 ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandbox. [paper] - ICLR 2026 VitaBench: Benchmarking LLM Agents with Versatile Interactive Tasks in Real-world Applications. [paper] - arXiv 2026 Schema First Tool APIs for LLM Agents: A Controlled Study of Tool Misuse, Recovery, and Budgeted Performance. [paper] - arXiv 2026 AutomationBench. [paper] - ICML 2026 UltraHorizon: Benchmarking LLM-Agent Capabilities in Ultra Long-Horizon Scenarios. [paper] - arXiv 2026 The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestration. [paper] - ICLR 2026 Benchmarking LLM Tool-Use in the Wild. [paper]

Active tool discovery - arXiv 2025 RAG-MCP: Mitigating Prompt Bloat in LLM Tool Selection via Retrieval-Augmented Generation. [paper] - ICML 2024 AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Calls. [paper] [code] - arXiv 2025 MCP-Zero: Active Tool Discovery for Autonomous LLM Agents. [paper] - ICLR 2025 ToolGen: Unified Tool Retrieval and Calling via Generation. [paper] [code] - WWW 2026 DeepAgent: A General Reasoning Agent with Scalable Toolsets. [paper] [code] - arXiv 2026 From Tool Orchestration to Code Execution: A Study of MCP Design Choices. [paper] - arXiv 2026 Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction. [paper] - arXiv 2025 From REST to MCP: An Empirical Study of API Wrapping and Automated Server Generation for LLM Agents. [paper] - arXiv 2025 Keyword search is all you need: Achieving RAG-Level Performance without vector databases using agentic tool use. [paper] - arXiv 2026 CodeScout: An Effective Recipe for Reinforcement Learning of Code Search Agents. [paper] - arXiv 2026 Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forward. [paper]

Skill libraries - TMLR 2024 Voyager: An Open-Ended Embodied Agent with Large Language Models. [paper] [code] - 2025 Introducing Agent Skills. [paper]

arXiv 2026 SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization. [paper]
arXiv 2026 Skill1: Unified Evolution of Skill-Augmented Agents via Reinforcement Learning. [paper]
arXiv 2025 Alita: Generalist Agent Enabling Scalable Agentic Reasoning with Minimal Predefinition and Maximal Self-Evolution. [paper]
COLM 2025 Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents. [paper]
ICML 2026 Agentic Proposing: Enhancing Large Language Model Reasoning via Compositional Skill Synthesis. [paper]
arXiv 2026 SkillCraft: Can LLM Agents Learn to Use Tools Skillfully?. [paper]
arXiv 2026 Organizing, Orchestrating, and Benchmarking Agent Skills at Ecosystem Scale. [paper]
arXiv 2026 When Single-Agent with Skills Replace Multi-Agent Systems and When They Fail. [paper]
arXiv 2026 SkillGraph: Skill-Augmented Reinforcement Learning for Agents via Evolving Skill Graphs. [paper]
arXiv 2025 Memento: Fine-tuning LLM Agents without Fine-tuning LLMs. [paper]
ICML 2026 Skill-Pro: Learning Reusable Skills from Experience via Non-Parametric PPO for LLM Agents. [paper]
ICML 2026 SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experience. [paper]
arXiv 2026 Beyond Static Tools: Test-Time Tool Evolution for Scientific Reasoning. [paper]
arXiv 2026 GraSP: Graph-Structured Skill Compositions for LLM Agents. [paper]
arXiv 2026 From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI. [paper]
arXiv 2026 A Comprehensive Survey on Agent Skills: Taxonomy, Techniques, and Applications. [paper]
arXiv 2026 SkillNet: Create, Evaluate, and Connect AI Skills. [paper]

Orchestration

Decomposition & roles - ICLR 2024 MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework. [paper] [code] - NeurIPS 2023 CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society. [paper] [code] - ACL 2024 ChatDev: Communicative Agents for Software Development. [paper] [code] - COLM 2024 AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversations. [paper] [code] - Neural Networks 2025 TDAG: A Multi-Agent Framework based on Dynamic Task Decomposition and Agent Generation. [paper] - ICLR 2025 Agent-Oriented Planning in Multi-Agent Systems. [paper]

Coordination topologies - arXiv 2024 Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks. [paper] [code] - ICLR 2024 AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors. [paper] [code] - ICLR 2025 Mixture-of-Agents Enhances Large Language Model Capabilities. [paper] [code] - arXiv 2023 A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration. [paper] [code] - ICLR 2025 Scaling Large Language Model-based Multi-Agent Collaboration. [paper] [code] - ICLR 2026 Flash-Searcher: Fast and Effective Web Agents via DAG-Based Parallel Execution. [paper] - NeurIPS 2024 Self-playing Adversarial Language Game Enhances LLM Reasoning. [paper] - NeurIPS 2025 Agint: Agentic Graph Compilation for Software Engineering Agents. [paper] - arXiv 2026 AgentRadio: Passive Awareness for Long-Horizon Multi-Agent Collaboration. [paper] [code] - arXiv 2025 Multi-Agent Collaboration Mechanisms: A Survey of LLMs. [paper]

Orchestration optimization - ICLR 2025 AFlow: Automating Agentic Workflow Generation. [paper] [code] - ICML 2024 GPTSwarm: Language Agents as Optimizable Graphs. [paper] [code] - ACL 2025 MasRouter: Learning to Route LLMs for Multi-Agent Systems. [paper] [code] - EMNLP 2025 SwarmAgentic: Towards Fully Automated Agentic System Generation via Swarm Intelligence. [paper] - NeurIPS 2025 Multi-Agent Collaboration via Evolving Orchestration. [paper] - arXiv 2025 AgentOrchestra: Orchestrating Multi-Agent Intelligence with the Tool-Environment-Agent (TEA) Protocol. [paper] - ICML 2026 AOrchestra: Automating Sub-Agent Creation for Agentic Orchestration. [paper] - arXiv 2026 CORAL: Towards Autonomous Multi-Agent Evolution for Open-Ended Discovery. [paper] - arXiv 2026 Drop the Hierarchy and Roles: How Self-Organizing LLM Agents Outperform Designed Structures. [paper]

Agent protocols - 2025 Agent2Agent (A2A) Protocol Specification. [paper] [code] - 2025 Agent Communication Protocol (ACP, IBM). [paper] - 2024 Model Context Protocol (MCP) Specification. [paper] [code] - arXiv 2025 A Survey of AI Agent Protocols. [paper] - arXiv 2025 A Survey of Agent Interoperability Protocols: Model Context Protocol (MCP), Agent Communication Protocol (ACP), Agent-to-Agent Protocol (A2A), and Agent Network Protocol (ANP). [paper] - arXiv 2024 A Scalable Communication Protocol for Networks of Large Language Models. [paper] - IBM Research 2025 Agent Communication Protocol (ACP). [paper] - arXiv 2026 Agentic Aggregation for Parallel Scaling of Long-Horizon Agentic Tasks. [paper] - arXiv 2026 Agentic Test-Time Scaling for WebAgents. [paper] - arXiv 2025 LOKA Protocol: A Decentralized Framework for Trustworthy and Ethical AI Agent Ecosystems. [paper]

Hooks and Middleware

Pre-defined rule-based hooks - ICSA 2025 Swiss Cheese Model for AI Safety: A Taxonomy and Reference Architecture for Multi-Layered Guardrails of Foundation Model Based Agents. [paper] - arXiv 2026 AEGIS: No Tool Call Left Unchecked — A Pre-Execution Firewall and Audit Layer for AI Agents. [paper] - arXiv 2026 Authenticated Workflows: A Systems Approach to Protecting Agentic AI. [paper] - arXiv 2025 Magentic-UI: Towards Human-in-the-loop Agentic Systems. [paper] - NDSS 2025 IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems. [paper]

Custom user-defined hooks - EMNLP 2023 NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails. [paper] [code] - ICSE 2026 AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents. [paper] - arXiv 2025 Progent: Securing AI Agents with Privilege Control. [paper] - ICML 2025 GuardAgent: Safeguard LLM Agents via Knowledge-Enabled Reasoning. [paper] - ICML 2025 ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning. [paper] - arXiv 2023 Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations. [paper] [code] - arXiv 2025 LlamaFirewall: An open source guardrail system for building secure AI agents. [paper] - SaTML 2026 Defeating Prompt Injections by Design. [paper] - ICLR 2026 Workshop (VerifAI-2) Enforcing Temporal Constraints for LLM Agents. [paper] - NeurIPS 2025 Workshop (RegML) Policy-as-Prompt: Turning AI Governance Rules into Guardrails for AI Agents. [paper] - arXiv 2026 Formal Policy Enforcement for Real-World Agentic Systems. [paper] - arXiv 2025 VeriGuard: Enhancing LLM Agent Safety via Verified Code Generation. [paper]

Runtime-adaptive hooks - Nature 2024 Detecting hallucinations in large language models using semantic entropy. [paper] - arXiv 2025 SentinelAgent: Graph-based Anomaly Detection in Multi-Agent Systems. [paper] - ACL 2025 AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detection. [paper] - ASE 2025 AdaptiveGuard: Towards Adaptive Runtime Safety for LLM-Powered Software. [paper] - AAAI 2026 Workshop (TrustAgent) Agent-SafetyBench: Evaluating the Safety of LLM Agents. [paper] [code] - AISTATS 2026 Enhancing LLM Safety Through a Theoretical Minimax Game Lens. [paper] - arXiv 2026 Quantifying Frontier LLM Capabilities for Container Sandbox Escape. [paper] - arXiv 2026 Neuro-Symbolic Verification on Instruction Following of LLMs. [paper] - ACL 2025 Uncertainty Propagation on LLM Agent. [paper] - arXiv 2026 TrajAD: Trajectory Anomaly Detection for Trustworthy LLM Agents. [paper] - arXiv 2025 ProbGuard: Probabilistic Runtime Monitoring for LLM Agent Safety. [paper] - arXiv 2026 Agentic Uncertainty Quantification. [paper]

Verification

Assessment targets - EMNLP 2023 SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models. [paper] [code] - ICML 2024 Improving Factuality and Reasoning in Language Models through Multiagent Debate. [paper] [code] - ICLR 2025 Workshop (MCDC) Multi-Agent Verification: Scaling Test-Time Compute with Multiple Verifiers. [paper] - ICLR 2025 AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents. [paper] - AAAI 2026 Workshop (TrustAgent) Agent-SafetyBench: Evaluating the Safety of LLM Agents. [paper] [code] - arXiv 2026 AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security. [paper] - arXiv 2026 AgentDoG 1.5: A Lightweight and Scalable Alignment Framework for AI Agent Safety and Security. [paper]

Verification levels - ICLR 2024 Let's Verify Step by Step. [paper] [code] - ICLR 2024 CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing. [paper] [code] - NeurIPS 2023 Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. [paper] [code] - ICML 2025 Agent-as-a-Judge: Evaluate Agents with Agents. [paper] [code] - NeurIPS 2025 Web-Shepherd: Advancing PRMs for Reinforcing Web Agents. [paper] - arXiv 2021 Training Verifiers to Solve Math Word Problems. [paper] - arXiv 2022 Solving math word problems with process- and outcome-based feedback. [paper] - ICLR 2025 Generative Verifiers: Reward Modeling as Next-Token Prediction. [paper] - arXiv 2026 SWE-TRACE: Optimizing Long-Horizon SWE Agents Through Rubric Process Reward Models and Heuristic Test-Time Scaling. [paper] - arXiv 2026 CollabEval: Enhancing LLM-as-a-Judge via Multi-Agent Collaboration. [paper] - arXiv 2026 Agentic Reward Modeling: Verifying GUI Agent via Online Proactive Interaction. [paper] - arXiv 2026 Agent-as-a-Judge. [paper] - COLM 2025 Putting the Value Back in RL: Better Test-Time Scaling by Unifying LLM Reasoners With Verifiers. [paper]

Verifier strategies - ACL 2024 Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations. [paper] - ICML 2025 Free Process Rewards without Process Labels. [paper] [code] - ICML 2024 Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models. [paper] [code] - NeurIPS 2024 ReST-MCTS: LLM Self-Training via Process Reward Guided Tree Search. [paper] [code] - ICLR 2025 Scaling LLM Test-Time Compute Optimally Can be More Effective than Scaling Parameters for Reasoning. [paper] - arXiv 2025 MCTS-Judge: Test-Time Scaling in LLM-as-a-Judge for Code Correctness Evaluation. [paper] - arXiv 2026 Scaling Medical Reasoning Verification via Tool-Integrated Reinforcement Learning. [paper] - NeurIPS 2025* Wider or Deeper? Scaling LLM Inference-Time Compute with Adaptive Branching Tree Search. [paper]

Optimization: Internalizing Long-Horizon Capability (Pillar II)

Section figure. The agentic training pipeline: an architectural substrate plus six training stages for internalizing long-horizon capability.

Architectural Substrate

arXiv 2020 Longformer: The Long-Document Transformer. [paper] [code]
NeurIPS 2020 Big Bird: Transformers for Longer Sequences. [paper]
COLM 2024 Mamba: Linear-Time Sequence Modeling with Selective State Spaces. [paper] [code]
ICML 2024 Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality. [paper] [code]
EMNLP 2023 GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints. [paper]
arXiv 2024 DeepSeek-V3 Technical Report. [paper] [code]
ICLR 2025 Jamba: Hybrid Transformer-Mamba Language Models. [paper]
arXiv 2025 Kimi Linear: An Expressive, Efficient Attention Architecture. [paper]
ICML 2024 EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty. [paper] [code]
ICML 2023 Fast Inference from Transformers via Speculative Decoding. [paper]
arXiv 2025 MoBA: Mixture of Block Attention for Long-Context LLMs. [paper]
arXiv 2025 Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference. [paper]
arXiv 2019 Fast Transformer Decoding: One Write-Head is All You Need. [paper]
arXiv 2020 Linformer: Self-Attention with Linear Complexity. [paper]
arXiv 2023 Retentive Network: A Successor to Transformer for Large Language Models. [paper]
arXiv 2023 Mistral 7B. [paper]
arXiv 2024 ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools. [paper]
arXiv 2024 Gemma 2: Improving Open Language Models at a Practical Size. [paper]
arXiv 2025 Gemma 3 Technical Report. [paper]
arXiv 2024 DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model. [paper]
arXiv 2025 GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models. [paper]
arXiv 2025 Qwen3 Technical Report. [paper]

Data and Environment Synthesis

ICLR 2026 TaskCraft: Automated Generation of Agentic Tasks. [paper] [code]
ICLR 2026 WebShaper: Agentically Data Synthesizing via Information-Seeking Formalization. [paper] [code]
arXiv 2026 SearchEyes: Towards Frontier Multimodal Deep Search Intelligence via Search World Simulation. [paper] [code]
CVPR 2026 Socratic-Geo: Synthetic Data Generation and Cross-Modal Geometric Reasoning via Multi-Agent Interaction. [paper] [code]
ICLR 2026 Expanding the Capability Frontier of LLM Agents with ZPD-Guided Data Synthesis. [paper]
ICML 2025 Training Software Engineering Agents and Verifiers with SWE-Gym. [paper] [code]
ICLR 2024 WebArena: A Realistic Web Environment for Building Autonomous Agents. [paper] [code]
NeurIPS 2024 OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments. [paper] [code]
TMLR 2025 Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agents. [paper] [code]
arXiv 2025 TOUCAN: Synthesizing 1.5M Tool-Agentic Data from Real-World MCP Environments. [paper]
ICLR 2026 AgentGym-RL: An Open-Source Framework to Train LLM Agents for Long-Horizon Decision Making via Multi-Turn RL. [paper] [code]
arXiv 2026 Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence. [paper]
arXiv 2026 GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents. [paper]
NeurIPS 2022 MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge. [paper]
arXiv 2026 AOI: Turning Failed Trajectories into Training Signals for Autonomous Cloud Diagnosis. [paper]
arXiv 2026 ASTRA: Automated Synthesis of agentic Trajectories and Reinforcement Arenas. [paper]
NeurIPS 2025 Workshop (MTI-LLM) Beyond Ten Turns: Unlocking Long-Horizon Agentic Search with Large-Scale Asynchronous RL. [paper]
ICLR 2026 AgentFold: Long-Horizon Web Agents with Proactive Context Folding. [paper]
arXiv 2026 AgentHER: Hindsight Experience Replay for LLM Agent Trajectory Relabeling. [paper]
arXiv 2025 AgentRL: Scaling Agentic Reinforcement Learning with a Multi-Turn, Multi-Task Framework. [paper]
COLM 2025 AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories. [paper]
ICLR 2026 AgentSynth: Scalable Task Generation for Generalist Computer-Use Agents. [paper]
arXiv 2026 AutoWebWorld: Synthesizing Infinite Verifiable Web Environments via Finite State Machines. [paper]
arXiv 2026 CLI-Gym: Scalable CLI Task Generation via Agentic Environment Inversion. [paper]
arXiv 2026 Code2World: A GUI World Model via Renderable Code Generation. [paper]
arXiv 2025 Scaling Long-Horizon LLM Agent via Context-Folding. [paper]
arXiv 2025 Cosmos World Foundation Model Platform for Physical AI. [paper]
arXiv 2025 CuES: A Curiosity-driven and Environment-grounded Synthesis Framework for Agentic RL. [paper]
arXiv 2025 Back to the Features: DINO as a Foundation for Video World Models. [paper]
arXiv 2026 DIVE: Scaling Diversity in Agentic Task Synthesis for Generalizable Tool Use. [paper]
CoRL 2025 DreamGen: Unlocking Generalization in Robot Learning through Video World Models. [paper]
arXiv 2026 World Action Models are Zero-shot Policies. [paper]
arXiv 2026 FinMTM: A Multi-Turn Multimodal Benchmark for Financial Reasoning and Agent Evaluation. [paper]
arXiv 2026 GEAR: Granularity-Adaptive Advantage Reweighting for LLM Agents via Self-Distillation. [paper]
arXiv 2025 GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulators. [paper]
CVPR 2026 HATS: Hardness-Aware Trajectory Synthesis for GUI Agents. [paper]
arXiv 2026 From Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flaws. [paper]
arXiv 2026 Computer Environments Elicit General Agentic Intelligence in LLMs. [paper]
arXiv 2026 The Long-Horizon Task Mirage? Diagnosing Where and Why Agentic Systems Break. [paper]
ICML 2026 On Training Large Language Models for Long-Horizon Tasks: An Empirical Study of Horizon Length. [paper]
NeurIPS 2025 Workshop (SEA) MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World Tasks via MCP Servers. [paper]
arXiv 2025 MCPVerse: An Expansive, Real-World Benchmark for Agentic Tool Use. [paper]
arXiv 2025 Matrix-Game: Interactive World Foundation Model. [paper]
arXiv 2025 MedAgentGym: A Scalable Agentic Training Environment for Code-Centric Reasoning in Biomedical Data Science. [paper]
arXiv 2025 MineWorld: a Real-Time and Open-Source Interactive World Model on Minecraft. [paper]
arXiv 2026 MobileDreamer: Generative Sketch World Model for GUI Agent. [paper]
ICLR 2026 OSWorld-MCP: Benchmarking MCP Tool Invocation In Computer-Use Agents. [paper]
arXiv 2026 OpenResearcher: A Fully Open Pipeline for Long-Horizon Deep Research Trajectory Synthesis. [paper]
arXiv 2026 PIVOT: Bridging Planning and Execution in LLM Agents via Trajectory Refinement. [paper]
arXiv 2025 PaperArena: An Evaluation Benchmark for Tool-Augmented Agentic Reasoning on Scientific Literature. [paper]
arXiv 2026 SWE-Hub: A Unified Production System for Scalable, Executable Software Engineering Tasks. [paper]
arXiv 2026 SWE-World: Building Software Engineering Agents in Docker-Free Environments. [paper]
arXiv 2026 Immersion in the GitHub Universe: Scaling Coding Agents to Mastery. [paper]
arXiv 2026 SciAgentGym: Benchmarking Multi-Step Scientific Tool-use in LLM Agents. [paper]
arXiv 2026 A Matter of TASTE: Improving Coverage and Difficulty of Agent Benchmarks. [paper]
arXiv 2026 Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces. [paper]
arXiv 2026 Terminal-World: Scaling Terminal-Agent Environments via Agent Skills. [paper]
arXiv 2025 LLMs as Scalable, General-Purpose Simulators For Evolving Digital Agent Training. [paper]
arXiv 2026 Verifiable Process Rewards for Agentic Reasoning. [paper]
arXiv 2026 Safe and Scalable Web Agent Learning via Recreated Websites. [paper]
arXiv 2025 WebSailor: Navigating Super-human Reasoning for Web Agent. [paper]
arXiv 2025 WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic Data and Scalable Reinforcement Learning. [paper]
arXiv 2026 WebWorld: A Large-Scale World Model for Web Agent Training. [paper]
arXiv 2026 Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application. [paper]
NeurIPS 2025 Workshop (SEA) Environment Scaling for Interactive Agentic Experience Collection: A Survey. [paper]
arXiv 2026 Memory-R2: Fair Credit Assignment for Long-Horizon Memory-Augmented LLM Agents. [paper]
NeurIPS 2025 seq-JEPA: Autoregressive Predictive Learning of Invariant-Equivariant World Models. [paper]
ICLR 2026 MCPMark: A Benchmark for Stress-Testing Realistic and Comprehensive MCP Use. [paper]
arXiv 2026 Qwen-AgentWorld: Language World Models for General Agents. [paper]

Pre-training and Mid-training

arXiv 2024 Qwen2.5 Technical Report. [paper] [code]
arXiv 2024 DeepSeek-V3 Technical Report. [paper] [code]
arXiv 2025 Kimi K2: Open Agentic Intelligence. [paper] [code]
ICLR 2024 YaRN: Efficient Context Window Extension of Large Language Models. [paper] [code]
ICLR 2024 LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models. [paper] [code]
ACL 2025 How to Train Long-Context Language Models (Effectively). [paper] [code]
arXiv 2024 Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution. [paper] [code]
arXiv 2025 InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models. [paper] [code]
NeurIPS 2023 DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining. [paper]
arXiv 2026 BabyVision: Visual Reasoning Beyond Language. [paper]
ICLR 2026 Vision Language Models are Biased. [paper]
ACCV 2024 Vision language models are blind. [paper]
CVPR 2024 Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs. [paper]
NeurIPS 2024 Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs. [paper]
arXiv 2024 Michelangelo: Long Context Evaluations Beyond Haystacks via Latent Structure Queries. [paper]
arXiv 2026 CL-bench: A Benchmark for Context Learning. [paper]
arXiv 2026 GLM-5: from Vibe Coding to Agentic Engineering. [paper]
arXiv 2023 Extending Context Window of Large Language Models via Positional Interpolation. [paper]
arXiv 2025 MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttraining. [paper]
arXiv 2025 MiMo-VL Technical Report. [paper]
arXiv 2025 Seed1.5-VL Technical Report. [paper]
arXiv 2026 Kimi K2.5: Visual Agentic Intelligence. [paper]
arXiv 2026 Qwen3.5-Omni Technical Report. [paper]
arXiv 2026 Qwen3-Coder-Next Technical Report. [paper]
arXiv 2025 InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency. [paper]
ICML 2026 OPUS: Towards Efficient and Principled Data Selection in Large Language Model Pre-training in Every Iteration. [paper]
NeurIPS 2022 Training Compute-Optimal Large Language Models. [paper]

Fine-tuning

ACL Findings 2024 AgentTuning: Enabling Generalized Agent Abilities for LLMs. [paper] [code]
arXiv 2025 LIMI: Less is More for Agency. [paper] [code]
ACL Findings 2025 ATLaS: Agent Tuning via Learning Critical Steps. [paper]
COLM 2025 LIMO: Less is More for Reasoning. [paper] [code]
EMNLP 2025 s1: Simple Test-Time Scaling. [paper] [code]
ICML 2024 Executable Code Actions Elicit Better LLM Agents. [paper] [code]
NeurIPS 2024 APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets. [paper] [code]
arXiv 2025 Agent-R: Training Language Model Agents to Reflect via Iterative Self-Training. [paper] [code]
NeurIPS 2025 Distilling LLM Agent into Small Models with Retrieval and Code Tools. [paper] [code]
arXiv 2023 FireAct: Toward Language Agent Fine-tuning. [paper]
arXiv 2025 LIMR: Less is More for RL Scaling. [paper]
arXiv 2026 Unified Data Selection for LLM Reasoning. [paper]
AAMAS 2026 Structured Agent Distillation for Large Language Model Agents. [paper]
arXiv 2026 HERO: Hindsight-Enhanced Reflection from Environment Observations for Agentic Self-Distillation. [paper]
arXiv 2026 AgentArk: Distilling Multi-Agent Intelligence into a Single LLM Agent. [paper]
ICLR 2026 Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoning. [paper]
ICLR 2026 Agent Data Protocol: Unifying Datasets for Diverse, Effective Fine-tuning of LLM Agents. [paper]
arXiv 2026 Learning with Rare Success but Rich Feedback via Reflection-Enhanced Self-Distillation. [paper]

Agentic Reinforcement Learning

Credit assignment · policy optimization · sampling strategy · interaction patterns. GitHub links follow the paper's Table (Agentic RL).

Credit Assignment - arXiv 2024 DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. [paper] [code] - COLM 2025 Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning. [paper] [code] - arXiv 2025 DeepRetrieval: Hacking Real Search Engines and Retrievers with Large Language Models via Reinforcement Learning. [paper] [code] - NeurIPS 2025 ToolRL: Reward is All Tool Learning Needs. [paper] [code] - SIGIR 2026 Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learning. [paper] [code] - arXiv 2026 Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoning. [paper] [code] - arXiv 2025 DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research. [paper] [code] - ACL 2026 OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignment. [paper] [code] - ACL 2026 CriticSearch: Fine-Grained Credit Assignment for Search Agents via a Retrospective Critic. [paper] - arXiv 2025 ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agents. [paper] - arXiv 2025 R3: Robust Rubric-Agnostic Reward Models. [paper] - arXiv 2025 Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains. [paper] - arXiv 2026 AdaTIR: Adaptive Tool-Integrated Reasoning via Difficulty-Aware Policy Optimization. [paper] - arXiv 2025 Chasing the Tail: Effective Rubric-based Reward Modeling for Large Language Model Post-Training. [paper] - arXiv 2025 Reinforcement Learning for Long-Horizon Interactive LLM Agents. [paper] - arXiv 2025 DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. [paper] - arXiv 2025 EAPO: Enhancing Policy Optimization with On-Demand Expert Assistance. [paper] - arXiv 2025 Reinforcement Learning with Rubric Anchors. [paper] - arXiv 2026 SmartSearch: Process Reward-Guided Query Refinement for Search Agents. [paper] - arXiv 2025 Tool-R1: Sample-Efficient Reinforcement Learning for Agentic Tool Use. [paper] - arXiv 2025 ToRL: Scaling Tool-Integrated RL. [paper] - ICML 2026 Optimizing Agentic Reasoning with Retrieval via Synthetic Semantic Information Gain Reward. [paper] - arXiv 2026 Self-Induced Outcome Potential: Turn-Level Credit Assignment for Agents without Verifiers. [paper]

Policy Optimization - arXiv 2025 REINFORCE++: A Simple and Efficient Approach for Aligning Large Language Models. [paper] [code] - NeurIPS 2025 DAPO: An Open-Source LLM Reinforcement Learning System at Scale. [paper] [code] - arXiv 2025 Understanding R1-Zero-Like Training: A Critical Perspective. [paper] [code] - arXiv 2025 Group Sequence Policy Optimization. [paper] - NIPS 2025 Group-in-Group Policy Optimization for LLM Agent Training. [paper] [code] - EACL 2026 Turn-PPO: Turn-Level Advantage Estimation with PPO for Improved Multi-Turn RL in Agentic LLMs. [paper] - arXiv 2026 StepPO: Step-Aligned Policy Optimization for Agentic Reinforcement Learning. [paper] - arXiv 2025 CURE: Critical-Token-Guided Re-Concatenation for Entropy-Collapse Prevention. [paper] [code] - arXiv 2025 EPO: Entropy-regularized Policy Optimization for LLM Agents Reinforcement Learning. [paper] [code] - EMNLP 2025 EIFBENCH: Extremely Complex Instruction Following Benchmark for Large Language Models. [paper] - arXiv 2025 Agentic Reinforced Policy Optimization. [paper] - arXiv 2025 On Entropy Control in LLM-RL Algorithms. [paper] - arXiv 2025 Agentic Entropy-Balanced Policy Optimization. [paper] - arXiv 2025 BNPO: Beta Normalization Policy Optimization. [paper] - arXiv 2025 The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models. [paper] - arXiv 2025 CPGD: Toward Stable Rule-based Reinforcement Learning for Language Models. [paper] - arXiv 2026 When Denser Credit Is Not Enough: Evidence-Calibrated Policy Optimization for Long-Horizon LLM Agent Training. [paper] - arXiv 2025 GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoning. [paper] - arXiv 2026 Hindsight Credit Assignment for Long-Horizon LLM Agents. [paper] - arXiv 2026 Hierarchy-of-Groups Policy Optimization for Long-Horizon Agentic Tasks. [paper] - arXiv 2026 HiPER: Hierarchical Reinforcement Learning with Explicit Credit Assignment for Large Language Model Agents. [paper] - arXiv 2025 MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention. [paper] - arXiv 2025 On-Policy RL with Optimal Reward Baseline. [paper] - arXiv 2025 PORTool: Tool-Use LLM Training with Rewarded Tree. [paper] - arXiv 2017 Proximal Policy Optimization Algorithms. [paper]

Sampling Strategy - NeurIPS 2025 WebDancer: Towards Autonomous Information Seeking Agency. [paper] [code] - ICLR 2025 WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning. [paper] [code] - ICLR 2026 Tree Search for LLM Agent Reinforcement Learning. [paper] [code] - arXiv 2025 TreePO: Bridging the Gap of Policy Optimization and Efficacy and Inference Efficiency with Heuristic Tree-based Modeling. [paper] [code] - EMNLP 2023 Reasoning with Language Model is Planning with World Model. [paper] [code] - arXiv 2026 LiteResearcher: A Scalable Agentic RL Training Framework for Deep Research Agent. [paper] [code] - COLM 2026 TCOD: Exploring Temporal Curriculum in On-Policy Distillation for Multi-turn Autonomous Agents. [paper] [code] - arXiv 2025 A Survey of Reinforcement Learning for Large Reasoning Models. [paper] - arXiv 2026 AT$^2$PO: Agentic Turn-based Policy Optimization via Tree Search. [paper] - arXiv 2025 R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning. [paper] - arXiv 2025 Toward Effective Tool-Integrated Reasoning via Self-Evolved Preference Learning. [paper] - arXiv 2025 Reinforcement Learning Optimization for Large-Scale Learning: An Efficient and User-Friendly Scaling Library. [paper] - arXiv 2025 WebResearcher: Unleashing unbounded reasoning capability in Long-Horizon Agents. [paper] - arXiv 2026 Training Multi-Turn Search Agent via Contrastive Dynamic Branch Sampling. [paper]

Interaction Patterns - ICML 2025 Divide and Conquer: Grounding LLMs as Efficient Decision-Making Agents via Offline Hierarchical Reinforcement Learning. [paper] [code] - arXiv 2026 SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning. [paper] [code] - ICLR 2026 THOR: Tool-Integrated Hierarchical Optimization via RL for Mathematical Reasoning. [paper] [code] - arXiv 2025 Multi-Agent Tool-Integrated Policy Optimization. [paper] [code] - arXiv 2025 Stronger-MAS: Multi-Agent Reinforcement Learning for Collaborative LLMs. [paper] [code] - ACL 2026 Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning. [paper] [code] - arXiv 2025 Agent Lightning: Train ANY AI Agents with Reinforcement Learning. [paper] [code] - arXiv 2025 MARS: Optimizing Dual-System Deep Research via Multi-Agent Reinforcement Learning. [paper] - arXiv 2025 Multi-Agent Deep Research: Training Multi-Agent Systems with M-GRPO. [paper] - arXiv 2025 SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learning. [paper] - arXiv 2025 Unlocking the Power of Multi-Agent LLM for Reasoning: From Lazy Agents to Deliberation. [paper] - arXiv 2026 FlowSteer: Interactive Agentic Workflow Orchestration via End-to-End Reinforcement Learning. [paper]

On-Policy Distillation

IJCNN 2024 Online Policy Distillation with Decision-Attention. [paper]
arXiv 2026 MAD-OPD: Breaking the Ceiling in On-Policy Distillation via Multi-Agent Debate. [paper] [code]
arXiv 2026 KAT-Coder-V2 Technical Report. [paper]
arXiv 2026 LiteGUI: Distilling Compact GUI Agents with Reinforcement Learning. [paper]
arXiv 2026 Revisiting DAgger in the Era of LLM-Agents. [paper]
arXiv 2026 SOD: Step-wise On-policy Distillation for Small Language Model Agents. [paper] [code]
arXiv 2026 StepOPSD: Step-Aware Online Preference Distillation for Agent Reinforcement Learning. [paper]
arXiv 2026 Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation. [paper] [code]
arXiv 2026 Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agents. [paper] [code]
arXiv 2026 π-Play: Multi-Agent Self-Play via Privileged Self-Distillation without External Data. [paper] [code]
arXiv 2026 f-OPD: Stabilizing Long-Horizon On-Policy Distillation with Freshness-Aware Control. [paper]
arXiv 2026 Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervision. [paper] [code]
arXiv 2026 Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes. [paper] [code]
arXiv 2026 TIP: Token Importance in On-Policy Distillation. [paper]
arXiv 2026 Reinforcement Learning via Self-Distillation. [paper]
arXiv 2026 Skill-Conditioned Gated Self-Distillation for LLM Reasoning. [paper]
arXiv 2026 ToolAnchor: Anchoring Counterfactual Context to Boost Agentic Tool-use Capability. [paper]

Self-Evolution

NeurIPS 2022 STaR: Bootstrapping Reasoning with Reasoning. [paper] [code]
ICML 2025 Self-Improving Language Models for Evolutionary Program Synthesis: A Case Study on ARC-AGI. [paper] [code]
EMNLP 2025 SAMULE: Self-Learning Agents Enhanced by Multi-level Reflection. [paper]
ICML 2025 rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking. [paper] [code]
ICLR 2026 R-Zero: Self-Evolving Reasoning LLM from Zero Data. [paper] [code]
NeurIPS 2025 Absolute Zero: Reinforced Self-play Reasoning with Zero Data. [paper] [code]
ICLR 2026 Workshop (RSI Oral) Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning. [paper]
ICLR 2026 CoMAS: Co-Evolving Multi-Agent Systems via Interaction Rewards. [paper]
ICML 2026 RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments. [paper]
arXiv 2026 Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence. [paper]
ICLR 2026 Don't Just Fine-tune the Agent, Tune the Environment. [paper]
arXiv 2025 A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems. [paper]
ICLR 2025 Workshop (Self-Improving Foundation Models Oral) A Self-Improving Coding Agent. [paper]
NeurIPS 2025 Self-Adapting Language Models. [paper]
arXiv 2025 RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning. [paper]
arXiv 2025 Socratic-Zero : Bootstrapping Reasoning via Data-Free Agent Co-evolution. [paper]
arXiv 2026 Socratic-SWE: Self-Evolving Coding Agents via Trace-Derived Agent Skills. [paper]
arXiv 2025 AgentEvolver: Towards Efficient Self-Evolving Agent System. [paper]
arXiv 2026 OpenSkill: Open-World Self-Evolution for LLM Agents. [paper]

Applications: Long-Horizon Agents in Practice

Applications grouped by agent-environment interface

Section figure. Representative long-horizon agent applications grouped by the agent–environment interface.

Software Engineering

arXiv 2026 Hackers or Hallucinators? A Comprehensive Analysis of LLM-Based Automated Penetration Testing. [paper] [code]
arXiv 2026 ProgramBench: Can Language Models Rebuild Programs From Scratch? [paper]
arXiv 2025 CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale. [paper]
arXiv 2025 ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluation. [paper]
arXiv 2026 SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution. [paper]
arXiv 2026 RepoZero: Can LLMs Generate a Code Repository from Scratch? [paper] [code]

Repository grounding - NeurIPS 2024 SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering. [paper] [code] - ISSTA 2024 AutoCodeRover: Autonomous Program Improvement. [paper] [code] - ICLR 2025 OpenHands: An Open Platform for AI Software Developers as Generalist Agents. [paper] [code] - 2023 aider is AI pair programming in your terminal. [code] - ICLR 2024 SWE-bench: Can Language Models Resolve Real-World GitHub Issues? [paper] [code] - ICML 2026 NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents. [paper] [code] [paper]

Workflow-level planning - 2025 Claude Code. [code] - 2025 Deep Agents: The Batteries-Included Agent Harness. [code] - arXiv 2025 Trae Agent: An LLM-based Agent for Software Engineering with Test-time Scaling. [paper] [code] - ICML 2026 SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?. [paper] [code] - Anthropic Engineering 2026 Scaling Managed Agents: Decoupling the brain from the hands. [paper] - arXiv 2025 AInsteinBench: Benchmarking Coding Agents on Scientific Repositories. [paper] - ICML 2025 SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering? [paper]

Feedback-driven repair - ICML 2025 Training Software Engineering Agents and Verifiers with SWE-Gym. [paper] [code] - NeurIPS 2025 SWE-smith: Scaling Data for Software Engineering Agents. [paper] [code] - FSE 2025 Demystifying LLM-Based Software Engineering Agents. [paper] [code] - NeurIPS 2025 Workshop (SEA Poster) When Agents go Astray: Course-Correcting SWE Agents with PRMs. [paper] - arXiv 2025 SWE-RM: Execution-free Feedback for Software Engineering Agents. [paper]

Information Seeking

NAACL 2025 Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation. [paper]

Deep search - EMNLP 2025 Search-o1: Agentic Search-Enhanced Large Reasoning Models. [paper] [code] - COLM 2025 Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning. [paper] [code] - NeurIPS 2025 WebDancer: Towards Autonomous Information Seeking Agency. [paper] [code] - arXiv 2025 WebSeer: Training Deeper Search Agents through RL with Self-Reflection. [paper] [code] - ICLR 2026 ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarization. [paper] [code] - arXiv 2025 Tongyi DeepResearch Technical Report. [paper] [code] - arXiv 2025 MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling. [paper] [code] - ICLR 2026 FinSearchComp: Towards a Realistic, Expert-Level Evaluation of Financial Search and Reasoning. [paper] - arXiv 2025 BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinese. [paper] - arXiv 2026 Retrieval-Infused Reasoning Sandbox: A Benchmark for Decoupling Retrieval and Reasoning Capabilities. [paper] - ICLR 2026 SealQA: Raising the Bar for Reasoning in Search-Augmented Language Models. [paper] - arXiv 2025 EviNote-RAG: Enhancing RAG Models via Answer-Supportive Evidence Notes. [paper] - arXiv 2025 Deep Research Agents: A Systematic Examination And Roadmap. [paper] - arXiv 2025 Atom-Searcher: Enhancing Agentic Deep Research via Fine-Grained Atomic Thought Reward. [paper] - arXiv 2025 FoldAct: Efficient and Stable Context Folding for Long-Horizon Search Agents. [paper] - arXiv 2025 A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applications. [paper] - ICML 2026 ReSeek: A Self-Correcting Framework for Search Agents with Instructive Rewards. [paper] - arXiv 2025 Beyond Turn Limits: Training Deep Search Agents with Dynamic Context Window. [paper]

Wide search - ICLR 2026 WideSearch: Benchmarking Agentic Broad Info-Seeking. [paper] [code] - 2025 Open Deep Research. [paper] [code] - ACL 2026 FlowSearch: Advancing Deep Research with Dynamic Structured Knowledge Flow. [paper] [code] - arXiv 2025 Laser: Governing Long-Horizon Agentic Search via Structured Protocol and Context Register. [paper] - arXiv 2025 GraphSearch: An Agentic Deep Searching Workflow for Graph Retrieval-Augmented Generation. [paper] - EMNLP 2024 AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks? [paper] [code] - arXiv 2026 WebSwarm: Recursive Multi-Agent Orchestration for Deep-and-Wide Web Search. [paper]

arXiv 2026 WideSeek: Advancing Wide Research via Multi-Agent Scaling. [paper]
arXiv 2026 InfoSeeker: A Scalable Hierarchical Parallel Agent Framework for Web Information Seeking. [paper]
arXiv 2026 WideSeek-R1: Exploring Width Scaling for Broad Information Seeking via Multi-Agent Reinforcement Learning. [paper]

Multimodal grounding - ACL 2024 WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models. [paper] [code] - ICLR 2025 Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agent. [paper] - ICLR 2026 WebWatcher: Breaking New Frontiers of Vision-Language Deep Research Agent. [paper] [code] - arXiv 2025 MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agents. [paper] - arXiv 2026 VSearcher: Long-Horizon Multimodal Search Agent via Reinforcement Learning. [paper] - arXiv 2026 Towards Long-Horizon Agentic Multimodal Search. [paper] - Findings of ACL 2025 MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answering. [paper] - CVPR 2025 OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations. [paper] - ICCV 2023 Document Understanding Dataset and Evaluation (DUDE). [paper] - NeurIPS 2025 OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning. [paper] - NeurIPS 2024 MMLONGBENCH-DOC: Benchmarking Long-context Document Understanding with Visualizations. [paper] - ACL 2025 LongDocURL: a Comprehensive Multimodal Long Document Benchmark Integrating Understanding, Reasoning, and Locating. [paper] - arXiv 2026 OpenSearch-VL: An Open Recipe for Frontier Multimodal Search Agents. [paper] - arXiv 2025 Agentic Keyframe Search for Video Question Answering. [paper] - arXiv 2026 Video-Browser: Towards Agentic Open-web Video Browsing. [paper] - arXiv 2026 Watching, Reasoning, and Searching: A Video Deep Research Benchmark on Open Web for Agentic Video Reasoning. [paper] - Visual Intelligence 2025 Large multimodal agents: a survey. [paper] - SIGIR 2026 Towards Mixed-Modal Retrieval for Universal Retrieval-Augmented Generation. [paper]

Research synthesis - ICLR 2026 WebWeaver: Structuring Web-Scale Evidence with Dynamic Outlines for Open-Ended Deep Research. [paper] [code] - NeurIPS 2025 WebThinker: Empowering Large Reasoning Models with Deep Research Capability. [paper] [code] - arXiv 2025 BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents. [paper] [code] - ICLR 2026 Towards Personalized Deep Research: Benchmarks and Evaluations. [paper]

ICLR 2026 DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents. [paper] [code]
NeurIPS 2024 Long-form factuality in large language models. [paper]
EMNLP 2023 FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation. [paper]
ACL 2026 FS-Researcher: Test-Time Scaling for Long-Horizon Research Tasks with File-System-Based Agents. [paper]
arXiv 2025 DeepTRACE: Auditing Deep Research AI Systems for Tracking Reliability Across Citations and Evidence. [paper]
arXiv 2026 AgentCPM-Report: Interleaving Drafting and Deepening for Open-Ended Deep Research. [paper]
arXiv 2025 Deep Research: A Survey of Autonomous Research Agents. [paper]
arXiv 2026 Toward Generalist Autonomous Research via Hypothesis-Tree Refinement. [paper]

Computer Use

arXiv 2026 Workspace-Bench 1.0: Benchmarking AI Agents on Workspace Tasks with Large-Scale File Dependencies. [paper]
arXiv 2026 WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation. [paper]

Browser agents - ICLR 2024 WebArena: A Realistic Web Environment for Building Autonomous Agents. [paper] [code] - NeurIPS 2023 Mind2Web: Towards a Generalist Agent for the Web. [paper] [code] - ACL 2024 WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models. [paper] [code] - 2024 browser-use: Make Websites Accessible for AI Agents. [code] - TMLR 2026 BrowserAgent: Building Web Agents with Human-Inspired Web Browsing Actions. [paper] [code] - EMNLP 2024 AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?. [paper] [code] - arXiv 2026 ClawBench: Can AI Agents Complete Everyday Online Tasks? [paper] - arXiv 2025 Recon-Act: A Self-Evolving Multi-Agent Browser-Use System via Web Reconnaissance, Tool Generation, and Task Execution. [paper] - arXiv 2025 WEBSERV: A Full-Stack and RL-Ready Web Environment for Training Web Agents at Scale. [paper] - ICLR 2025 Workshop (SSI-FM Poster) NNetNav: Unsupervised Learning of Browser Agents Through Environment Interaction in the Wild. [paper] - ACL 2026 Industry Track ColorBrowserAgent: Complex Long-Horizon Browser Agent with Adaptive Knowledge Evolution. [paper]

Desktop GUI agents - NeurIPS 2024 OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments. [paper] [code] - arXiv 2025 UI-TARS: Pioneering Automated GUI Interaction with Native Agents. [paper] [code] - CVPR 2024 CogAgent: A Visual Language Model for GUI Agents. [paper] [code] - ACM MM 2025 ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use. [paper] [code] - 2024 Anthropic Computer Use (Quickstarts). [code] - ICLR 2025 Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents. [paper] [code] - arXiv 2026 Workflow-GYM: Towards Long-Horizon Evaluation of Computer-use Agentic tasks in Real-World Professional Fields. [paper] - arXiv 2026 VISUALSKILL: Multimodal Skills for Computer-Use Agents. [paper] - arXiv 2026 AliyunConsoleAgent: Training Web Agents in Real-World Cloud Environments via Distillation and Reinforcement Learning. [paper] - NeurIPS 2025 GUI-Actor: Coordinate-Free Visual Grounding for GUI Agents. [paper]

Mobile agents - ICLR 2024 Workshop Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perception. [paper] [code] - ICLR 2025 AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents. [paper] [code] - ACL 2024 SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents. [paper] [code] - arXiv 2025 Mobile-Agent-v3: Fundamental Agents for GUI Automation. [paper] [code] - NeurIPS 2025 Workshop (SEA Oral) Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks. [paper] [code] - ACL 2026 MobileWorld: Benchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environments. [paper] - MobiCom 2026 V-Droid: Advancing Mobile GUI Agent Through Generative Verifiers. [paper] - arXiv 2025 MobileGUI-RL: Advancing Mobile GUI Agent through Reinforcement Learning in Online Environment. [paper] - arXiv 2025 AutoEval: A Practical Framework for Autonomous Evaluation of Mobile Agents. [paper] - arXiv 2026 Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agents. [paper]

Multimodal Agents

arXiv 2024 Vibe-Eval: A hard evaluation suite for measuring progress of multimodal language models. [paper]
ICCV 2025 SimpleVQA: Multimodal Factuality Evaluation for Multimodal Large Language Models. [paper]
CVPR 2024 HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models. [paper]
NeurIPS 2025 MMLongBench: Benchmarking Long-Context Vision-Language Models Effectively and Thoroughly. [paper]
AAAI 2026 Video SimpleQA: Towards Factuality Evaluation in Large Video Language Models. [paper]
ICLR 2025 CG-Bench: Clue-grounded Question Answering Benchmark for Long Video Understanding. [paper]
TMLR 2026 VideoEval-Pro: Robust and Realistic Long Video Understanding Evaluation. [paper]
AAAI 2026 CrossVid: A Comprehensive Benchmark for Evaluating Cross-Video Reasoning in Multimodal Large Language Models. [paper]
CVPR 2024 Workshop (SynData4CV; Best Short Paper) GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generation. [paper]
arXiv 2024 Agent AI: Surveying the Horizons of Multimodal Interaction. [paper]

Multimodal understanding - ECCV 2024 VideoAgent: Long-form Video Understanding with Large Language Model as Agent. [paper] [code] - NeurIPS 2025 Deep Video Discovery: Agentic Search with Tool Use for Long-form Video Understanding. [code] [paper] - KDD 2026 VideoRAG: Retrieval-Augmented Generation with Extreme Long-Context Videos. [paper] - CVPR 2025 Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis. [paper] [code] - arXiv 2025 Qwen2.5-VL Technical Report. [paper] [code] - CVPR 2026 VideoSeek: Long-Horizon Video Agent with Tool-Guided Seeking. [paper] [code] - NeurIPS 2025 StreamForest: Efficient Online Video Understanding with Persistent Event Memory. [paper] - NeurIPS 2025 AdaVideoRAG: Omni-Contextual Adaptive Retrieval-Augmented Efficient Long Video Understanding. [paper] - ICLR 2026 A.I.R.: Enabling Adaptive, Iterative, and Reasoning-based Frame Selection For Video Question Answering. [paper] - arXiv 2026 AgentVista: Evaluating Multimodal Agents in Ultra-Challenging Realistic Visual Scenarios. [paper] - arXiv 2026 LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMs. [paper]

Multimodal generation - arXiv 2025 MM-StoryAgent: Immersive Narrated Storybook Video Generation with a Multi-Agent Paradigm across Text, Image and Audio. [paper] [code] - arXiv 2026 Qwen-Image-2.0 Technical Report. [paper] - arXiv 2025 Seedream 3.0 Technical Report. [paper] - arXiv 2026 MUSE: A Multi-agent Framework for Unconstrained Story Envisioning via Closed-Loop Cognitive Orchestration. [paper] - arXiv 2026 Mind-Brush: Integrating Agentic Cognitive Search and Reasoning into Image Generation. [paper] - arXiv 2026 Towards Verifiable Multimodal Deep Research: A Multi-Agent Harness for Interleaved Report Generation. [paper] [code] - arXiv 2025 Generative Universal Verifier as Multimodal Meta-Reasoner. [paper] - arXiv 2025 SeedEdit 3.0: Fast and High-Quality Generative Image Editing. [paper] - arXiv 2026 GenAgent: Scaling Text-to-Image Generation via Agentic Multimodal Reasoning. [paper] - arXiv 2025 Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling. [paper] - arXiv 2026 MemoGen: Can Past Experience Improve Future Text-to-Image Generation?. [paper] - arXiv 2026 Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesis. [paper] - arXiv 2025 Emerging Properties in Unified Multimodal Pretraining. [paper] - arXiv 2026 Seedance 2.0: Advancing Video Generation for World Complexity. [paper] - arXiv 2026 Crayotter: Traceable Multi-Agent Workflows for Long-Form Video Editing. [paper]

Omnimodal agency - arXiv 2025 Qwen3-Omni Technical Report. [paper] [code] - arXiv 2025 Agent-Omni: Test-Time Multimodal Reasoning via Model Coordination for Understanding Anything. [paper] [code] - arXiv 2026 OmniGAIA: Towards Native Omni-Modal AI Agents. [code] - TrustCom 2025 OmniNova: A General Multimodal Agent Framework. [paper] - arXiv 2026 Orchestra-o1: Omnimodal Agent Orchestration. [paper] [code] - arXiv 2026 Agentic Active Omni-Modal Perception for Multi-Hop Audio-Visual Reasoning. [paper] - ICML 2026 Native Active Perception as Reasoning for Omni-Modal Understanding. [paper] - arXiv 2025 Qwen2.5-Omni Technical Report. [paper]

General-Purpose Agents

Personal assistants - 2023 AutoGPT: Autonomous Goal Pursuit. [code] - 2025 Manus: A General Autonomous Agent Product. [paper] - ICLR 2024 GAIA: a benchmark for General AI Assistants. [paper] - arXiv 2026 SemaClaw: A Step Towards General-Purpose Personal AI Agents through Harness Engineering. [paper] - arXiv 2026 APEX-Agents. [paper] - 2025 OpenManus: An Open-Source Framework for Building General AI Agents. [code] - 2025 Claude Code overview. [paper] - 2025 Introducing Codex. [paper] - arXiv 2026 PresentBench: A Fine-Grained Rubric-Based Benchmark for Slide Generation. [paper] - NeurIPS 2024 SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulation. [paper] - arXiv 2026 Claw-Eval: Towards Trustworthy Evaluation of Autonomous Agents. [paper] - arXiv 2026 ClawMark: A Living-World Benchmark for Multi-Turn, Multi-Day, Multimodal Coworker Agents. [paper] - arXiv 2026 MCP-Atlas: A Large-Scale Benchmark for Tool-Use Competency with Real MCP Servers. [paper] - arXiv 2026 OfficeQA Pro: An Enterprise Benchmark for End-to-End Grounded Reasoning. [paper] - ICLR 2026 The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution. [paper]

Embodied agents and world models - arXiv 2025 Gemini Robotics: Bringing AI into the Physical World. [paper] - arXiv 2025 GR00T N1: An Open Foundation Model for Generalist Humanoid Robots. [paper] [code] - CoRL 2024 OpenVLA: An Open-Source Vision-Language-Action Model. [paper] [code] - arXiv 2025 π₀.₅: a Vision-Language-Action Model with Open-World Generalization. [paper] [code] - arXiv 2025 V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning. [paper] [code] - ICML 2025 DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning. [paper] [code] - AAAI 2026 Seeing from Another Perspective: Evaluating Multi-View Understanding in MLLMs. [paper] - ICLR 2026 MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence. [paper] - NeurIPS 2025 RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Robotics. [paper] - arXiv 2025 PhyX: Does Your Model Have the "Wits" for Physical Reasoning? [paper] - ICML 2024 ContPhy: Continuum Physical Concept Learning and Reasoning from Videos. [paper] - ICLR 2025 PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding. [paper] - ACL 2024 EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Models. [paper] - arXiv 2025 World Simulation with Video Foundation Models for Physical AI. [paper] - arXiv 2026 Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Models. [paper] - arXiv 2026 Qwen-RobotNav Technical Report: A Scalable Navigation Model Designed for an Agentic Navigation System. [paper] - arXiv 2026 Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments. [paper] - arXiv 2025 A Survey of Robotic Navigation and Manipulation with Physics Simulators in the Era of Embodied AI. [paper] - arXiv 2026 OSCAR: Omni-Embodiment Action-Conditioned World Model for Robotics. [paper] - arXiv 2026 Advancing Omnimodal Embodied Agents from Isolated Skills to Everyday Physical Autonomy. [paper] - arXiv 2026 $\tau_0\text{-}\mathbf{WM}$: A Unified Video-Action World Model for Robotic Manipulation. [paper] - ICML 2025 EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents. [paper]

Productive agents - arXiv 2024 The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery. [paper] [code] - arXiv 2025 AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery. [paper] [code] - arXiv 2025 Kosmos: An AI Scientist for Autonomous Discovery. [paper] [code] - ICAIF 2024 Workshop FinRobot: AI Agent for Equity Research and Valuation with Large Language Models. [paper] [code] - ACL 2025 LegalAgentBench: Evaluating LLM Agents in Legal Domain. [paper] [code] - NEJM AI 2025 MedAgentBench: A Virtual EHR Environment to Benchmark Medical LLM Agents. [paper] [code] - arXiv 2026 OneMillion-Bench: How Far are Language Agents from Human Experts? - arXiv 2025 GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks. [paper] - arXiv 2026 PostTrainBench: Can LLM Agents Automate LLM Post-Training? [paper] - arXiv 2026 HorizonMath: Measuring AI Progress Toward Mathematical Discovery with Automatic Verification. [paper] - arXiv 2026 FrontierScience: Evaluating AI's Ability to Perform Expert-Level Scientific Tasks. [paper] - arXiv 2026 BABE: Biology Arena BEnchmark. [paper] - ICLR 2022 miniF2F: a cross-system benchmark for formal Olympiad-level mathematics. [paper] - NeurIPS 2024 PutnamBench: Evaluating Neural Theorem-Provers on the Putnam Mathematical Competition. [paper] - arXiv 2025 HealthBench: Evaluating Large Language Models Towards Improved Human Health. [paper] - ICML 2025 MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understanding. [paper] - arXiv 2025 Robin: A multi-agent system for automating scientific discovery. [paper] - arXiv 2025 InternAgent: When Agent Becomes the Scientist -- Building Closed-Loop System from Hypothesis to Verification. [paper] - arXiv 2025 Prover Agent: An Agent-Based Framework for Formal Mathematical Proofs. [paper] - arXiv 2025 Seed-Prover 1.5: Mastering Undergraduate-Level Theorem Proving via Learning from Experience. [paper] - arXiv 2026 LawThinker: A Deep Research Legal Agent in Dynamic Environments. [paper] - arXiv 2026 MedCTA: A Benchmark for Clinical Tool Agents. [paper] - arXiv 2025 Finance Agent Benchmark: Benchmarking LLMs on Real-world Financial Research Tasks. [paper] - arXiv 2026 Parthenon Law: A Self-Evolving Legal-Agent Framework. [paper] - arXiv 2026 Benchmark Test-Time Scaling of General LLM Agents. [paper] - arXiv 2026 PhysicianBench: Evaluating LLM Agents in Real-World EHR Environments. [paper] - arXiv 2025 From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discovery. [paper] - ICML 2026 APE-Bench: Evaluating Automated Proof Engineering for Formal Math Libraries. [paper] - arXiv 2025 The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search. [paper]

Benchmarks and Resources

A consolidated set of open-source benchmarks and reusable systems, organized by application domain (matching the paper's resource table). Links point to public code repositories.

Software Engineering

OpenAI 2024 SWE-bench Verified. [code]
arXiv 2025 SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?. [code]
arXiv 2026 Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces. [code]
ACL 2026 OctoBench: Benchmarking Scaffold-Aware Instruction Following in Repository-Grounded Agentic Coding. [code]
Findings of ACL 2024 DebugBench: Evaluating Debugging Capability of Large Language Models. [code]
arXiv 2026 RepoZero: Can LLMs Generate a Code Repository from Scratch? [code]

Information Seeking

EMNLP 2024 AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?. [code]
arXiv 2025 BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents. [code]
ICLR 2026 WideSearch: Benchmarking Agentic Broad Info-Seeking. [code]
ICLR 2026 DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents. [code]
arXiv 2025 Tongyi DeepResearch Technical Report. [code]
GitHub Resource 2025 DeepConsult: A Deep Research Benchmark for Consulting / Business Queries [code]
Findings of ACL 2025 MMInA: Benchmarking Multihop Multimodal Internet Agents. [code]
arXiv 2026 GISA: A Benchmark for General Information-Seeking Assistant. [code]

Computer Use

ICLR 2024 WebArena: A Realistic Web Environment for Building Autonomous Agents. [code]
NeurIPS 2024 OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments. [code]
ACL 2024 SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents. [code]
ICLR 2025 AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents. [code]
ACL 2025 AndroidLab: Training and Systematic Benchmarking of Android Autonomous Agents. [code]
arXiv 2025 PointArena: Probing Multimodal Grounding Through Language-Guided Pointing. [code]

Multimodal Agents

CVPR 2025 Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis. [code]
NeurIPS 2024 LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding. [code]
NeurIPS 2023 EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding. [code]
CVPR 2024 Workshop Evaluating and Improving Compositional Text-to-Visual Generation. [code]
arXiv 2026 OmniGAIA: Towards Native Omni-Modal AI Agents. [code]
CVPR 2024 MVBench: A Comprehensive Multi-modal Video Understanding Benchmark. [code]
ICASSP 2026 StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understanding. [code]
ICLR 2026 OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMs. [code]

General-Purpose Agents

ICLR 2025 τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. [code]
ACL 2024 AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents. [code]
ICML 2025 EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents. [code]
NeurIPS 2025 WorldModelBench: Judging Video Generation Models As World Models. [code]
ACL 2025 LegalAgentBench: Evaluating LLM Agents in Legal Domain. [code]
NEJM AI 2025 MedAgentBench: A Virtual EHR Environment to Benchmark Medical LLM Agents. [code]
arXiv 2026 OneMillion-Bench: How Far are Language Agents from Human Experts?
arXiv 2026 YC-Bench: Benchmarking AI Agents for Long-Term Planning and Consistent Execution. [code]
arXiv 2026 Seed2.0 Model Card: Towards Intelligence Frontier for Real-World Complexity. [code]
ECCV 2020 Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments. [code]
Nature Machine Intelligence 2024 Augmenting large language models with chemistry tools. [code]

Additional autonomy-stressing suites: MLE-bench (ML engineering) [code], PaperBench (paper replication) [code], and the METR time-horizon measurements [code].

Frontiers: Open Problems

We group open problems into four axes spanning nine concrete directions. A recurring thread: the harness, not the model alone, is where much of the next advance must happen.

Axis	Frontier	Core open challenge
I. Evolution	Self-evolving harness & agents	Objective is a hand-set metric; gains stay in-distribution; long runs overfit/drift
	Harness transferability	Models bind to one harness; rankings swing across providers; no standard protocol
	Continual & lifelong learning	External memory is shallow; internal updates risk forgetting
II. Effectiveness	Real-world environment interaction	No direct training in live systems; synthesis & world-models face a fidelity test
	From digital to embodied agents	Timescale conflict; physics/dimensionality gap; coarse-vs-fine feedback
III. Efficiency	Cost- & budget-aware agency	Budget-blind; no calibrated cost sense; no runtime ceilings; no budget↔success law
	Multimodal & omni harness	Multimodality bolted on; heuristic visual-token budgeting; unreliable cross-modal verification
IV. Trustworthiness	Reflection & error robustness	Late failure detection; unreliable intrinsic self-correction; errors compound into goal drift
	Safety & governance	Injected error/hazardous experience reuse; no unified safety standard; self-evolution erodes invariants

Representative references for the frontiers: - ICLR 2024 Large Language Models Cannot Self-Correct Reasoning Yet. [paper] - ICLR 2025 Scaling LLM Test-Time Compute Optimally. [paper] - ICLR 2025 RouteLLM: Learning to Route LLMs with Preference Data. [paper] [code] - NeurIPS 2025 TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks. [paper] [code] - ICLR 2026 Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents. [paper]

Contributing

Contributions are very welcome! Please open a Pull Request to add a missing paper or fix a link. When adding a work, keep the per-line format consistent:

- **`Venue Year`** Title. [[paper](URL)] [[code](URL)]

Guidelines: - Prefer the acceptance venue (e.g., ICML 2024, NeurIPS 2023, ICLR 2025); use arXiv YYYY only when a work has no conference venue. - Place each paper under the subsection that best matches its primary contribution. - Prioritize representative, high-impact works to keep each list readable.

Awesome-Long-Horizon-Agents

News

Introduction

Citation

Table of Contents

Foundations: Formalizing Long-Horizon Agents

Evolution: From Prompting to Runtime

Stage I — Prompt Engineering (2020–2023)

Stage II — Context Engineering (2023–2025)

Stage III — Runtime Harnesses (2025–Present)

Harnesses: Externalizing Long-Horizon Capability (Pillar I)

Loops and Workflows

Context and Memory

Tools, MCP, and Skills

Orchestration

Hooks and Middleware

Verification

Optimization: Internalizing Long-Horizon Capability (Pillar II)

Architectural Substrate

Data and Environment Synthesis

Pre-training and Mid-training

Fine-tuning

Agentic Reinforcement Learning

On-Policy Distillation

Self-Evolution

Applications: Long-Horizon Agents in Practice

Software Engineering

Information Seeking

Computer Use

Multimodal Agents

General-Purpose Agents

Benchmarks and Resources

Software Engineering

Information Seeking

Computer Use

Multimodal Agents

General-Purpose Agents

Frontiers: Open Problems

Contributing

Star History