The original repository for paper collection is [Link].

Inductive Reasoning

Inductive reasoning involves drawing general conclusions from specific observations. The main characteristics of inductive reasoning are its particular-to-general thinking process and the non-uniqueness of its answers. Considering how humans perceive the world, they typically make judgments by drawing analogies from past experiences to current situations, rather than always going through a strictly logical process as in deductive reasoning. We can assume that the inductive mode is key to knowledge generalization and better aligns with human cognition. We give two examples of inductive reasoning in the figure below.

About the Survey

Among all the papers shown in the repository for paper collection ([Link]), our survey only focuses on synthesizing and analyzing those related to inductive tasks and language models, while also including additional relevant works beyond this repository. This is the link to our survey [ARXIV].

If you find any points in our survey worth discussing or notice any mistakes, feel free to open an issue and share your thoughts!

Citation

If you think this survey helps, please cite our paper.

@misc{chen2025surveyinductivereasoninglarge,
      title={A Survey of Inductive Reasoning for Large Language Models}, 
      author={Kedi Chen and Dezhao Ruan and Yuhao Dan and Yaoting Wang and Siyu Yan and Xuecheng Wu and Yinqi Zhang and Qin Chen and Jie Zhou and Liang He and Biqing Qi and Linyang Li and Qipeng Guo and Xiaoming Shi and Wei Zhang},
      year={2025},
      eprint={2510.10182},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2510.10182}, 
}

The Selected Papers

Introduction and Background

• Multitask Learning: A Knowledge-Based Source of Inductive Bias [ICML1993] [paper link]

• Inductive Reasoning and Bounded Rationality [The American Economic Review, 1994] [paper link]

• Inductive policy: The pragmatics of bias selection [Machine Learning, 20, 1995] [paper link]

• Properties ofinductive reasoning [Psychonomic Bulletin & Review, 2000] [paper link]

• The perception of the environment: essays on livelihood, dwelling and skill [Book, 2000] [paper link]

• A General Inductive Approach for Qualitative Data Analysis [American Journal of Evaluation, 27(2), 2003] [paper link]

• Essentials of Logic [Book, 2004] [paper link]

• When Is Inductive Inference Possible? [NIPS2024] [paper link]

• Inductive reasoning in humans and large language models [Cognitive Systems Research, Volume 83, 2024] [paper link]

Enhancement

Post-training

Synthetic Data

• LIME: Learning Inductive Bias for Primitives of Mathematical Reasoning [ICML2021] [paper link]

• Knowledge Base Question Answering for Space Debris Queries [ACL2023] [paper link]

• CESAR: Automatic Induction of Compositional Instructions for Multi-turn Dialogs [EMNLP2023] [paper link]

• Linguistic Rule Induction Improves Adversarial and OOD Robustness in Large Language Models [COLING2024] [paper link]

• ItD: Large Language Models Can Teach Themselves Induction through Deduction [ACL2024] [paper link]

• Code-Driven Inductive Synthesis: Enhancing Reasoning Abilities of Large Language Models with Sequences [Arxiv2025] [paper link]

• In the LLM era, Word Sense Induction remains unsolved [ACL2025] [paper link]

IRL-style Optimization

• A survey of inverse reinforcement learning: Challenges, methods and progress [Artificial Intelligence, 2021] [paper link]

• Query-Dependent Prompt Evaluation and Optimization with Offline Inverse RL [ICLR2024] [paper link]

• Solving the Inverse Alignment Problem for Efficient RLHF [Arxiv2024] [paper link]

• Approximated Variational Bayesian Inverse Reinforcement Learning for Large Language Model Alignment [AAAI2025] [paper link]

• Inverse Reinforcement Learning Meets Large Language Model Post-Training: Basics, Advances, and Opportunities [Arxiv2025] [paper link]

• Your Reward Function for RL is Your Best PRM for Search: Unifying RL and Search-Based TTS [Arxiv2025] [paper link]

Test-time Scaling

• From Reasoning to Learning: A Survey on Hypothesis Discovery and Rule Learning with Large Language Models [Transactions on Machine Learning Research, 2025] [paper link]

Hypothesis Selection

• Feature selection and hypothesis selection models of induction [CogSci, volume 12, 1990] [paper link]

• Private hypothesis selection [Advances in Neural Information Processing Systems, 32, 2019] [paper link]

• Hypothesis Search: Inductive Reasoning with Language Models [ICLR2024] [paper link]

• Hypothesis Search: Inductive Reasoning with Language Models [ICLR2024] [paper link]

• Generating Diverse Hypotheses for Inductive Reasoning [NAACL2025] [paper link]

• Measuring What Matters: Evaluating Ensemble LLMs with Label Refinement in Inductive Coding [ACL2025] [paper link]

Hypothesis Iteration

• From methodology to practice: Inductive iteration in comparative research [Comparative Political Studies, 2015] [paper link]

• Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement [ICLR2024] [paper link]

• ARISE: Iterative Rule Induction and Synthetic Data Generation for Text Classification [NAACL2025] [paper link]

• Patterns Over Principles: The Fragility of Inductive Reasoning in LLMs under Noisy Observations [ACL2025] [paper link]

• IDEA: Enhancing the Rule Learning Ability of Large Language Model Agent through Induction, Deduction, and Abduction [ACL2025] [paper link]

Hypothesis Evolution

• The hypothesis of interactive evolution [Kybernetes, 2011] [paper link]

• Towards continuous scientific data analysis and hypothesis evolution [AAAI2017] [paper link]

• Neuro-Symbolic Hierarchical Rule Induction [ICML2022] [paper link]

• Zero-Shot On-the-Fly Event Schema Induction [EACL2023] [paper link]

• Open-Domain Hierarchical Event Schema Induction by Incremental Prompting and Verification [ACL2023] [paper link]

• PRIMO: Progressive Induction for Multi-hop Open Rule Generation [COLING2024] [paper link]

• Exploring the Evolution-Coupling Hypothesis: Do Enzymes’ Performance Gains Correlate with Increased Dissipation? [Entropy, 2025] [paper link]

Data Augmentation

Human Intervention

• Semi-supervised New Event Type Induction and Event Detection [EMNLP2020] [paper link]

• Large Scale Substitution-based Word Sense Induction [ACL2022] [paper link]

• Human-in-the-Loop Schema Induction [ACL2023] [paper link]

• Semi-supervised New Event Type Induction and Description via Contrastive Loss-Enforced Batch Attention [EACL 2023] [paper link]

• A(More) Realistic Evaluation Setup for Generalisation of Community Models on Malicious Content Detection [NAACL2024] [paper link]

External Knowledge

• “A Little Birdie Told Me ... ”- Inductive Biases for Rumour Stance Detection on Social Media [EMNLP2020] [paper link]

• Knowledge-Enriched Event Causality Identification via Latent Structure Induction Networks [ACL2021] [paper link]

• Bilingual Lexicon Induction via Unsupervised Bitext Construction and Word Alignment [ACL2021] [paper link]

• Video-aided Unsupervised Grammar Induction [NAACL2021] [paper link]

• Fire Burns, Sword Cuts: Commonsense Inductive Bias for Exploration in Text-based Games [ACL2022] [paper link]

• IAG: Induction-Augmented Generation Framework for Answering Reasoning Questions [EMNLP2023] [paper link]

• Limitations of Language Models in Arithmetic and Symbolic Induction [ACL2023] [paper link]

• Re-evaluating the Need for Multimodal Signals in Unsupervised Grammar Induction [NAACL2024] [paper link]

• How Lexicalis Bilingual Lexicon Induction? [NAACL2024] [paper link]

• INDUCT-LEARN: Short Phrase Prompting with Instruction Induction [EMNLP2024] [paper link]

• Inductive Linguistic Reasoning with Large Language Models [ACL2025] [paper link]

• Decision Tree Induction Through LLMs via Semantically-Aware Evolution [ICLR2025] [paper link]

Structured Signals

• Probing as Quantifying Inductive Bias [ACL2022] [paper link]

• On Bilingual Lexicon Induction with Large Language Models [EMNLP2023] [paper link]

• Learning Query Adaptive Anchor Representation for Inductive Relation Prediction [ACL2023] [paper link]

• Learning Rule-Induced Subgraph Representations for Inductive Relation Prediction [NIPS2023] [paper link]

• Leveraging Grammar Induction for Language Understanding and Generation [EMNLP2024] [paper link]

Benchmarks

Benchmark Name	Paper Name	Paper Link
SCAN	Human few-shot learning of compositional instructions	paper link
ARC	On the Measure of Intelligence	paper link
List Functions	The child as hacker: building more human-like models of learning	paper link
PROGES	A large-scale benchmark for few-shot program induction and synthesis	paper link
SyGuS	BUSTLE: bottom-up program synthesis through learningguided exploration	paper link
ACRE	ACRE: Abstract Causal REasoning Beyond Covariation	paper link
ILP	Neuro-Symbolic Hierarchical Rule Induction	paper link
Instructions	Instruction Induction: From Few Examples to Natural Language Task Description	paper link
Arithmetics	Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks	paper link
Levy/Holt	Explicit Inductive Inference using Large Language Models	paper link
NutFrame	NutFrame: Frame-based Conceptual Structure Induction with LLMs	paper link
DEER	Language Models as Inductive Reasoners	paper link
RULEARN	IDEA: Enhancing the Rule Learning Ability of Large Language Model Agent through Induction, Deduction, and Abduction	paper link
Cryptography	Patterns Over Principles: The Fragility of Inductive Reasoning in LLMs under Noisy Observations	paper link
GeoILP	Geoilp: A synthetic dataset to guide large-scale rule induction	paper link
InductionBench	InductionBench: LLMs Fail in the Simplest Complexity Class	paper link
CodeSeq	Code-driven Number Sequence Calculation: Enhancing the inductive Reasoning Abilities of Large Language Models	paper link

Analysis

• Demystifying Inductive Biases for (Beta-)VAE Based Architectures [ICML2021] [paper link]

• Effects of Parameter Norm Growth During Transformer Training: Inductive Bias from Gradient Descent [EMNLP2021] [paper link]

• Examining the Inductive Bias of Neural Language Models with Artificial Languages [ACL2021] [paper link]

• What they do when in doubt: a study of inductive biases in seq2seq learners [ICLR2021] [paper link]

• Predicting Inductive Biases of Pre-Trained Models [ICLR2021] [paper link]

• The Inductive Bias of ReLU Networks on Orthogonally Separable Data [ICLR2021] [paper link]

• Inductive Biases and Variable Creation in Self-Attention Mechanisms [ICML2022] [paper link]

• Fast Rates for Noisy Interpolation Require Rethinking the Effects of Inductive Bias [ICML2022] [paper link]

• The SSL Interplay: Augmentations, Inductive Bias, and Generalization [ICML2023] [paper link]

• Injecting structural hints: Using language models to study inductive biases in language learning [EMNLP2023] [paper link]

• Inductive Bias Is in the Eye of the Beholder [EMNLP2023] [paper link]

• Measuring Inductive Biases of In-Context Learning with Underspecified Demonstrations [ACL2023] [paper link]

• Instruction Induction: From Few Examples to Natural Language Task Descriptions [ACL2023] [paper link]

• How to Plant Trees in LMs: Data and Architectural Effects on the Emergence of Syntactic Inductive Biases [ACL2023] [paper link]

• A theoretical study of inductive biases in contrastive learning [ICLR2023] [paper link]

• Strong inductive biases provably prevent harmless interpolation [ICLR2023] [paper link]

• Identifying Semantic Induction Heads to Understand In-Context Learning [ACL2024] [paper link]

• The Evolution of Statistical Induction Heads: In-Context Learning Markov Chains [NIPS2024] [paper link]

• Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers [NIPS2024] [paper link]

• Inductive biases of multi-task learning and finetuning: multiple regimes of feature reuse [NIPS2024] [paper link]

• What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation [ICML2024] [paper link]

• Position: The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learning [ICML2024] [paper link]

• Induction Heads as an Essential Mechanism for Pattern Matching in In-context Learning [NAACL2025] [paper link]

• Can Input Attributions Explain Inductive Reasoning in In-Context Learning? [ACL2025] [paper link]

• Beyond Induction Heads: In-Context Meta Learning Induces Multi-Phase Circuit Emergence [ICML2025] [paper link]

• When Diffusion Models Memorize: Inductive Biases in Probability Flow of Minimum-Norm Shallow Neural Nets [ICML2025] [paper link]

• Geometric Inductive Biases of Deep Networks: The Role of Data and Architecture [ICLR2025] [paper link]

Star History