Active Learning

Mailing List

There is a group intended for announcements and discussions regarding active learning research.

Subscribe to Active Learning (Machine Learning)

Email:

Visit this group

Active Learning Literature Survey

Burr Settles
Computer Sciences Technical Report 1648
University of Wisconsin-Madison

This is an online publication that will be updated periodically to reflect new advances in the field of active learning. Please cite the survey in your work as suggested in Section 1. Your feedback is highly welcome to alleviate errors, and to incorporate material that is either new or has been overlooked in the current version. Please send comments to bsettles@cs.cmu.edu.

Abstract

The key idea behind active learning is that a machine learning algorithm can achieve greater accuracy with fewer training labels if it is allowed to choose the data from which it learns. An active learner may pose queries, usually in the form of unlabeled data instances to be labeled by an oracle (e.g., a human annotator). Active learning is well-motivated in many modern machine learning problems, where unlabeled data may be abundant or easily obtained, but labels are difficult, time-consuming, or expensive to obtain.

This report provides a general introduction to active learning and a survey of the literature. This includes a discussion of the scenarios in which queries can be formulated, and an overview of the query strategy frameworks proposed in the literature to date. An analysis of the empirical and theoretical evidence for successful active learning, a summary of problem setting variants and practical issues, and a discussion of related topics in machine learning research are also presented.

Archived Versions: 09-jan-2009

Other Links and Resources

Software

• DUALIST: Utility for Active Learning with Instances and Semantic Terms
• Vowpal Wabbit - contains selective sampling algorithms

Recent Workshops

• Combining Learning Strategies to Reduce Label Cost (ICML 2011)
• Active Learning and Experimental Design (AISTATS 2010)
• Active Learning for NLP (NAACL-HLT 2010)
• Budgeted Learning Workshop (ICML 2010)
• Towards Closing the Loop: Active Learning for Robotics (RSS 2010)
• Active Learning for NLP (NAACL-HLT 2009)
• Adaptive Sensing, Active Learning, and Experimental Design (NIPS 2009)
• Cost-Sensitive Learning (NIPS 2008)

Online Lectures

• Active Learning Tutorial (ICML 2009) — Dasgupta & Langford
• Theory, Methods and Applications of Active Learning (MLSS 2009) — Nowak & Castro
• Miscellaneous Other Talks from videolectures.net