Get Started

What is ASReview LAB?

ASReview LAB is a free (Libre) open-source machine learning tool for screening and systematically labeling a large collection of textual data. It’s sometimes referred to as a tool for title and abstract screening in systematic reviews or meta-analyses, but it can handle any type of textual data that must be screened systematically, see the paper published in Nature Machine Intelligence.

ASReview LAB implements three different options:

  • Oracle: Screen textual data in interaction with the active learning model. The reviewer is the ‘oracle’, making the labeling decisions.

  • Simulation: Evaluate the performance of active learning models on fully labeled data.

  • Validation: Validate labels provided by another screener or derived from an LLM or AI, and explore benchmark datasets without being an oracle.

ASReview LAB is one of the products of the ASReview research project initiated at Utrecht University, which has grown into a vivid community of researchers, users, and developers from around the world.

What is active learning?

Artificial Intelligence (AI) and machine learning has allowed the development of AI-aided pipelines that assist in finding relevant texts for search tasks. A well-established approach to increasing the efficiency of screening large amounts of textual data is screening prioritization through Active Learning: a constant interaction between a human who labels records and a machine learning model which selects the most likely relevant record based on a minimum training dataset. The active learning cycle is repeated until the annotator is sufficiently confident they have seen all relevant records. Thus, the machine learning model is responsible for ranking the records and the human provides the labels, this is called Researcher-In-The-Loop (RITL).

It allows the screening of large amounts of text in an intelligent and time-efficient manner. ASReview LAB, published in Nature Machine Intelligence, has shown the benefits of active learning, reducing up to 95% of the required screening time.

Labeling workflow with ASReview

Start and finish a systematic labeling process with ASReview LAB by following these steps:

  1. Create a dataset with potentially relevant records you want to screen systematically. Improve the quality of the data and specify clear reviewing (inclusion/exclusion) criteria

  2. Specify a stopping criterion

  3. Start ASReview LAB

  4. Create a project

  5. Import your dataset

  6. Select Prior Knowledge

  7. Select the four components of the Active learning model (feature extractor, classifier, balancing method, query strategy)

  8. Wait until the warm up of the AI is ready (the software is extracting the features and trains the classifier on the prior knowledge)

  9. Start Screening until you reach your stopping criterion

  10. At any time, you can export the dataset the labeling decisions or the entire project.

Quick start

  1. Check if Python 3.8 or later is installed (if not, install Python)

python --version
  1. Install ASReview LAB

pip install asreview
  1. Open ASReview LAB

asreview lab
  1. Click Create to create a project

  2. Select a mode (Oracle, Validation, Simulation)

  3. Name the project, and if you want, add an author name(s) and type a description

  4. Import a dataset you want to review, or select a benchmark dataset (only available for the Validation and Simulation mode)

  5. Add prior knowledge. Select at least 1 relevant and 1 irrelevant record to warm up the AI. You can search for a specific record or request random records

  6. Select the four components of the active learning model, or rely on the default settings that have shown fast and excellent performance in many simulation studies

  7. ASReview LAB starts extracting the features and runs the classifier with the prior knowledge

You’re ready to start labeling your data! All your labeling actions are automatically saved, so there is no need to click the save button (we don’t even have one).

ASReview LAB terminology

When you do text screening for a systematic review in ASReview LAB, it can be useful to know some basic concepts about systematic reviewing and machine learning to understand. The following overview describes some terms you might encounter as you use ASReview LAB.

Active learning model

An active learning model is the combination of four elements: a feature extraction technique, a classifier, a balance, and a query strategy.


ASReview stands for Active learning for Systematic Reviews or AI-assisted Systematic Reviews, depending on context. Avoid this explanation, only use as tagline.

ASReview CLI

ASReview CLI is the command line interface that is developed for advanced options or for running simulation studies.


Data includes dataset, prior knowledge, labels, and notes.


A dataset is the collection of records that the user imports and exports.


ELAS stands for “Electronic Learning Assistant”. It is the name of ASReview mascot. It is used for storytelling and to increase explainability.


Export is the action of exporting a dataset or a project from ASReview LAB.


An extension is the additional element to the ASReview LAB, such as the ASReview Datatools extension.


Import is the action of importing a dataset or a project into ASReview LAB.

Model configuration

Model configuration is the action of the user to configure the active learning model.


A note is the information added by the user in the note field and stored in the project file. It can be edited on the History page.


A project is a project created in ASReview LAB.

Projects dashboard

The project dashboard is the landing page containing an overview of all projects in ASReview LAB.

Project file

The project file is the .asreview file containing the data and model configuration. The file is exported from ASReview LAB and can be imported back.

Project mode

The project mode includes oracle, simulation, and validation in ASReview LAB:

Oracle mode is used when a user reviews a dataset systematically with interactive artificial intelligence (AI).

Validation mode is used when a user validates existing labels or engages in a review process without being an oracle

Simulation mode is used when a user simulates a review on a completely labeled dataset to see the performance of ASReview LAB.


The project status is the stage that a project is at in ASReview LAB.

Setup refers to the fact that the user adds project information, imports the dataset, selects the prior knowledge, configures the model and initiates the first iteration of model training.

In Review refers to the fact that in oracle or validation mode, the user adds labels to records, or in simulation mode, the simulation is running.

Finished refers to the fact that in oracle or validation mode, the user decides to complete the reviewing process or has labeled all the records, or in simulation mode, the simulation has been completed.

Published refers to the fact that the user publishes the dataset and project file in a repository, preferably with a Digital Object Identifier (DOI).


A record is the data point that needs to be labeled. A record can contain both information that is used for training the active learning model, and information that is not used for this purpose.

In the case of systematic reviewing, a record is meta-data for a scientific publication. Here, the information that is used for training purposes is the text in the title and abstract of the publication. The information that is not used for training typically consists of other metadata, for example, the authors, journal, or DOI of the publication.


Reviewing is the decision-making process on the relevance of records (“irrelevant” or “relevant”). It is interchangeable with Labeling, Screening, and Classifying.


The human annotator is the person who labels records.


Replacement term when the context is PRISMA-based reviewing.

Key principles

The use of ASReview LAB comes with five fundamental principles:

  1. Humans are the oracle;

  2. Code is open & results are transparent;

  3. Decisions are unbiased;

  4. The interface shows an AI is at work;

  5. Users are responsible for importing high quality data.


The ASReview LAB software doesn’t collect any information about the usage or its user. Great, isn’t it!