Command Line Usage

This contains the command line usage for each of the standard scripts we release. These are each included in parlai/scripts, and all may be invoked with the “parlai” supercommand.

The parlai supercommand may be invoked from the command line by running parlai after installing ParlAI. Its default output looks like this:

usage: parlai [-h] [--helpall] COMMAND ...

       _
      /")
     //)
  ==//'=== ParlAI
   /

optional arguments:
  -h, --help               show this help message and exit
  --helpall                show all commands, including advanced ones.

Commands:

  display_data (dd)        Display data from a task
  display_model (dm)       Display model predictions.
  eval_model (em, eval)    Evaluate a model
  train_model (tm, train)  Train a model
  interactive (i)          Interactive chat with a model on the command line
  safe_interactive         Like interactive, but adds a safety filter
  self_chat                Generate self-chats of a model

The remainder of this page describes each of the commands, their possible arguments,
and some examples of their usage.

display_data

Short description: Display data from a task

Aliases: dd Basic example which iterates through the tasks specified and prints them out. Used for verification of data loading and iteration.

For example, to make sure that bAbI task 1 (1k exs) loads one can run and to see a few of them:

Examples

parlai display_data -t babi:task1k:1

CLI Arguments

Argument

Description

-o, --init-opt

Path to json file of options. Note: Further Command-line arguments override file-based options.

--allow-missing-init-opts

Warn instead of raising if an argument passed in with –init-opt is not in the target opt.

-t, --task

ParlAI task(s), e.g. “babi:Task1” or “babi,cbt”

-dt, --datatype

Choose from: train, train:ordered, valid, test. to stream data add “:stream” to any option (e.g., train:stream). by default train is random with replacement, valid is ordered, test is ordered.
Choices: train, train:stream, train:ordered, train:ordered:stream, train:stream:ordered, train:evalmode, train:evalmode:stream, train:evalmode:ordered, train:evalmode:ordered:stream, train:evalmode:stream:ordered, valid, valid:stream, test, test:stream
Default: train:ordered.

-bs, --batchsize

Batch size for minibatch training schemes
Default: 1.

-dynb, --dynamic-batching

Use dynamic batching
Choices: full, batchsort, None

-v, --verbose

Print all messages

-dp, --datapath

Path to datasets, defaults to {parlai_dir}/data

-m, --model

The model class name. can match parlai/agents/ for agents in that directory, or can provide a fully specified module for from X import Y via -m X:Y (e.g. -m parlai.agents.seq2seq.seq2seq:Seq2SeqAgent)

-mf, --model-file

Model file name for loading and saving models

-im, --init-model

Initialize model weights and dict from this file

-n, -ne, --num-examples

Default: 10.

-mdl, --max-display-len

Default: 1000.

--display-add-fields

Display these fields when verbose is off (e.g., “–display-add-fields label_candidates,beam_texts”)

--ignore-agent-reply

Default: True.


display_model

Short description: Display model predictions.

Aliases: dm Basic example which iterates through the tasks specified and runs the given model on them.

Examples

parlai display_model -t babi:task1k:1 -m "repeat_label"
parlai display_model -t "#MovieDD-Reddit" -m "ir_baseline" -mp "-lp 0.5" -dt test

CLI Arguments

Argument

Description

-o, --init-opt

Path to json file of options. Note: Further Command-line arguments override file-based options.

--allow-missing-init-opts

Warn instead of raising if an argument passed in with –init-opt is not in the target opt.

-t, --task

ParlAI task(s), e.g. “babi:Task1” or “babi,cbt”

-dt, --datatype

Choose from: train, train:ordered, valid, test. to stream data add “:stream” to any option (e.g., train:stream). by default train is random with replacement, valid is ordered, test is ordered.
Choices: train, train:stream, train:ordered, train:ordered:stream, train:stream:ordered, train:evalmode, train:evalmode:stream, train:evalmode:ordered, train:evalmode:ordered:stream, train:evalmode:stream:ordered, valid, valid:stream, test, test:stream
Default: valid.

-bs, --batchsize

Batch size for minibatch training schemes
Default: 1.

-dynb, --dynamic-batching

Use dynamic batching
Choices: full, batchsort, None

-v, --verbose

Print all messages

-dp, --datapath

Path to datasets, defaults to {parlai_dir}/data

-m, --model

The model class name. can match parlai/agents/ for agents in that directory, or can provide a fully specified module for from X import Y via -m X:Y (e.g. -m parlai.agents.seq2seq.seq2seq:Seq2SeqAgent)

-mf, --model-file

Model file name for loading and saving models

-im, --init-model

Initialize model weights and dict from this file

-n, -ne, --num-examples

Default: 10.

--display-add-fields

Display these fields when verbose is off (e.g., “–display-add-fields label_candidates,beam_texts”)


eval_model

Short description: Evaluate a model

Aliases: em, eval Basic example which iterates through the tasks specified and evaluates the given model on them.

Examples

parlai eval_model -t "babi:Task1k:2" -m "repeat_label"
parlai eval_model -t "#CornellMovie" -m "ir_baseline" -mp "-lp 0.5"

CLI Arguments

Argument

Description

-o, --init-opt

Path to json file of options. Note: Further Command-line arguments override file-based options.

--allow-missing-init-opts

Warn instead of raising if an argument passed in with –init-opt is not in the target opt.

-t, --task

ParlAI task(s), e.g. “babi:Task1” or “babi,cbt”

-dt, --datatype

Choose from: train, train:ordered, valid, test. to stream data add “:stream” to any option (e.g., train:stream). by default train is random with replacement, valid is ordered, test is ordered.
Choices: train, train:stream, train:ordered, train:ordered:stream, train:stream:ordered, train:evalmode, train:evalmode:stream, train:evalmode:ordered, train:evalmode:ordered:stream, train:evalmode:stream:ordered, valid, valid:stream, test, test:stream
Default: valid.

-bs, --batchsize

Batch size for minibatch training schemes
Default: 1.

-dynb, --dynamic-batching

Use dynamic batching
Choices: full, batchsort, None

-v, --verbose

Print all messages

-dp, --datapath

Path to datasets, defaults to {parlai_dir}/data

-m, --model

The model class name. can match parlai/agents/ for agents in that directory, or can provide a fully specified module for from X import Y via -m X:Y (e.g. -m parlai.agents.seq2seq.seq2seq:Seq2SeqAgent)

-mf, --model-file

Model file name for loading and saving models

-im, --init-model

Initialize model weights and dict from this file

-rf, --report-filename

Saves a json file of the evaluation report either as an extension to the model-file (if begins with a “.”) or a whole file path. Set to the empty string to not save at all.

--save-world-logs

Saves a jsonl file containing all of the task examples and model replies. Must also specify –report-filename.

--save-format

Choices: conversations, parlai
Default: conversations.

-ne, --num-examples

Default: -1.

-d, --display-examples

-ltim, --log-every-n-secs

Default: 10.

-mcs, --metrics

List of metrics to show/compute, e.g. all, default,or give a list split by , like ppl,f1,accuracy,hits@1,rouge,bleuthe rouge metrics will be computed as rouge-1, rouge-2 and rouge-l
Default: default.

-micro, --aggregate-micro

Report micro-averaged metrics instead of macro averaged metrics.

--log-keep-fields

Fields to keep when logging. Should be a comma separated list
Default: all.

-tblog, --tensorboard-log

Tensorboard logging of metrics, default is False

-tblogdir, --tensorboard-logdir

Tensorboard logging directory, defaults to model_file.tensorboard


interactive

Short description: Interactive chat with a model on the command line

Aliases: i Basic script which allows local human keyboard input to talk to a trained model.

Examples

parlai interactive -m drqa -mf "models:drqa/squad/model"

When prompted, enter something like: Bob is Blue.\nWhat is Bob?

Input is often model or task specific, but in drqa, it is always context '\n' question.

CLI Arguments

Argument

Description

-o, --init-opt

Path to json file of options. Note: Further Command-line arguments override file-based options.

--allow-missing-init-opts

Warn instead of raising if an argument passed in with –init-opt is not in the target opt.

-t, --task

ParlAI task(s), e.g. “babi:Task1” or “babi,cbt”
Default: interactive.

-dt, --datatype

Choose from: train, train:ordered, valid, test. to stream data add “:stream” to any option (e.g., train:stream). by default train is random with replacement, valid is ordered, test is ordered.
Choices: train, train:stream, train:ordered, train:ordered:stream, train:stream:ordered, train:evalmode, train:evalmode:stream, train:evalmode:ordered, train:evalmode:ordered:stream, train:evalmode:stream:ordered, valid, valid:stream, test, test:stream
Default: train.

-bs, --batchsize

Batch size for minibatch training schemes
Default: 1.

-dynb, --dynamic-batching

Use dynamic batching
Choices: full, batchsort, None

-v, --verbose

Print all messages

-dp, --datapath

Path to datasets, defaults to {parlai_dir}/data

-m, --model

The model class name. can match parlai/agents/ for agents in that directory, or can provide a fully specified module for from X import Y via -m X:Y (e.g. -m parlai.agents.seq2seq.seq2seq:Seq2SeqAgent)

-mf, --model-file

Model file name for loading and saving models

-im, --init-model

Initialize model weights and dict from this file

-d, --display-examples

--display-prettify

Set to use a prettytable when displaying examples with text candidates

--display-add-fields

Display these fields when verbose is off (e.g., “–display-add-fields label_candidates,beam_texts”)

-it, --interactive-task

Create interactive version of task
Default: True.

--outfile

Saves a jsonl file containing all of the task examples and model replies. Set to the empty string to not save at all

--save-format

Format to save logs in. conversations is a jsonl format, parlai is a text format.
Choices: conversations, parlai
Default: conversations.

-fixedCands, --local-human-candidates-file

File of label_candidates to send to other agent

--single-turn

If on, assumes single turn episodes.

--log-keep-fields

Fields to keep when logging. Should be a comma separated list
Default: all.


safe_interactive

Short description: Like interactive, but adds a safety filter

Script for safety protected interaction between a local human keyboard input and a trained model.

CLI Arguments

Argument

Description

-o, --init-opt

Path to json file of options. Note: Further Command-line arguments override file-based options.

--allow-missing-init-opts

Warn instead of raising if an argument passed in with –init-opt is not in the target opt.

-t, --task

ParlAI task(s), e.g. “babi:Task1” or “babi,cbt”
Default: interactive.

-dt, --datatype

Choose from: train, train:ordered, valid, test. to stream data add “:stream” to any option (e.g., train:stream). by default train is random with replacement, valid is ordered, test is ordered.
Choices: train, train:stream, train:ordered, train:ordered:stream, train:stream:ordered, train:evalmode, train:evalmode:stream, train:evalmode:ordered, train:evalmode:ordered:stream, train:evalmode:stream:ordered, valid, valid:stream, test, test:stream
Default: train.

-bs, --batchsize

Batch size for minibatch training schemes
Default: 1.

-dynb, --dynamic-batching

Use dynamic batching
Choices: full, batchsort, None

-v, --verbose

Print all messages

-dp, --datapath

Path to datasets, defaults to {parlai_dir}/data

-m, --model

The model class name. can match parlai/agents/ for agents in that directory, or can provide a fully specified module for from X import Y via -m X:Y (e.g. -m parlai.agents.seq2seq.seq2seq:Seq2SeqAgent)

-mf, --model-file

Model file name for loading and saving models

-im, --init-model

Initialize model weights and dict from this file

-d, --display-examples

--display-prettify

Set to use a prettytable when displaying examples with text candidates

--display-add-fields

Display these fields when verbose is off (e.g., “–display-add-fields label_candidates,beam_texts”)

-it, --interactive-task

Create interactive version of task
Default: True.

--safety

Apply safety filtering to messages
Choices: all, none, classifier, string_matcher
Default: all.

-fixedCands, --local-human-candidates-file

File of label_candidates to send to other agent

--single-turn

If on, assumes single turn episodes.


self_chat

Short description: Generate self-chats of a model

Allows a model to self-chat on a given task.

CLI Arguments

Argument

Description

-o, --init-opt

Path to json file of options. Note: Further Command-line arguments override file-based options.

--allow-missing-init-opts

Warn instead of raising if an argument passed in with –init-opt is not in the target opt.

-t, --task

ParlAI task(s), e.g. “babi:Task1” or “babi,cbt”
Default: self_chat.

-dt, --datatype

Choose from: train, train:ordered, valid, test. to stream data add “:stream” to any option (e.g., train:stream). by default train is random with replacement, valid is ordered, test is ordered.
Choices: train, train:stream, train:ordered, train:ordered:stream, train:stream:ordered, train:evalmode, train:evalmode:stream, train:evalmode:ordered, train:evalmode:ordered:stream, train:evalmode:stream:ordered, valid, valid:stream, test, test:stream
Default: train.

-bs, --batchsize

Batch size for minibatch training schemes
Default: 1.

-dynb, --dynamic-batching

Use dynamic batching
Choices: full, batchsort, None

-v, --verbose

Print all messages

-dp, --datapath

Path to datasets, defaults to {parlai_dir}/data

-m, --model

The model class name. can match parlai/agents/ for agents in that directory, or can provide a fully specified module for from X import Y via -m X:Y (e.g. -m parlai.agents.seq2seq.seq2seq:Seq2SeqAgent)

-mf, --model-file

Model file name for loading and saving models

-im, --init-model

Initialize model weights and dict from this file

--seed

Default: 42.

-d, --display-examples

Default: True.

--display-add-fields

Display these fields when verbose is off (e.g., “–display-add-fields label_candidates,beam_texts”)

-st, --selfchat-task

Create a self chat version of the task
Default: True.

--num-self-chats

Number of self chats to run
Default: 1.

--selfchat-max-turns

The number of dialogue turns before self chat ends
Default: 6.

--seed-messages-from-task

Automatically seed conversation with messages from task dataset.

--outfile

File to save self chat logs

--save-format

Format to save logs in. conversations is a jsonl format, parlai is a text format.
Choices: conversations, parlai
Default: conversations.

-pmf, --partner-model-file

Define a different partner for self chat

--partner-opt-file

Path to file containing opts to override for partner

--log-keep-fields

Fields to keep when logging. Should be a comma separated list
Default: all.


train_model

Short description: Train a model

Aliases: tm, train Training script for ParlAI.

The standard way to train a model. After training, also computes validation and test error.

The user must provide a model (with --model) and a task (with --task).

Examples

parlai train_model -m ir_baseline -t dialog_babi:Task:1 -mf /tmp/model
parlai train_model -m seq2seq -t babi:Task10k:1 -mf '/tmp/model' -bs 32 -lr 0.5 -hs 128
parlai train_model -m drqa -t babi:Task10k:1 -mf /tmp/model -bs 10

CLI Arguments

Argument

Description

-o, --init-opt

Path to json file of options. Note: Further Command-line arguments override file-based options.

--allow-missing-init-opts

Warn instead of raising if an argument passed in with –init-opt is not in the target opt.

-t, --task

ParlAI task(s), e.g. “babi:Task1” or “babi,cbt”

-dt, --datatype

Choose from: train, train:ordered, valid, test. to stream data add “:stream” to any option (e.g., train:stream). by default train is random with replacement, valid is ordered, test is ordered.
Choices: train, train:stream, train:ordered, train:ordered:stream, train:stream:ordered, train:evalmode, train:evalmode:stream, train:evalmode:ordered, train:evalmode:ordered:stream, train:evalmode:stream:ordered, valid, valid:stream, test, test:stream
Default: train.

-bs, --batchsize

Batch size for minibatch training schemes
Default: 1.

-dynb, --dynamic-batching

Use dynamic batching
Choices: full, batchsort, None

-v, --verbose

Print all messages

-dp, --datapath

Path to datasets, defaults to {parlai_dir}/data

-m, --model

The model class name. can match parlai/agents/ for agents in that directory, or can provide a fully specified module for from X import Y via -m X:Y (e.g. -m parlai.agents.seq2seq.seq2seq:Seq2SeqAgent)

-mf, --model-file

Model file name for loading and saving models

-im, --init-model

Initialize model weights and dict from this file

-et, --evaltask

Task to use for valid/test (defaults to the one used for training)

-eps, --num-epochs

Default: -1.

-ttim, --max-train-time

Default: -1.

-vtim, --validation-every-n-secs

Validate every n seconds. Saves model to model_file (if set) whenever best val metric is found
Default: -1.

-stim, --save-every-n-secs

Saves the model to model_file.checkpoint after every n seconds (default -1, never).
Default: -1.

-sval, --save-after-valid

Saves the model to model_file.checkpoint after every validation (default False).

-veps, --validation-every-n-epochs

Validate every n epochs. Saves model to model_file (if set) whenever best val metric is found
Default: -1.

-vp, --validation-patience

Number of iterations of validation where result does not improve before we stop training
Default: 10.

-vmt, --validation-metric

Key into report table for selecting best validation
Default: accuracy.

-vmm, --validation-metric-mode

How to optimize validation metric (max or min)
Choices: max, min

-mcs, --metrics

List of metrics to show/compute, e.g. all, default,or give a list split by , like ppl,f1,accuracy,hits@1,rouge,bleuthe rouge metrics will be computed as rouge-1, rouge-2 and rouge-l
Default: default.

-micro, --aggregate-micro

Report micro-averaged metrics instead of macro averaged metrics.

-tblog, --tensorboard-log

Tensorboard logging of metrics, default is False

-tblogdir, --tensorboard-logdir

Tensorboard logging directory, defaults to model_file.tensorboard

--bpe-vocab

Path to pre-trained tokenizer vocab

--bpe-merge

Path to pre-trained tokenizer merge

--bpe-dropout

Use BPE dropout during training.