Language models

View docs

If you are interested in evaluating the performance of specific language models, you can browse the available models below. In a future SyntaxGym release, we will allow users to add their own models as Docker containers.

Available language models
Name Description Author Date added
Docker image
Status Average performance
Name Description Author Date added Docker image Status Average performance
Transformer XL None Zihang Dai et al. 2020-01-21 cpllab/language-models:transformer-xl Validated 76.81%
GPT-2 None Radford et al. (OpenAI) 2020-01-21 cpllab/language-models:gpt-2 Validated 88.30%
GPT-2 XL None Radford et al. (OpenAI) 2020-01-21 cpllab/language-models:gpt-2-xl Validated 90.85%
JRNN None Josefowicz et al. 2020-01-21 cpllab/language-models:jrnn Validated 76.83%
RNNG None Dyer et al. 2020-01-30 cpllab/language-models:rnng Validated 74.38%
Ordered Neurons None Shen et al. 2020-01-30 cpllab/language-models:ordered-neurons Validated 74.32%
Vanilla LSTM None Hochreiter & Schmidhuber 2020-01-30 cpllab/language-models:vanilla-lstm Validated 65.59%