SyntaxGym is a unified platform where language and NLP researchers can design psycholinguistic tests and visualize the performance of language models. Our goal is to make psycholinguistic assessment of language models more standardized, reproducible, and accessible to a wide variety of researchers. The project is run out of the MIT Computational Psycholinguistics Laboratory.

Test suites

Interested in viewing or designing psycholinguistic test suites? Create a new test suite online or upload one as a .json file.

Language models

Have a model you want to evaluate? Add a model as a Docker container, and it will automatically be evaluated on existing test suites.

Visualizations

Want to compare the results of different models across test suites? Visualize model performance through interactive charts.

Not sure where to start? Read more or take a look at the documentation.