Acknowledgments

SyntaxGym is maintained by the MIT Computational Psycholinguistics Laboratory.

We gratefully acknowledge support from the following sources:

• NIH (award number T32NS105587)

• NSF Graduate Research Fellowship

• Open Philanthropy AI Fellowship

• MIT-IBM Watson AI Lab

• Google Faculty Research Award

• Newton Brain Science Award

Citations

If you use the website or command-line tools in your research, we ask that you please cite the ACL 2020 system demonstration paper:

@inproceedings{gauthier-etal-2020-syntaxgym,
    title = "{S}yntax{G}ym: An Online Platform for Targeted Evaluation of Language Models",
    author = "Gauthier, Jon and Hu, Jennifer and Wilcox, Ethan and Qian, Peng and Levy, Roger",
    booktitle = "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations",
    month = jul,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/2020.acl-demos.10",
    pages = "70--76",
}

If you use the original test suites, models, or results presented on the website, please cite the ACL 2020 long paper:

@inproceedings{hu-etal-2020-systematic,
    title = "A Systematic Assessment of Syntactic Generalization in Neural Language Models",
    author = "Hu, Jennifer and Gauthier, Jon and Qian, Peng and Wilcox, Ethan and Levy, Roger",
    booktitle = "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics",
    month = jul,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/2020.acl-main.158",
    pages = "1725--1744",
}