GLM-130B
4 August 2022
GLM-130B from Tsinghua University, language model published 2022-08-04. Notable for: sota improvement.
Abstract
GLM-130B (ICLR 2023) is an open bilingual (English & Chinese) bidirectional dense model with 130 billion parameters, pre-trained using the General Language Model (GLM) algorithm1. It is designed to support inference tasks with the 130B parameters on a single A100 (40G * 8) or V100 (32G * 8) server. As of July 3rd, 2022, GLM-130B has been trained on over 400 billion text tokens (200B each for Chinese and English)
Sources: Epoch AI. This entry was generated from a dataset and has not been edited by hand.