CommonGlot
The research suite

GlotSuite

GlotSuite is the family of research releases at the heart of CommonGlot. Each project solves one link in the chain from raw web text to working language technology, and each one feeds the next.

Visit the GlotSuite homepage ↗ All projects
GlotSuite logo

How the pieces fit

Identify

GlotLID labels the language, GlotScript the writing system.

GlotLIDGlotScript

Collect

GlotCC and GlotWeb gather and filter web text with those labels.

GlotCCGlotWebGlotStoryBookGlotSparse

Model

Glot500 learns from the collected corpora.

Glot500

Evaluate

GlotOCR Bench and per-language results show where gaps remain.

GlotOCR Bench

Timeline

Used in the wild

Corpus filtering

GlotLID is used as a language filter in large open multilingual web corpora, including FineWeb-2.

Research baselines

Glot500-m is a standard baseline for massively multilingual evaluation of tail languages.

Script-aware pipelines

GlotScript is installed from PyPI to detect and split scripts in multilingual preprocessing.