Website
Fast Searchable Multilingual LLM Corpus
Explore a fast searchable corpus derived from LLM training data. Features multiple languages, including a massive 103B token Japanese dataset for researchers and developers.
- Website
- #Corpus
- #LLM
- #Multilingual
Shared by 天 Staff
Published ·Updated
Details
This resource provides a fast and searchable corpus compiled from training data for Large Language Models (LLMs).
Key features include:
* Multilingual Support: Contains data in several languages.
* Massive Japanese Corpus: Includes a substantial 103 billion tokens of Japanese data.
It's an excellent resource for researchers and developers working with LLMs, especially those interested in multilingual or Japanese language models.