RETRO-7B
DeepMindLanguage modeling/generationLanguage modeling
RETRO-7B is a language modeling/generation model from DeepMind released in 2022 with 7500000000.0 parameters.
About RETRO-7B
We enhance auto-regressive language models by conditioning on document chunks retrieved from a large corpus, based on local similarity with preceding tokens. With a 2 trillion token database, our Retrieval-Enhanced Transformer (RETRO) obtains compara
Details
- Provider
- DeepMind
- Task
- Language modeling/generation,Language modeling
- Parameters
- 7500000000.0
- Released
- 2022-02-07
- Open weights
- No