ERNIE-Doc (247M)
BaiduLanguage modelingLanguage modeling/generationOpen weights
The ERNIE-Doc (247M) model is an open-weights language modeling model from Baidu with 247000000.00000003 parameters.
About ERNIE-Doc (247M)
Transformers are not suited for processing long documents, due to their quadratically increasing memory and time consumption. Simply truncating a long document or applying the sparse attention mechanism will incur the context fragmentation problem or
Details
- Provider
- Baidu
- Task
- Language modeling,Language modeling/generation
- Parameters
- 247000000.00000003
- Released
- 2020-12-31
- Open weights
- Yes