gpt-tokenizer
Version:
A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models
| Filename | Content Type | Size | |
|---|---|---|---|
| 9.73 MB | |||
| 7.05 MB | |||
| 15.4 MB | |||
| 9.58 MB | |||
| 4.35 MB | |||
| 1.08 kB | |||
| 17.3 kB | |||
| 5.4 kB |
A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models
| Filename | Content Type | Size | |
|---|---|---|---|
| 9.73 MB | |||
| 7.05 MB | |||
| 15.4 MB | |||
| 9.58 MB | |||
| 4.35 MB | |||
| 1.08 kB | |||
| 17.3 kB | |||
| 5.4 kB |