UNPKG

hx-tokenizer

Version:

Split text into tokens.

github.com/hexacta/tokenizer

hexacta/tokenizer

9 files

	Filename	Content Type	Size
	.npmignore	text/plain	27 B
	.travis.yml	text/yaml	133 B
	join.test.js	text/javascript	601 B
	license	text/plain	1.08 kB
	package.json	application/json	507 B
	readme.md	text/markdown	849 B
	tokenize.test.js	text/javascript	1.53 kB
	tokenizer.js	text/javascript	4.72 kB
	yarn.lock	text/plain	93.7 kB

About

github.com/hexacta/tokenizer

hexacta/tokenizer

Releases

9 years ago