Package Usage: pypi: langsegment
This is a multilingual tokenization tool that currently supports for zh/ja/en/ko, and more languages.
20 versions
Latest release: about 1 year ago
11,414 downloads last month
View more package details: https://packages.ecosystem.code.gouv.fr/registries/pypi.org/packages/langsegment