Package Usage: pypi: LangSegment
This is a multilingual tokenization tool that currently supports for zh/ja/en/ko, and more languages.
20 versions
Latest release: environ 2 ans ago
11 414 downloads last month
View more package details: https://packages.ecosystem.code.gouv.fr/registries/pypi.org/packages/LangSegment