Package Usage: pypi: robotexclusionrulesparser
A robots.txt parser alternative to Python's robotparser module
10 versions
Latest release: almost 9 years ago
32,017 downloads last month
View more package details: https://packages.ecosystem.code.gouv.fr/registries/pypi.org/packages/robotexclusionrulesparser