Package Usage: rubygems: webrobots

This library helps write robots.txt compliant web robots in Ruby.
16 versions
Latest release: over 9 years ago
9 dependent packages
29,737,025 downloads total

View more package details: https://packages.ecosystem.code.gouv.fr/registries/rubygems.org/packages/webrobots

Dependent Repos 1

LINCnil/tosback2 Fork of tosdr/tosback2

Reimplementing TOSBack with Ruby and using git to see TOS changes!

Size: 160 MB - Last synced: 7 days ago - Pushed: about 7 years ago