robots.txt class

Give this class the contents of a robots.txt file or the URL to it and it will parse it for you. As an option you can give it the name or your robot/spider/crawler and it will return only the section that applies to your case. Works through proxies with the help of Snoopy (see script for more details).

Reviews

There are no comments available yet.

Here you can write a comment


Please enter at least 10 characters.
Loading... Please wait.
* Pflichtangabe
ANG GES G P
PHP Klassen zum erzeugen von Atom Feeds. Setzt PHP 5.1RC voraus. Eine deutsche Anleitung ist unter http://www.flaimo.com/php/atom zu finden.
ANG GES G P
Paranoia was designed to check the validity of the parameters that a php page will receive after a form submision. It can be used to check the v ...
ANG GES G P
Embed those pesky little gif files right into your scripts.
ANG GES G P
phpSorm is mainly a base-class which provides a simple way to map a table from a relational database to a PHP-object. The extended functionality i ...
ANG GES G P
Another MySQL interface class for php. FUNCTIONS: GetDatabases, CreateDB, DropDB, CopyDB, SelectDB, GetTableList, GetFieldList, Delete, Update, ...
ANG GES G P
Eine einfacheTemplate class in PHP 5 , mit Unterstützung für Platzhalter und Sections.
ANG GES G P
This class allows one to easily query the whois database and returns the information for a specific domain name. It also automatically retrieves t ...
ANG GES G P
html table builder is written in and for php4 and allows you to dynamically create html tables within your php code. some of the features are... ...
917 x viewed
0 x memorized
Details to the advertisement
ANG GES G P
Make a request

You are no longer logged in as a user. Please enter your name and your E-mail-adress!

Loading... Please wait.
Details to the seller

toon.goedhart@