robots.txt class

Give this class the contents of a robots.txt file or the URL to it and it will parse it for you. As an option you can give it the name or your robot/spider/crawler and it will return only the section that applies to your case. Works through proxies with the help of Snoopy (see script for more details).

Reviews

There are no comments available yet.

Here you can write a comment


Please enter at least 10 characters.
Loading... Please wait.
* Pflichtangabe
ANG GES G P
This class provides an easy to use library to the PHP GD-based image functions. Features: Many smart and useful resize options. TTF text generati ...
ANG GES G P
entspricht Version 2.1 der vCard-Spezifikation (http://www.imc.org/pdi/) erlaubt folgende Felder: Name Formatted Name Telefon-/Faxnummern ...
ANG GES G P
A PHP library to do full text indexing with MySQL
ANG GES G P
Diese Zip-Klasse verpackt rekursiv Ordner und entpackt diese auch rekursiv. Die Zip-Klasse beinhaltet einen Dateifilter, dieser kann mit Blacklis ...
ANG GES G P
Provide an easy way to seed a script with debug statements, with screen or file output, and halt or continue operation.
ANG GES G P
PHP Klasse für die Administration einer mod_auth_mysql Datenbank, benötigt Apache mit mod_auth_mysql! Ein einfaches Demo Interface gibts auch..
ANG GES G P
This is a basic class to list the contents of a given directory. This class is written for UNIX file systems but could work in DOS with a few ch ...
ANG GES G P
displays content of variables in human-readable format. just call debug_var($VAR) and an window opens with the struktur and content
668 x viewed
0 x memorized
Details to the advertisement
ANG GES G P
Make a request

You are no longer logged in as a user. Please enter your name and your E-mail-adress!

Loading... Please wait.
Details to the seller

toon.goedhart@