robots.txt class

Give this class the contents of a robots.txt file or the URL to it and it will parse it for you. As an option you can give it the name or your robot/spider/crawler and it will return only the section that applies to your case. Works through proxies with the help of Snoopy (see script for more details).

Rezensionen

Es sind noch keine Kommentare vorhanden.

Hier Kannst Du einen Kommentar verfassen


Bitte gib mindestens 10 Zeichen ein.
Wird geladen... Bitte warte.
* Pflichtangabe
ANG GES G P
This is a class for sending email; it has support for To:, Cc:, Bcc: and Reply-To: headers. It requires that you have sendmail installed.
ANG GES G P
Bei cphplib handelt es sich um eine Sammlung von PHP Funktionen. Diese können einfach in bestehende PHP-Scripte eingebunden und sofort genutzt we ...
ANG GES G P
html table builder is written in and for php4 and allows you to dynamically create html tables within your php code. some of the features are... ...
ANG GES G P
phpSorm is mainly a base-class which provides a simple way to map a table from a relational database to a PHP-object. The extended functionality i ...
ANG GES G P
This class select data of PostgreSQL DataBase by SQL instructions and write results into TXT file (SDF). Alignment and lenghts of columns are con ...
ANG GES G P
webring provides an easy-to-configure solution for creating a webring. All URLs are stored in one file (webring.urls), and each link in the webri ...
ANG GES G P
Here another html template implementation. 2 main features are: * Allows nested templates, * Easy to use.. (I hope so.. :) )
ANG GES G P
jfile class is a wrapper class for fopen/fclose and flock. With this class, its easy to open new files which are locked and released after use
1019 x angesehen
0 x gemerkt
Details zur Anzeige
ANG GES G P
Anfrage stellen

Du bist nicht als Benutzer angemeldet. Bitte gebe Deinen Namen und E-Mailadresse an!

Wird geladen... Bitte warte.
Details zur Anbieter

toon.goedhart@