robots.txt class

Give this class the contents of a robots.txt file or the URL to it and it will parse it for you. As an option you can give it the name or your robot/spider/crawler and it will return only the section that applies to your case. Works through proxies with the help of Snoopy (see script for more details).

Reviews

There are no comments available yet.

Here you can write a comment


Please enter at least 10 characters.
Loading... Please wait.
* Pflichtangabe
ANG GES G P
Die Nested Set Klasse bearbeitet Nested Set Bäume die in einer MySQL-DB gespeichert werden. Diese Funktionen kann die Klasse ausführen: Knoten ...
ANG GES G P
class that makes it very easy to read and write XML files. It uses an in-memory tree structure to accomplish this. It allows you to create ni ...
ANG GES G P
Das Projekt Backlink Analyse Tool wurde ins Leben gerufen, um ambitionierten Webmastern, die Möglichkeit zu geben, die Struktur von Backlinks bes ...
ANG GES G P
A base class and 2 derivated classes to generate clients for dictionary servers that follow the DICT protocol (RFC 2229), such as the server at ...
ANG GES G P
Encryption library for php
ANG GES G P
i18n ist eine sammlung von klassen um mehrsprachige php seiten besser managen zu können. zahlen, währungen, datumsangaben usw. können ebenfalls ...
ANG GES G P
Embed those pesky little gif files right into your scripts.
ANG GES G P
DX0 is a PHP library that generates Dynamic HTML specific to the visitor's browser. DX0 focuses on generating tight, compact DHTML. DX0 allows cla ...
894 x viewed
0 x memorized
Details to the advertisement
ANG GES G P
Make a request

You are no longer logged in as a user. Please enter your name and your E-mail-adress!

Loading... Please wait.
Details to the seller

toon.goedhart@