robots.txt class

Give this class the contents of a robots.txt file or the URL to it and it will parse it for you. As an option you can give it the name or your robot/spider/crawler and it will return only the section that applies to your case. Works through proxies with the help of Snoopy (see script for more details).

Reviews

There are no comments available yet.

Here you can write a comment


Please enter at least 10 characters.
Loading... Please wait.
* Pflichtangabe
ANG GES G P
phpdl-server.health.class.php bietet Euch und Eurem Server die Möglichkeit, Serverdienste (apache,mysql u.s.w.) restarten zu lassen. Ihr konfig ...
ANG GES G P
A collection of classes and resources to process XML using PHP
ANG GES G P
This is a class to simplify the listing of classes and allows for commandline-like wildcards (i.e. '*.txt'). The class currently returns arrays ...
ANG GES G P
Allow send POST headers to any sites, icq, boards, and other...
ANG GES G P
A generalized parsed template caching mechanism, that can work with a variety of existing template classes.
ANG GES G P
emailform is written in and for php4. it is a very easy way to send data gathered with a form to an email recepient. some of the features are... ...
ANG GES G P
Class library for easy creation of hybrid WAP pages. The page is defined in a object oriented generic way. Depending on the user's browser eithe ...
ANG GES G P
Simple class that is useful to hold mysql database link information.
845 x viewed
0 x memorized
Details to the advertisement
ANG GES G P
Make a request

You are no longer logged in as a user. Please enter your name and your E-mail-adress!

Loading... Please wait.
Details to the seller

toon.goedhart@