robots.txt class

Give this class the contents of a robots.txt file or the URL to it and it will parse it for you. As an option you can give it the name or your robot/spider/crawler and it will return only the section that applies to your case. Works through proxies with the help of Snoopy (see script for more details).

Reviews

There are no comments available yet.

Here you can write a comment


Please enter at least 10 characters.
Loading... Please wait.
* Pflichtangabe
ANG GES G P
Compile many classes into one class. This class get around the limited class extension that PHP offers. IE class A extends classb, classc etc is ...
ANG GES G P
A PHP library to do full text indexing with MySQL
ANG GES G P
Here another html template implementation. 2 main features are: * Allows nested templates, * Easy to use.. (I hope so.. :) )
ANG GES G P
This project includes some classes which are used to generate very simple MS Excel file (xls) via PHP. The generated xls file can be obtained by ...
ANG GES G P
Class for composing and generating XML documents. It features: - Definition of tags and data sections. - Support input and output iso-8859-1 and ...
ANG GES G P
Class to do various things with strings, such as strip all slashes from the string or an (multi-dimension) array of them, as well as add the sla ...
ANG GES G P
phpdl-count003.class ist eine Counterklasse Features: - Freeware - deutsche Quellcode - Dokumentation - keine Datenbank wird benötigt - auto ...
ANG GES G P
HAWHAW is a PHP class library you can use to create hybrid web pages. With HAWHAW you can publish dynamic WML pages and control the output with a ...
1029 x viewed
0 x memorized
Details to the advertisement
ANG GES G P
Make a request

You are no longer logged in as a user. Please enter your name and your E-mail-adress!

Loading... Please wait.
Details to the seller

toon.goedhart@