robots.txt class

Give this class the contents of a robots.txt file or the URL to it and it will parse it for you. As an option you can give it the name or your robot/spider/crawler and it will return only the section that applies to your case. Works through proxies with the help of Snoopy (see script for more details).

Reviews

There are no comments available yet.

Here you can write a comment


Please enter at least 10 characters.
Loading... Please wait.
* Pflichtangabe
ANG GES G P
Very Simple Template allows you to dissociate very simply HTML code from PHP code. A programer will easily rewrite an HTML project done by a grap ...
ANG GES G P
Class that connects to a FTP server to work with files and directores. It features: - connect to ftp server - change current working dir - ...
ANG GES G P
simple implementation of the nntp protocol which allows php to make postings to a newsgroup. You can also reply to your own postngs on the same ...
ANG GES G P
Almost every PHP and SQL site a search. The problem is that these searches almost never support booleans. If they do it's usually pretty bad. The ...
ANG GES G P
A bunch of classes for a HTTP connection to a webserver
ANG GES G P
MiniXML provides a simple, API to generating and parsing XML. Its advantages are ease-of-use and the fact that no additional libraries are require ...
ANG GES G P
A class for MySQL managment, it returns an array of these things you selected in the database. It returns if update, delete, insert command was su ...
ANG GES G P
Compile many classes into one class. This class get around the limited class extension that PHP offers. IE class A extends classb, classc etc is ...
1075 x viewed
0 x memorized
Details to the advertisement
ANG GES G P
Make a request

You are no longer logged in as a user. Please enter your name and your E-mail-adress!

Loading... Please wait.
Details to the seller

toon.goedhart@