robots.txt class

Give this class the contents of a robots.txt file or the URL to it and it will parse it for you. As an option you can give it the name or your robot/spider/crawler and it will return only the section that applies to your case. Works through proxies with the help of Snoopy (see script for more details).

Reviews

There are no comments available yet.

Here you can write a comment


Please enter at least 10 characters.
Loading... Please wait.
* Pflichtangabe
ANG GES G P
This is a class for sending email; it has support for To:, Cc:, Bcc: and Reply-To: headers. It requires that you have sendmail installed.
ANG GES G P
Diese kleine PHP Klasse hilft bei der Parameterverarbeitung via $_GET und $_POST. Werte können komfortabel eingelesen, in der Session gespeichert ...
ANG GES G P
Es kann die Differenz des 1. zum 2. Wert (egal ob nach oben oder nach unten) in prozentualer Entwicklung oder absolut ausgegeben werden.Auf der We ...
ANG GES G P
Class that connects to a FTP server to work with files and directores. It features: - connect to ftp server - change current working dir - ...
ANG GES G P
Authentification and user track. A simple session manager for all.
ANG GES G P
A class for maintaining CSV files
ANG GES G P
i18n ist eine sammlung von klassen um mehrsprachige php seiten besser managen zu können. zahlen, währungen, datumsangaben usw. können ebenfalls ...
ANG GES G P
Simple and easy class for reading/writing 3dim arrays as CSV files.
755 x viewed
0 x memorized
Details to the advertisement
ANG GES G P
Make a request

You are no longer logged in as a user. Please enter your name and your E-mail-adress!

Loading... Please wait.
Details to the seller

toon.goedhart@