robots.txt class

Give this class the contents of a robots.txt file or the URL to it and it will parse it for you. As an option you can give it the name or your robot/spider/crawler and it will return only the section that applies to your case. Works through proxies with the help of Snoopy (see script for more details).

Reviews

There are no comments available yet.

Here you can write a comment


Please enter at least 10 characters.
Loading... Please wait.
* Pflichtangabe
ANG GES G P
Returns client information based on HTTP_USER_AGENT
ANG GES G P
Es kann die Differenz des 1. zum 2. Wert (egal ob nach oben oder nach unten) in prozentualer Entwicklung oder absolut ausgegeben werden.Auf der We ...
ANG GES G P
The HtmlSource class is a simple way of retrieving, and stripping, HTML source from any website, given a URL. Supports GET and POST and also coo ...
ANG GES G P
Classic two frame menu. Simple to use yet easily customised
ANG GES G P
Full Featured php Form Generation Class, by Strudleman. Simple to use, full CSS support, XHTML compliant output.
ANG GES G P
First version of class for formatting of text (TTF) on images. Will allow you to more easily place ttfboxes on images.
ANG GES G P
Generates HTML using templates. The data to be inserted is in array form so that a list of information can be displayed. The format for the data ...
ANG GES G P
Diese kleine PHP Klasse hilft bei der Parameterverarbeitung via $_GET und $_POST. Werte können komfortabel eingelesen, in der Session gespeichert ...
972 x viewed
0 x memorized
Details to the advertisement
ANG GES G P
Make a request

You are no longer logged in as a user. Please enter your name and your E-mail-adress!

Loading... Please wait.
Details to the seller

toon.goedhart@