robots.txt class

Give this class the contents of a robots.txt file or the URL to it and it will parse it for you. As an option you can give it the name or your robot/spider/crawler and it will return only the section that applies to your case. Works through proxies with the help of Snoopy (see script for more details).

Reviews

There are no comments available yet.

Here you can write a comment


Please enter at least 10 characters.
Loading... Please wait.
* Pflichtangabe
ANG GES G P
max-> LoadChecker($) is detecting the exact amount of microseconds when it is called from the main script. Calculating then the difference betwee ...
ANG GES G P
A collection of classes and resources to process XML using PHP
ANG GES G P
Diese Template-Klasse wurde für kleine Webprojekte unter dem Gesichtspunkt entwickelt, dass die Benutzung so einfach wie nur irgendwie möglich s ...
ANG GES G P
MiniXML provides a simple, API to generating and parsing XML. Its advantages are ease-of-use and the fact that no additional libraries are require ...
ANG GES G P
CMailFile is a PHP class that allows you to attach files to an email you are sending from a PHP script.
ANG GES G P
Here another html template implementation. 2 main features are: * Allows nested templates, * Easy to use.. (I hope so.. :) )
ANG GES G P
The HtmlSource class is a simple way of retrieving, and stripping, HTML source from any website, given a URL. Supports GET and POST and also coo ...
ANG GES G P
esay - isert/add elements to any position of your array(s) - delete elements from your array(s) - move elements within your array(s) - replace e ...
915 x viewed
0 x memorized
Details to the advertisement
ANG GES G P
Make a request

You are no longer logged in as a user. Please enter your name and your E-mail-adress!

Loading... Please wait.
Details to the seller

toon.goedhart@