robots.txt class

Give this class the contents of a robots.txt file or the URL to it and it will parse it for you. As an option you can give it the name or your robot/spider/crawler and it will return only the section that applies to your case. Works through proxies with the help of Snoopy (see script for more details).

Reviews

There are no comments available yet.

Here you can write a comment


Please enter at least 10 characters.
Loading... Please wait.
* Pflichtangabe
ANG GES G P
This project is effectively a library class for reading/writing/displaying XML documents, for manipulating them in memory by adding/deleting/chang ...
ANG GES G P
Es kann die Differenz des 1. zum 2. Wert (egal ob nach oben oder nach unten) in prozentualer Entwicklung oder absolut ausgegeben werden.Auf der We ...
ANG GES G P
Mit dieser Klasse,können Sie ganz leicht ein komplettes Verzeichnis mit Bildern verkleinern oder vergrößern lassen.Die neuen Größen müssen a ...
ANG GES G P
  This debugger is build with serveral interacting technologies. The core element is a PHP based function, which can be integrated into the appl ...
ANG GES G P
class that caches dynamic content. easy to use an integrate into current scripts.
ANG GES G P
Class library for easy creation of hybrid WAP pages. The page is defined in a object oriented generic way. Depending on the user's browser eithe ...
ANG GES G P
Encryption library for php
ANG GES G P
Sessionara is a handy php class to manage sessions and realize member areas. In a developer's life, the job to make admin areas is a common task ...
764 x viewed
0 x memorized
Details to the advertisement
ANG GES G P
Make a request

You are no longer logged in as a user. Please enter your name and your E-mail-adress!

Loading... Please wait.
Details to the seller

toon.goedhart@