robots.txt class

Give this class the contents of a robots.txt file or the URL to it and it will parse it for you. As an option you can give it the name or your robot/spider/crawler and it will return only the section that applies to your case. Works through proxies with the help of Snoopy (see script for more details).

Rezensionen

Es sind noch keine Kommentare vorhanden.

Hier Kannst Du einen Kommentar verfassen


Bitte gib mindestens 10 Zeichen ein.
Wird geladen... Bitte warte.
* Pflichtangabe
ANG GES G P
A PHP library to do full text indexing with MySQL
ANG GES G P
Give this class the contents of a robots.txt file or the URL to it and it will parse it for you. As an option you can give it the name or your rob ...
ANG GES G P
This is a very easy to use class for making tables with various styles. It also has an example included.
ANG GES G P
Class to support MySQL sessions with many servers
ANG GES G P
This project is effectively a library class for reading/writing/displaying XML documents, for manipulating them in memory by adding/deleting/chang ...
ANG GES G P
HAWHAW is a PHP class library you can use to create hybrid web pages. With HAWHAW you can publish dynamic WML pages and control the output with a ...
ANG GES G P
Embed those pesky little gif files right into your scripts.
ANG GES G P
webring provides an easy-to-configure solution for creating a webring. All URLs are stored in one file (webring.urls), and each link in the webri ...
1237 x angesehen
0 x gemerkt
Details zur Anzeige
ANG GES G P
Anfrage stellen

Du bist nicht als Benutzer angemeldet. Bitte gebe Deinen Namen und E-Mailadresse an!

Wird geladen... Bitte warte.
Details zur Anbieter

toon.goedhart@