robots.txt class

Give this class the contents of a robots.txt file or the URL to it and it will parse it for you. As an option you can give it the name or your robot/spider/crawler and it will return only the section that applies to your case. Works through proxies with the help of Snoopy (see script for more details).

Reviews

There are no comments available yet.

Here you can write a comment


Please enter at least 10 characters.
Loading... Please wait.
* Pflichtangabe
ANG GES G P
HtmlReader is a PHP class for mananging html templates. You can insert variables, multi level if-blocks and multi level loops
ANG GES G P
PHP&Databases Managment (Select, Delete, Insert, Update)
ANG GES G P
This MySQL class has the following features: - Debug messages (can be activate or not); - Error handling (can be activate or not); - Basic MySQ ...
ANG GES G P
Here another html template implementation. 2 main features are: * Allows nested templates, * Easy to use.. (I hope so.. :) )
ANG GES G P
Generates HTML using templates. The data to be inserted is in array form so that a list of information can be displayed. The format for the data ...
ANG GES G P
SteamInfo: Steam-Server Informationen abfragen. Mit SteamInfo lassen sich simpel Informationen über einen Steam-Gameserver erfragen. Dab ...
ANG GES G P
Almost every PHP and SQL site a search. The problem is that these searches almost never support booleans. If they do it's usually pretty bad. The ...
ANG GES G P
displays content of variables in human-readable format. just call debug_var($VAR) and an window opens with the struktur and content
1176 x viewed
0 x memorized
Details to the advertisement
ANG GES G P
Make a request

You are no longer logged in as a user. Please enter your name and your E-mail-adress!

Loading... Please wait.
Details to the seller

toon.goedhart@