Give this class the contents of a robots.txt file or the URL to it and it will parse it for you. As an option you can give it the name or your robot/spider/crawler and it will return only the section that applies to your case. Works through proxies with the help of Snoopy (see script for more details).
This MySQL class has the following features:
- Debug messages (can be activate or not);
- Error handling (can be activate or not);
- Basic MySQ ...
class that caches dynamic content. easy to use an integrate into current
scripts.
Give this class the contents of a robots.txt file or the URL to it and it will parse it for you. As an option you can give it the name or your rob ...
This class builds completely an HTML Form based in a description of the
fields, it's ideal to avoid writing one form any new table you have to
f ...
