HTML Tutorials
A Guide to Robots.txt
Posted by Isabelle Newton on January 23th 2009 via (www.moreniche.com)
This guide looks a setting up your own robots.txt file so that you can control spiders from search engines as well as more malicious sources.
Views:
102
Comments:
0
Bookmark:
0
You need login to post a comment. Please log in or register here. It's free.
Comments
There's no comments yet.
Tutorial's Category
- + HTML RSS Feeds
-
- » Basics
- » Custom Error Pages
- » DHTML
- » Forms
- » Frames
- » Images
- » Tables
- » Tips and Trick
- » Website Optimization
- » XHTML
- + CSS RSS Feeds
- + Adobe Photoshop RSS Feeds
- + Adobe Illustrator RSS Feeds
- + Adobe Flash RSS Feeds
- + Adobe Fireworks RSS Feeds
- + PHP RSS Feeds
-
- » AJAX
- » Basic
- » Calendars
- » Chat Scripts
- » Cookies and Sessions
- » Date and Time
- » E-Commerce
- » File Upload
- » Forms
- » Forums and Guestbooks
- » General Development
- » Images
- » MySQL
- » RSS Feeds
- » Security
- » XML
- + Javascript RSS Feeds
- + Adobe Dreamweaver RSS Feeds
-
- » Basic
- » CSS
- » Database
- » Forms
- » Images
- » Layers
- » Layout
- » Navigation
- » Tables
- » Tips and Trick
- + MS Expression Web RSS Feeds
- + ASP.net RSS Feeds
- + Perl RSS Feeds
- + Ruby on Rails RSS Feeds
-
- » Basic
- » Database
- » Forms
- » Forum & Guestbook
- » RSS Feeds
- » Security
- » Tips and Trick