Time2Dine.co.nz is Online Restaurant Booking
 
 New Zealand Restaurants Forum
Restaurant Forum | Contact Time2Dine | Book Online & Save | About Time2Dine

Go Back   New Zealand Restaurants & Food Forum > Search Engine Optimisation > SEO vBulletin
Register

Reply
 
LinkBack Thread Tools
  #1  
Old 12th Jul 07, 09:44 AM
Junior Reviewer
 
Join Date: Jul 2007
Posts: 3
Angel8552 is on a distinguished road
Default Xenu LinkSleuth Options

One of the people on my own staff used Xenu to spider Xenite.Org yesterday. To say I was not pleased is an understatement. I've now blocked Xenu in my robots.txt and I hope the software honors that file because I don't see how else to stop people from hammering my server.

Xenu grabbed 2400 files in less than a minute. I had to manually update my server log file so that the traffic didn't artificially inflate my daily statistics update.

Xenu uses the prefetching protocol that Google Web Accelerator uses to slam servers. It's just not a server-friendly methodology and I consider use of Xenu to be totally "black hat", totally unethical, and completely unacceptable.
__________________
Outdoor fountain lecterns
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On

Similar Threads
Thread Thread Starter Forum Replies Last Post
Options Takeaways T2DMan Warkworth Restaurants 0 2nd Dec 04 01:57 AM



Powered by: vBulletin Copyright ©2000 -2012, Jelsoft Enterprises Limited.
Search Engine Friendly URLs by vBSEO 3.2.0
© 2010 Time2Dine