AceBIT

Support Forum
It is currently Mon Mar 15, 2010 1:51 pm

View unanswered posts | View active topics


Board index » English Forum » AceBIT SEO Professionals Suite (ASEOPS, successor product of Hello Engines! Professional)

All times are UTC + 1 hour [ DST ]



Post new topic Reply to topic  Page 1 of 1
  [ 2 posts ] 
  Print view Previous topic | Next topic 
Author Message
richfulton
 Post subject: Site Checker Stops When Robots.txt Contains Exclusions
PostPosted: Mon Dec 01, 2008 11:37 pm 
Offline
Rookie
Rookie

Joined: Mon Dec 01, 2008 11:23 pm
Posts: 1
On 6.58, the site checker (HTML Validator, Load Time Checker, Code Shrinker) appear to all stop when they first hit an excluded directory in the robots.txt file on my server. The remaining files on the server whether excluded or not are not checked.

What entry needs to be added to robots.txt to allow "Hello Engines! Professional" to spider the entire site?
Top
 Profile  
 
Support (AceBIT)
 Post subject: Re: Site Checker Stops When Robots.txt Contains Exclusions
PostPosted: Tue Dec 02, 2008 6:27 pm 
Hello,

the Site Checker does not use the file robots.txt but only the embedded meta-tags "ROBOTS". If it contains "NOFOLLOW", the Site Checker does not follow links from the current page. Unfortunately this is not customizable in the current version.

Best regards
Top
  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  Page 1 of 1
  [ 2 posts ] 

Board index » English Forum » AceBIT SEO Professionals Suite (ASEOPS, successor product of Hello Engines! Professional)

All times are UTC + 1 hour [ DST ]

Who is online

Users browsing this forum: No registered users and 0 guests

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group