AutoMapIt Site Map Creation

[pvcordova]

Fairly new here, so bear with me.

Question: If I don't have pages that I want ignored, should I just delete the robot.txt file from my webserver?

[MaxPowers]

Short answer, yes... go ahead but...

The long answer is that this has changed recently allowing you to notify the search engines of the location of your sitemap using the robots.txt file. Add a line similar to...

Sitemap: http://www.example.com/sitemap.xml

...so that the URL makes sense for your site. When the search engines visit your site and check the robots.txt file, they will know where to get that sitemap from. This only works with the XML version of your sitemap (the Google Sitemap).

Other than this, without any pages to ignore, the robots.txt doesn't do much for you. This is neither good nor bad. You don't have to ignore things and you don't get bonuses whether you do or don't ignore stuff.

[pvcordova]

Great, thanks!

[brant]

Is it necessary to have a robots.txt file pointing to the hosted site map if I have uploaded the sitemap.xml?

[MaxPowers]

It is not necessary to use robots.txt to point to the hosted sitemap.

This is suggested as a last resort if you are unable to use FTP. A few hosting providers do not provide FTP to their customers, so this is their only option.