AutoMapIt Sitemap Creation Service

Cross Domain sitemaps are located on one domain, but mapping a different domain. For example, you may decide to place your sitemaps from xyz.com over at abc.com. In the past, this was not allowed and would generate an error.

This news means BIG things for AutoMapIt customers since FTP is no longer a necessity… you don\’t need to host the sitemap on your own site, you just need to add a link to your robots.txt file that points to your sitemap on AutoMapIt\’s servers! No need for FTP and no need to remember to make your own uploads… Once you signup and activate your domain, your site will be picked up by the search engines!

It has never been easier to make sitemaps for your website than it is at AutoMapIt right now!

AutoMapIt now uses If-Modified-Since headers with the crawlers. What this means for you is that pages that haven\’t been updated since the last time this system crawled your website won\’t need to be loaded again. This saves bandwidth and means faster crawls of your website.

Google has used these headers when requesting your pages for years and I\’ve written about how they can help your website before, but the crawlers at AutoMapIt haven\’t adopted this practice until now.

Before you can take advantage of this bandwidth saving feature, you will need to make sure that your site supports this HTTP header request. The Header Checker here at AutoMapIt has been re-designed to show you whether your server supports this feature or not. The bottom line of the header report states whether your website will ~likely~ support this because depending on the URL that you check, it may or may not hold true for other pages of yours.

As long as the Header Checker reports that your site supports the If-Modified-Since header, there is nothing else for you to do. When AutoMapIt crawls your pages, we will automatically ask if the page has been modified since the last time we checked it. If there have been no changes, the data from the last crawl will be used and we will not download the page from your site. The extreme scenario is that this will help you save up to 7.5 GB of bandwidth per update, but this will depend on how many URLs you have and how large your files are.

What gets in the way?

Apache is built to handle these headers automatically, no tweaking needed… unless your URLs are parsed through the PHP interpreter or other server-side \’page creators\’. I know for a fact that PHP breaks this ability on Apache… not sure of other servers or pre-processing languages (ASP, PERL, ColdFusion, RUby, Python, etc.). I have a fix for this in PHP by downloading my ServHead zip file (unzip it and include() the file into your pages at the top of the page).

This file is intended to be included into your pages and works with static files passed through the PHP interpreter. If your pages are entirely generated from a database (php-nuke, osCommerce, Joomla, phpBB, and many many more), then this script will require a tweak to get the lastmod datestamp from your database. Of course, if you can\’t get this to work yourself, I can be contracted to make it work for a small fee The code in that zip is free for you to use and modify, but it is my code… I\’m keeping the rights to it so that I can continue to offer it for free… personal or commercial use.

I just read a great blog post about Defensible websites which talks about certain metrics for a websites success. I thought that a lot more people could benefit from this understanding so I figured I\’d share it with you!

The basic idea is that you try to remove single points of income or traffic from your site. I don\’t mean remove Google as a source of traffic, but if they account for more than 75% your traffic and for whatever reason, they change their algo and shake you loose… you wouldn\’t be getting any more users to your site. If the only source of income for your site is Google Adsense (or ONLY any other single source) and that source dries up by going out of business, canceling your membership, or any other problems… you wouldn\’t be making anything else from your site.

With multiple sources of traffic, with multiple revenue streams… your site will stand the test of time and weather what comes it\’s way. The article at tropicalseo.com will give you specific metrics to measure your site against that will help make your site more defensible, virtually eliminating the risk that any single change in a factor outside of your control will permanently sink your website.

Long time subscribers of AutoMapIt have seen a few of the tools that I provide go into the mainstream search engines in one way or another. I like to think of it as a testament to my staying a step or two ahead of the search engines and designing tools to help keep you, the webmasters and SEO community, there on the front lines. Yahoo has just released the first-ever Beta launch of \’Dynamic URL Rewriting\’ in Site Explorer. This tool is very similar to the \”Ignore URL Key\” filter that has been available to AutoMapIt users for a good, long time and allows you help filter a portion of a URL on your site. It is an awesome tool that helps you to be sure that session IDs do not make it into the URLs indexed for your site.

Most session IDs use a 32 character string of numbers 0-9 and letters A-F (hex-based). This means 16 possibilities for each character to the 32 (character) power or 3 with 38 zeroes after it… yup, a lot of potential page with different URLs all saying the exact same thing as each other. By stripping out the session IDs, these pages are consolidated into one.

Another issue that a tool like this could help to fix would be the ability to exclude URLs with certain keys or key/value pairs altogether. Yes, AutoMapIt has these as well. These types of filters are handy for skipping past pages such as action=buynow or redirect_to=htpp:///xxxxxx which typically all look the same and don\’t really contain any unique content to the other pages like these…

Imagine a 1,000 page catalog… every item description page has a link to a \’print version\’, \’buy now\’, \’tell-a-friend\’… it usually doesn\’t matter which product is shown… the tell-a-friend has the same form on it with a unique identifier. The only difference between one page and the other is the product ID code. Aside from possible a 6-8 digit ID number, those are 1,000 tell-a-friend pages that are exactly the same as each other. Plus 1,000 buy now pages that are exactly like each other… and 1,000 pages worth of print versions exactly like their parent pages.

That\’s a TON of bandwidth that can be saved by using filters and tools like Dynamic URL Rewriting from Yahoo! Well, Yahoo has taken the first step toward this bright future and hopefully webmasters can continue to help Yahoo help the webmasters.

If you haven\’t done so already, get over to Yahoo! and setup your filters… then come back to AutoMapIt and use the same filters AND MORE to help out the other search engines.

tags: Dynamic URL Rewriting, sitemap, seo

With all of the stats and numbers and reports (oh my!), it\’s sometimes hard to notice the lighter side of things. I just read a blog entry on another site that I thought would help with a ring of truth among webmasters.

It\’s about a new trend in making money for websites. I can back this up from an SEO perspective… if you build a unique and genuinely useful website, you are almost doomed to succeed. It seems the new trend will favor sites that step away from the pack and concern themselves with visitors first and income… also first.

Actually, this has been true for a good, long time. The pioneers and groundbreakers have traditionally done well, but so have many who have taken a \’repeat\’ idea and pushed it\’s boundaries into new territory. One thing about most internet success stories… they focus on the visitors and what keeps them coming back. From features to service, putting your customers first is good business sense.