Historic Index Update

By Alex July 5, 2013

We have now updated our Historic Index with data for start of June 2013. Here are the new stats:

Historic Index –

Unique Pages crawled: 501,925,972,687

Unique URLs: 2,008,056,406,541

Date range: 07 Nov 2007 to 09 Jun 2013

There is an increase in unique crawled pages but reduction of unique URLs due to roll out of quality improvement algorithm used initially in Fresh index.
- See more at: http://blog.majesticseo.com/index-updates/historic-index-update-17/

Posted In: Updates

One Response to “Historic Index Update”

  1. F-3000 said:

    July 24, 2013 at 3:18 pm

    A bit stupid spot to drop this, but anyhow…

    Thanks to Majestic-12 crawler’s stubborness (and overall behavior), I found a bug from my selfmade CMS.

    For a time I was wondering that why in the heck MJ’s crawler attempts to go for pages that simply don’t exist, and shouldn’t be linked to in anywhere. Until I figured out that in every pages’ head-section, a link-element with rel=”relative” was pointing to those non-existent pages, and this was a bug.

    Thanks.