We have now updated our Historic Index with data for start of June 2013. Here are the new stats:
Historic Index –
Unique Pages crawled: 501,925,972,687
Unique URLs: 2,008,056,406,541
Date range: 07 Nov 2007 to 09 Jun 2013
There is an increase in unique crawled pages but reduction of unique URLs due to roll out of quality improvement algorithm used initially in Fresh index.
– See more at: http://blog.majesticseo.com/index-updates/historic-index-update-17/
Founder and Managing Director of Majestic-12 Ltd (trading as Majestic-SEO)
Latest posts by Alex (see all)
- Historic Index Update - July 26, 2015
- July Historic Index Update Published - July 31, 2014
- April Historic Index Update - April 16, 2014
A bit stupid spot to drop this, but anyhow…
Thanks to Majestic-12 crawler’s stubborness (and overall behavior), I found a bug from my selfmade CMS.
For a time I was wondering that why in the heck MJ’s crawler attempts to go for pages that simply don’t exist, and shouldn’t be linked to in anywhere. Until I figured out that in every pages’ head-section, a link-element with rel=”relative” was pointing to those non-existent pages, and this was a bug.
Thanks.
July 24, 2013 at 3:18 pm