We have now updated our Historic Index with data until end of October 2013. Here are the new stats:
Historic Index –
Unique Pages crawled: 601,375,778,666
Unique URLs: 2,329,017,065,332
Date range: 25 Mar 2008 to 26 Oct 2013
Founder and Managing Director of Majestic-12 Ltd (trading as Majestic-SEO)
Latest posts by Alex (see all)
- Historic Index Update - July 26, 2015
- July Historic Index Update Published - July 31, 2014
- April Historic Index Update - April 16, 2014
by “we have” you mean who? I am newbie here – you have your own crawling robots that are indexing webpages or what?
December 2, 2013 at 2:34 pmHi Dirk – welcome.
December 2, 2013 at 4:07 pmYes – we have our own crawlers. we are the largest Link Intelligence database on the planet. We crawl around 187 billion URLs in the last 90 days, and discovered nearly 700 billion URLs. “We” are a UK based company crawling through an international project. Our bot is called “MJ12 Bot” and it obeys robots.txt.