Historic Backlinks Index Update

By Alex September 5, 2011

Today we’ve updated our Historic backlinks index, stats are as follows: 352,953,418,877 unique pages crawled 3,529,437,390,545 unique URLs in total. All fresh data crawled in period up to 16 August 2011 was added to this update.

Posted In: Updates

4 Responses to “Historic Backlinks Index Update”

  1. Joe Lewis said:

    September 12, 2011 at 3:30 pm

    I appreciate that there may be no way to normalise your historical data, but when i look at multiple sites on the non cumulative graphs and see a 1000% spike on the same month because your spider has deepened its crawl, I could easily make the mistake of thinking that this was an increase in link acquisition velocity when it is in fact just a data anomaly. Before I realised this I told clients that there was a suspicious spike in activity that needed investigating. now I’m not sure how to trust the whole historic index in terms of trends.

    • Dixon said:

      September 12, 2011 at 4:01 pm

      Actually – you will be glad to know there is a “normalize” button in the drop downs that you can use to offset this effect. However – this won’t work on cumulative representations for this data.

      While we are here – there are a few other limitations of the data that one should be aware of. The history charts track URL DISCOVERY, but does not track URL DELETIONS – so if a site’s links decay at (say) 10% per month, then there will be increasingly large over-estimations if this chart is used to portray the existence of links at this point in time. The Fresh data is probably a better estimate of “LIVE” links, but with the Fresh index, we do not know whether a link found on any given day is new within the last 30 days or whether it was already in the historic index and just re-crawled – but (in my opinion) a link that has not been seen by our crawlers for over 30 days is probably buried pretty deep and is therefore not very powerful.

  2. Cork Web Design said:

    September 12, 2011 at 4:00 pm

    I saw the peak and was wondering what the reason for this was. how often do you actually do this?

    • Dixon said:

      September 12, 2011 at 4:07 pm

      You can see the dates we update our Historic Index on as we try to make a note of them here. By contrast, the Fresh Index is updating every day in general and is worth using most of the time.