Archive for the ‘Updates’ Category
Index Update – nearly 2.9 trillion unique URLs
We have updated our main backlinks index that now contains nearly 2.9 trillion unique URLs. 
Index Updates now with increased frequency
This afternoon the team released another update. We hit the 2.7 Trillion mark.
There were a few new things happening in this crawl which I will be able to reveal more next week if you are attending Pubcon. If not, ask your favourite blogger to attend the “MajesticSEO to the Max” session on Wednesday or get a Tweeter. I will also be trying to link my PowerPoint slides to the @MajesticSEO Twitter profile… Which may or may not work.
What should be clear by now, though, is that we are trying to update TWICE a month and it looks like we have reached that milestone. This is especially useful for agencies who want to be able to report to their clients on progress more regularly. I encourage SEOs to force updates to their reports every few weeks to keep them up to date. We do not charge extra for forcing a new analysis, so it means you can keep your clients informed more regularly.
If you are using the API (platinum users and above) you can “poll” the index to see if it has updated before drawing down new data.
Index update, second this month
This afternoon we have been able to launch our update this month. The new index is a little short of 2.7 trillion URLs.
We are delighted that we have been able to continue updating with this increased regularity. Bringing down the turnaround time will be a major advantage for SEOs and agencies who want to be able to confidently report back to their clients more than once a month.
Index Update 4th October – 2.6 Trillion
In all the effort that we are putting in to the new site design (available to view in beta), we reported the new index update on the main site but I am only just getting around to posting it here – for any of you using RSS feeds to keep a track of updates. I hope you won’t mind the slight delay there!
The new update was our third in 31 days. That is starting to get seriously quick and I don’t want to make promises, but it looks like this will not be our last update in October.
New Index turn around in just 16 days
We are today announcing another new update. This is our second this month and the team worked hard to turn it around in record time – just 16 days.
Why is this cool?
A year ago, it was taking us at least six weeks between creating a “stop” point for the crawl, and then running the program to build the index. Yes – it was taking six weeks at least to process the data into a new index, and that was with just a trillion known URLs. Now with well over twice the known internet universe, we have cut that down considerably, largely due to our TeraFlop server.
Anything else new?
The lag between us starting our index update and getting the data online is also down considerably on this update. If you are lucky, you will find links in this update which we discovered as recently as the 11th of September – just nine days ago.
Can I see data fresher than nine days ago?
If you want to see links fresher than nine days ago, you need to be ALREADY TRACKING root domains in our advanced reports to see fresh crawl data. You cannot start tracking this data until a few days after the ordering the advanced reports.
So how big is the index now?
We now have 2.5 trillion urls in the index.