Daily Crawl Data Improvement
Today we can announce some improvements to the “fresh Crawl” reports that users can see in their control panel.
Click to Enlarge
In addition to our now monthly full index updates, we track, daily, any new back-links that we discover to any given domain. The problem was that many of the flags on these back-links were not being parsed until the full update and the “mentions” of a domain (where a domain might be listed, but with no link) was appearing in these results. Whilst this is not necessarily wrong, it is hardly helpful for SEO.
So from the 26th of November onwards, mentions are removed from the daily updates.
Click to Enlarge
Whilst this will reduce the numbers on this report, at least your clients won’t go clicking on all the links and saying “but there’s no link there!”.
Please note that this report lists when we FOUND new links, not when they were actually created.
You can download these reports into CSV files pretty easy, with the link at the bottom left of the page.
We hope this is helpful? It was my suggestion, so if it wasn’t, I’ll have to hang my head in shame and go back to the development team with cap in hand… 
MajesticSEO hits 10,000 users
Do you think we have a Google founder as our 10,000th registered user? It was only the 22nd of July that we registered our 5,000th user, so it was encouraging when Alex told me that we have just hit out 10,000th user. Congratulations to a person we believe is called “Sergiy”.
We doubt it, but we would like to say a big thank you to this person, in the form of 100 credits – just as soon as they respond to our email. Only problem is that MajesticSEO is a truly international project, with not guarantees that the registrant speaks English. He or she might find themselves with a 100 extra credits and have no idea why!
Thank you – really – to every one of the 10,000 registered users.
Majestic SEO’s Pubcon Treasure Hunt – win 25 credits!
Win 25 credits free at our Pubcon Treasure hunt. Majestic SEO will be sculking around Pubcon in the form of its Marketing Director, Dixon Jones AKA Receptional. All you need to do is to track him down and give him a business card with the word “TREASURE” written on it. How can you find Dixon / Receptional in several thousand people? Well he’ll be trying to keep his badge on for the whole week. He’s also on the agenda on three separate sessions and will be at the Pubcon party on the Friday. Your treasure hunt might also bear fruit at the Treasure Island hotel bar, but please don’t knock on his room! The small print: The competition is open to both new and existing Majestic SEO users, but only one prize per person and per company. You must, however, be registered on MajesticSEO.com before the end of 14 November 2009, using the email address on your business card (or write the email address you wish to enter on the card). Majestic SEO’s decision is final. Your business card wil only be used in relation to this incentive and will then be destroyed. First 10 people to find Dixon get the prize!
Majestic SEO vs Google Webmaster Tools – responding to Dave Naylor!
Dave Naylor made a tweet last week observing the difference shown by his new geo-location tool using backlinks pointing to his domain from Majestic and Google. We actually missed this tweet but were kindly pointed to by our friends at Wordtracker. It is very nice to be referred to as “majestic“, but at the same time it is a bit hard to catch such tweets when there is no SEO word around it.
Before we looked into specifics we wanted to check if the top 50 domains that Google show are also present in our list of 7,541 known referring root domains with backlinks pointing to Dave’s site. It turns out that 43 out of 45 (that’s over 95%!) of the root domains in Google’s top list are also in the list of backlinks that we have for his site. This means that even though they may not have made it into the top 50 domains ranked by number of backlinks, we actually had backlinks found from them in our data set!
Did it surprise you to learn that we’ve matched 95%+ of Google’s top referring domains? You should not be, as we’ve been running this sort of index quality analysis comparisons since 2007, we only stopped doing it once we found that we were matching over 90% of referring domains from both Yahoo and Google. Since then our index has grown by 500%.
So why is Top 50 referring domains in our data is different from that of Google? Majestic database shows backlinks “as is”, or more precisely “as they were found” in our trillion URLs index. Some of the stranger domains in our Top 50 list included www.zippy.co.uk (50k backlinks) and adcentercommunity.com (42k), in the first instance majority of them were redirecting to Dave’s site and in the second case it was blog roll type of link present many pages. It is easy to exclude these domains from analysis on our site using the domain Analysis Options.
While we don’t know for sure, we believe that Google uses a subset of their complete web graph in Google’s Webmaster Tools. Can one be certain they include all the backlinks they come across with? Probably not, but in our case you can be certain that we show you all we’ve got and that seems to match Google’s data nicely!
Majestic Upgrade – Minor on the surface. Huge underneath.
We have take a next step towards making the site a little more intuitive, today. We are also paving the way for some considerably more complex changes coming up.
The most important change for me is to exclude “Nofollows”, “Deleted”, “AltText” and “Mention only” from the default reports after you retrieve the back-link data. We have been carefully noting what people have been saying about Majestic and one criticism has been “the data is out of date”. We have been working hard to make sure that this criticism won’t stick around for long. Investing in servers and infrastructure is part of this, but also – many people take all the links (our bad – now fixed) instead of filtering out the stuff they don’t need for SEO analysis. You can always get the data back, by going into the options panel. Here you will see new domains with the following exclusions by default:
This will reduce the number of links you see, but will concentrate your mind on ones more likely to count.
The summary overview screen will let you know how many backlinks we have excluded for you.
Most Customers to the site may notice it now by the changed “Control Panel Tools” menu.
We previously described the actions a little differently, as well as having the drop down before the domain box. We hope that these three options are more self explanatory than previously. If you are seeing a fourth box above, then you are part of some more advanced work, which should come down the line in a few weeks.
I hope these changes are helpful to people. We’d love your feedback, so I have updated the Poll on the top right of this blog page. Your vote wins nothing, but we do value your opinions!
Dixon.