Submit URLs to Majestic SEO’s Crawler for Free
It has been one of our most requested features and now our URL Submitter is live. Search engines used to ask you to “suggest a site”. Today we are pleased to announce a new URL submitter tool for Majestic, allowing you to request our crawler visit any URL and help us improve our crawl and you to improve your reporting. One of the great things about this feature is that even FREE accounts can add up to 100 urls a day (Sign up for a free account here).
See your Links in Hours, not Days
Why build a URL submitter? In short, it means we can crawl the pages YOU care about, and get them indexed faster. In hours not days.
- Seeing different links in different data sources? Curate them ALL in one place.
- We will work out Flow Metrics for these URLs for you
- If you get a link, get it in our metrics faster to show your client
When deciding which URLs to crawl next, there are lots to choose from on a worldwide scale. Our Historic index has 4 trillion plus, so we have our own decision making process, but there’s nothing like letting our users have an influence too!
If you are link building and see a link that we have not picked up, this tool will mostly help it get indexed faster by our crawlers, so you can report on it faster to your clients.
If you aren’t link building, but simply tracking what is happening in the world, showing us URLs that you think are important will help the Flow Metrics report better.
How Does it Work?
If you do not yet have a free account, sign up for one. You get 100 url submissions a day free. If you want more, you get plenty on a paid account. After 100 URLs, we charge a small resource of 50 analysis units per URL. To give you an idea of how much this is, a silver account (US$49.99 / €30.99 / £29.99) a month has 5 million of these a month, so if ALL you did with a silver account was submit URLs, you could submit 100,000 a month PLUS 100 per day.
Here’s how you find the URL Submitter tool:
You can get to the URL submitter from the Tools dropdown. Look – there near the bottom of the list.
You have two easy ways to add URL’s. If you have 100 or less, you can just cut and paste them straight into the box, as shown above. Put each URL on a separate line. Cut and paste from an Excel list worked perfectly for me when testing.
If you have more than 100 URLs, then we have also given you the ability to easily upload a file. Just select the upload file tab:
You have all sorts of file options. If you are starting with a long Excel file, I suggest you first save that file in “.CSV” format as this removes potentially complicated control characters. You then hit “upload file” and after a sanity check, we let you upload the URLs to our submitter.
Does URL Submission GUARANTEE a Crawl?
That would not be realistic or acceptable. Whilst in the majority of cases we will crawl the URLs you submit within a few hours and most likely have them in the index by next day, we still obey Robots.txt and also we listen to the crawl-delay protocol. There are also other ways in which websites might block our crawlers and we honour those requests from webmasters and hosting companies. IN ADDITION, we will not overly stress a web domain. It will not matter how many millions of URLs you add for a website, we of course will not be hammering a web server even if you asked us to. We’ll assume you were just naïve, rather than a black hat… but just in case, every URL submission needs to come via a registered account.
If the URLs you submit are continually poor in quality, by the way, then we will pick up this poorness of quality just as quickly as picking up signals of good quality, so spamming the system may be expensive fun for someone, but is not likely to increase a URL’s quality score. The more likely result will be more confidence of a low quality score.
Can you submit via the API?
When we have this documented, you will be able to see it at http://developer-support.majesticseo.com
We really hope that you like this new feature.