Submit URLs to Majestic SEO’s Crawler for Free

By Dixon Jones February 18, 2013

Add URL icon

It has been one of our most requested features and now our URL Submitter is live. Search engines used to ask you to “suggest a site”. Today we are pleased to announce a new URL submitter tool for Majestic, allowing you to request our crawler visit any URL and help us improve our crawl and you to improve your reporting. One of the great things about this feature is that even FREE accounts can add up to 100 urls a day (Sign up for a free account here).

See your Links in Hours, not Days

Why build a URL submitter? In short, it means we can crawl the pages YOU care about, and get them indexed faster. In hours not days.

  • Seeing different links in different data sources? Curate them ALL in one place.
  • We will work out Flow Metrics for these URLs for you
  • If you get a link, get it in our metrics faster to show your client

When deciding which URLs to crawl next, there are lots to choose from on a worldwide scale. Our Historic index has 4 trillion plus, so we have our own decision making process, but there’s nothing like letting our users have an influence too!

If you are link building and see a link that we have not picked up, this tool will mostly help it get indexed faster by our crawlers, so you can report on it faster to your clients.

If you aren’t link building, but simply tracking what is happening in the world, showing us URLs that you think are important will help the Flow Metrics report better.

How Does it Work?

If you do not yet have a free account, sign up for one. You get 100 url submissions a day free. If you want more, you get plenty on a paid account. After 100 URLs, we charge a small resource of 50 analysis units per URL. To give you an idea of how much this is, a silver account (US$49.99 / €30.99 / £29.99) a month has 5 million of these a month, so if ALL you did with a silver account was submit URLs, you could submit 100,000 a month PLUS 100 per day.

Some Screenshots

Here’s how you find the URL Submitter tool:

Find from the Tools menu

Find from the Tools menu

You can get to the URL submitter from the Tools dropdown. Look – there near the bottom of the list.

Screen Shot 2013-02-05 at 09.40.26

You have two easy ways to add URL’s. If you have 100 or less, you can just cut and paste them straight into the box, as shown above. Put each URL on a separate line. Cut and paste from an Excel list worked perfectly for me when testing.

If you have more than 100 URLs, then we have also given you the ability to easily upload a file. Just select the upload file tab:

Screen Shot 2013-02-05 at 09.41.00

You have all sorts of file options. If you are starting with a long Excel file, I suggest you first save that file in “.CSV” format as this removes potentially complicated control characters. You then hit “upload file” and after a sanity check, we let you upload the URLs to our submitter.

Does URL Submission GUARANTEE a Crawl?

That would not be realistic or acceptable. Whilst in the majority of cases we will crawl the URLs you submit within a few hours and most likely have them in the index by next day, we still obey Robots.txt and also we listen to the crawl-delay protocol. There are also other ways in which websites might block our crawlers and we honour those requests from webmasters and hosting companies. IN ADDITION, we will not overly stress a web domain. It will not matter how many millions of URLs you add for a website, we of course will not be hammering a web server even if you asked us to. We’ll assume you were just naïve, rather than a black hat… but just in case, every URL submission needs to come via a registered account.

If the URLs you submit are continually poor in quality, by the way, then we will pick up this poorness of quality just as quickly as picking up signals of good quality, so spamming the system may be expensive fun for someone, but is not likely to increase a URL’s quality score. The more likely result will be more confidence of a low quality score.

Can you submit via the API?

When we have this documented, you will be able to see it at http://developer-support.majesticseo.com

We really hope that you like this new feature.

 

 

Posted In: Development

36 Responses to “Submit URLs to Majestic SEO’s Crawler for Free”

  1. mohsin said:

    February 18, 2013 at 1:57 pm

    This is a very good tool and much needed. However, I suggest that “100 url submissions a day free” might not be the best idea to deliver this service. Because of multiple reasons:

    1- It will increase the fake account creating on your site dramatically – as people will try to submit over the limit by creating multiple accounts.

    2- For paid account holders this limit is very low – as many might have hundreds or thousands of pending URLs not indexed by Majestic SEO.

    3- There should be a much easier way for paid account holders to calculate their allowance.

    • Alex said:

      February 18, 2013 at 2:09 pm

      Hi,

      Thanks for your thoughts.

      We will keep an eye on free submissions to make sure they don’t abuse the system.

      Paid allowances are much bigger than free ones – our dev team will see if there is a better way to present such allowance to the user.

  2. Sander Tamaëla said:

    February 18, 2013 at 8:15 pm

    This is really great, I think most of us SEO have been waiting for this.

    Maybe Majestic could also do something with the crawl data from GWT? Although I understand that comparing dates from different sources might lead to wrong conclusions.

    • Dixon Jones said:

      February 18, 2013 at 8:45 pm

      There is no Google API call to export their link data, but you can easily export their list into CSV and upload it to us to check.

  3. Jan-Willem said:

    February 18, 2013 at 9:01 pm

    I really have been waiting for the functionality! Thanks guys and keep up the good work, Alex, are you also joining the Brighton SEO conference?

    • Dixon Jones said:

      February 18, 2013 at 10:27 pm

      I’ll be there if course – but Alex doesn’t get out much!

  4. PrimeMagazine said:

    February 21, 2013 at 1:56 pm

    Hope this could help our traffic thirsty websites :D
    Submitted 25 URLs.
    Thanks

    • Dixon Jones said:

      February 21, 2013 at 5:08 pm

      It will not help with traffic directly. It only helps with information.

  5. Michael said:

    February 21, 2013 at 3:18 pm

    Is it not dangerous to add so many links per day? how to defend against google ban?

    • Dixon Jones said:

      February 21, 2013 at 5:07 pm

      You are adding links to OUR crawler, not to theirs. We will not crawl huge numbers of links for the same domain – we respect the server’s robots.txt and also obey the crawl-delay command. You will not get more traffic – just a better understanding of how influential each URL is. We do not scrape Google SERPS!, we crawl the actual websites and map the web.

      • Michael said:

        March 01, 2013 at 1:24 pm

        > Now it is clear, thank you very much for your response.

  6. Jeremy said:

    February 22, 2013 at 4:45 pm

    Thanks for making that available to the public free of charge, hopefully it will help optimizing some sites a bit.

  7. Christian said:

    February 23, 2013 at 11:27 am

    Hello!
    Is it free to be a member on your side, or does it cost anything?

    Best regards
    Christian

    • Dixon Jones said:

      February 23, 2013 at 12:04 pm

      We have free and paid offerings. If you register free, you can submit up to 100 urls a day.

      • admin2 said:

        March 17, 2013 at 11:06 pm

        > thank you, can try for free. This is really nice
        Yesterday, I’ve been doing so for movemymom dot net. But whether it is external link increased. how do i check it ?
        Thank you very much.

      • Dixon Jones said:

        March 18, 2013 at 10:00 am

        Hi,
        You can “Create Report” then choose “Create Tracking Report”. This will start recording your links day by day. Also – you can look at the new links found and lost day by day in Site Explorer. For example, here: https://www.majesticseo.com/reports/site-explorer/new-backlinks?folder=&q=example.com&oq=example.com&IndexDataSource=F

  8. haber said:

    February 25, 2013 at 2:13 pm

    Thanks for making that available to the public free of charge, hopefully it will help optimizing some sites a bit.

  9. Farhan Fawzer said:

    February 28, 2013 at 11:42 am

    This is what I love MajesticSEO. They always give users to try out for free as well.

  10. Nathan K Smith said:

    February 28, 2013 at 6:17 pm

    So this tool is to help YOUR crawlers index links or pages faster? Does that mean it helps our links or pages get indexed faster by google?

    • Alex said:

      February 28, 2013 at 7:57 pm

      That’s correct – URL submission will help our crawlers get to data quicker, this would enhance our web graph model and make reported data more robust.

      It will have no effect on Google’s indexing I am afraid.

  11. Veda said:

    March 01, 2013 at 1:34 am

    Well again a great job done by MajesticSeo Team. I just love you guys, you make our research and SEO work easy.
    Love the URL Submitter.

    Thanks

  12. Sam said:

    March 01, 2013 at 7:34 am

    I have about 10000 links that I got from Google webmasters tools, would I able to upload all the links at once?

    If this could be done then it would be a real fun!

    • Dixon Jones said:

      March 01, 2013 at 7:45 am

      We are agnostic about where the urls come from – but as long as Google’s TOS doesn’t mind, then Yes. It will cost a small amount of your monthly resource (so you need a paid account basically, after 100 urls) but certainly you can. Doing this over and over again won’t make any difference though! Just the once is fine.

  13. Trucoseo said:

    March 01, 2013 at 8:32 am

    Great for new pages!

  14. Ardi @BlogBizTutor said:

    March 01, 2013 at 7:21 pm

    Thank you Dixon…
    I like this tool so much. I use it for my blog

  15. Chris said:

    March 01, 2013 at 11:00 pm

    Another reason why MajesticSEO is the best SEO tool available today. Period! Top work guys.

  16. Christian said:

    March 03, 2013 at 11:18 am

    Hello and tanks to Dixon Jones.

    It´s a verry good tool.

    Best regards
    Christian.

  17. Tai game mien phi said:

    March 05, 2013 at 6:18 am

    Thank you Dixon… I like majesticseo

  18. Marco said:

    March 25, 2013 at 6:16 pm

    That is a very good news!I will check it out.

  19. Thien said:

    March 27, 2013 at 3:53 am

    Thanks, I thinks it great !

  20. jupe said:

    March 30, 2013 at 4:39 pm

    I want to put my link on majesticseo but I do not know how! how?

    • Alex said:

      March 30, 2013 at 4:44 pm

      Please use URL submission and note that it is not instant.

  21. Ankur said:

    April 14, 2013 at 8:45 pm

    My website has over 3000 urls. How much does it cost for us to submit all the urls?

    • Dixon Jones said:

      April 15, 2013 at 6:08 am

      First see which ones we already have. Most likely we already have them all. Simply get your own site verified and create an advanced report on your site to check this.