Big Data! What On Earth Should Majestic Do With It All?
In 2012, the term “Big Data” was thrown around with as much abandon as “Social Media” was for the previous couple of years.
A one-size-fits-many-things moniker, “Big Data” (like its cousin “Social Media”) has been the subject of many a blog post, conference session and CMO’s sleepless night, as industries realize its essence is important, but are not quite sure what to do with it (a bit like they did (still do?) with “social media”).
Do a search for the phrase over the last month and you can see a bit of a backlash against startups using the term “Big” in front of their wares and all that information we know and love.
It’s always been “Big” in relative terms to whatever we’ve been mining; it’s just that now (thanks to the web) it’s getting bigger.
Photo on Flickr D Sharon Pruitt
An article in the New York Times offers:
“Big Data proponents point to the Internet for examples of triumphant data businesses, notably Google. But many of the Big Data techniques of math modeling, predictive algorithms and artificial intelligence software were first widely applied on Wall Street.”
Others are suggesting what we mean when we say it, is a “smart use of data”, and that marketers are just using it as a “pernicious” way to rebrand an old adage and start a new fad.
The NYT piece mentions a McKinsey report that concluded that soon the US will need 140-190 thousand workers with “deep analytical” expertise and 1.5 million “data-literate managers”.
It goes on to quote (and here’s the gem) Rachel Schutt, a senior statistician at Google Research, who defines a good data scientist as someone who’s not just savvy with math(s) and computer science, but:
“…someone who has a deep, wide-ranging curiosity, is innovative and is guided by experience as well as data.”
She must be talking about SEOs surely?!
As seasoned professionals in the search industry, we know there’s more to a successful web strategy than a bunch of numbers (just listen to Jim Sterne talk about the importance of asking your customers how they “feel”) but it’s those numbers, coupled with curiosity about human behaviour, that keeps us pushing the envelope and wanting to do better.
At Majestic, we have a lot of numbers to talk about. A database with 4 trillion URLs is nothing to sniff at, but (as my wife keeps reminding me) it’s what you do with it that really counts.
So beyond SEO, we’re starting to think of other ways businesses can use all this gorgeous data. We’re talking to the finance industry about how they can pre-screen credit applications from ecommerce sites. We’re taking to social platforms about how they can use the data to improve their algorithms, and to advertising networks about how they use Majestic with pricing or fraud detection or targeting.
The more we think about it and get “curious”, the more “innovative” we think we could be.
So my question to you is: What DO you or what WOULD you use Majestic data for other than link-building and SEO?
We’re grateful for all the kudos we get for being agile enough in bringing new features, bells and whistles to the platform, but we want to do more and be better.
So we’re curious to know what would you do with a distributed crawler and 4,060,890,688,027 unique URLs?
Let us know in the comments below.
Oh and please keep it useful and “White Hat”!
To slightly misquote Rachel Schutt in that NYT article, “We only worship THAT machine.”