I’m at the O’Reilly Strata Big Data Conference in Santa Clara, CA this week where there’s lots of buzz about the value and reality of big data. It’s a fun time to be part of a hot new market in technology. But, of course, a hot new market brings a new set of challenges.
After talking to several attendees, I would not be surprised if someone took out an advertisement in the San Francisco Guardian that reads:
SEEKING BDT (Big Data Talent)
“Middle-aged attractive company seeks hot-to-trot data geek for mutually enjoyable discrete relationship, mostly involving analytics. Must enjoy long discussions about wild statistical models, short walks to the break room and large quantities of caffeine.”
The feedback from the presentations and attendees at Strata mimics the results from a Big Data survey that Pentaho released last week showing there is a lack of current skills to address new big data technologies such as Hadoop among existing staff and more generally on the market. This is good news for folks looking for jobs in Big Data and a good indication for others who want to learn new skills.
The market has created the perfect storm – the combination of hot new technology mixed with a myriad of very complex systems plus highly complicated statistical models and calculations. This storm is preventing the typical IT generalist or BI expert from applying. More experienced data scientists who can spin models on their head with a twist of a mouse are in high demand. The need to garner value quickly from Big Data means there is little time to look for the “perfect match.”
It seems like new companies and technologies pop up almost every week, each with the promise of business benefits, but with the added cost of high complexity. Shouldn’t things get easier with new technologies?
Pentaho’s Visual MapReduce is a prime example of things getting easier. Getting data out of Hadoop quickly can be a challenge. However, with Visual MapReduce any IT professional could pull the right information from a Hadoop cluster, improve the performance of a MapReduce job and make results available in the optimal format for business users.
New technologies might need new talent, but in the case of Pentaho Visual MapReduce, new technologies might only need new tools to help address them.
Looks like Pentaho is the perfect match.
Chuck Yarbrough Technical Solutions Marketing