← Blog Home
March 27, 2013
Existing data integration and business analytics tools are generally built for relational and structured file data sources, and aren’t architected to take advantage of Hadoop’s massively scalable, but high-latency, distributed data management architecture. Here’s a list of requirements for tools that are truly built for Hadoop. A data integration and data management tool built for Hadoop must: Run In-Hadoop: fully leverage the power of Hadoop’s distributed data storage and processing. It should do this via native integration with the Hadoop Distributed Cache, to automate distribution across the cluster. Generating inefficient Pig scripts doesn’t count. Maximize resource usage on each Hadoop node: each node is a computer, with memory and multiple CPU cores. Tools must fully leverage the power of each node, through multi-threaded parallelized execution of data management tasks and high-performance in-memory caching of intermediate results, customized to the hardware characteristics of nodes. Leverage Hadoop ecosystem tools: tools must natively leverage the rapidly growing ecosystem of Hadoop add-on projects. For example, using Sqoop for bulk loading of huge datasets or Oozie for sophisticated coordination of Hadoop job workflows. The widely distributed nature of Hadoop means accessing data can take minutes,...
March 18, 2013
Summary: Everyone talks about how big data is the key to business success, but the process of getting value from big data is time intensive and complex. Examining the big data analytics workflow provides clues to getting to big data results faster. Most organizations recognize that big data analytics is key to their future business success, but efforts to implement are often slowed due to operational procedures and workflow issues. At the heart of the issue is the big data analytics workflow including loading, ingesting, manipulating, transforming, accessing, modeling and, finally, visualizing and analyzing data. Each step requires manual intervention by IT with a great amount of hand coding and tools that invite mistakes and delays. New technologies such as Hadoop and NoSQL databases also require specialized skills. Once the data is prepared, business users often have new requests to IT for additional data sources and the linear process begins again. Given the potential problems that can crop up in managing and incorporating big data into decision-making processes, organizations need easy-to-use solutions that can address today’s challenges, with the flexibility to adapt to meet future challenges. These solutions require data...
March 12, 2013
Make your voice heard! Participate in the 2013 Wisdom of Crowds ® Business Intelligence Market Study and get a complimentary copy of the study findings. Dresner Advisory Services is inviting all Business Intelligence (BI) users to participate in its annual examination of the state of the BI marketplace focusing on BI usage, deployment trends, and products. The 2013 report will build on previous years’ research and will expand to include questions on the latest and emerging trends such as Collaborative BI, BI in the Cloud, and Embedded BI. It will also rank vendors and products, providing an important tool for organizations seeking to invest in BI solutions. BI users in all roles and throughout all industries are invited to contribute their insight, which should take approximately 15 minutes. The final report is scheduled to be out in late Spring, and qualified survey participants will receive a complimentary copy. Click here to start the survey today!
March 7, 2013
“There isn’t a cluster big enough to hold your ego!” While Gartner describes a “trough of disillusionment” to describe the hangover that follows a period of commercial hype, on the IT side, I see a corresponding “mountain of ego”. Don’t get me wrong. This is not about a sales guy trying to go after the development community - one I proudly belonged to for many years and where I started my journey in this industry. But ask any developer how long it takes to code something and prepare to be amazed by how fast and easy it all is. Three budget cycles later and a couple of delay notifications and we all know better. Agile development tries to cope with this but it’s no silver bullet. As companies plough ahead with big data initiatives, the relationship between IT and the business has never been more important. IT and data integration specialists lead most of today’s big data initiatives; it’s uncharted territory, pioneering work and a place to shine a bright and powerful spotlight on IT’s capabilities and potential to add great value to the business. Challenged by the promise of...
March 6, 2013
L - Dave Henry, Pentaho | R - Ben Llyod, NetApp At Strata 2013 last week, Pentaho had the privilege to host a speaking session with Ben Lloyd, Sr. Program Manager, AutoSupport (ASUP) at NetApp. Ben leads a project called ASUP.Next, which has the goal of implementing a mission-critical data infrastructure for a worldwide customer support program for NetApp’s storage appliances. With design and development assistance from Think Big Analytics and Accenture, NetApp has reached the “go-live” milestone for ASUP.Next and will go into production this month. A Big Data Problem More than 250,000 NetApp devices are deployed worldwide; they “phone home” with device statistics and diagnostic information and represent a continuously growing collection of structured data that must be reliably captured, parsed, interpreted and aggregated to support a large collection of use cases. Ben’s presentation highlighted the business and IT challenges of the legacy AutoSupport environment: The total cost of processing, storing and managing data represents a major ongoing expense ($15M / year). The storage required for ASUP-related data doubles every 16 months -- by the end of 2013 NetApp will have more than 1PB of ASUP-related data available...
March 1, 2013
Erik Nolten In my first blog post for Pentaho, I would like to join our Italian channel partner BNova in a celebration. No, this is not about Italy’s recent elections (I don’t want to get involved in politics!) but a matter closer to my heart. On March 13, I will meet BNova’s Massimiliano Vitali and Serena Arrighi in London to take part in the IT Europa European IT and Software Excellence awards dinner in recognition for the ground-breaking work it did for its customer Infocamere. You can read more about the Infocamere story here . This is a proud milestone in our partnership with BNova , which was one of our very first European channel partners. We worked with Bnova from day one, helping the company design its business strategy, marketing plan and train its people to sell and support Pentaho with confidence. BNova has built a thriving, profitable business whose revenue has tripled since its reseller agreement with Pentaho began in early 2009. All this has taken place against the backdrop of Italy and the Eurozone’s tough economic situation, proving that its services offer public and private sector companies...