Eclipse Authors: Pat Romanski, Elizabeth White, Liz McMillan, David H Deans, JP Morgenthal

Blog Feed Post

Hadoop’s Impact on the Future of Data Management: Insights from Mike Olson at Strata and Hadoop World 2013


big_data_5Below and at this link is a video of Mike Olson at the opening of the 2013 Strata Conference and Hadoop World. The context in this discussion is important for technologists, strategists, analysts and executives alike, it provides insights in easy to understand ways and succinctly articulates the impact of some key trends in the environment.

The ecosystem around Apache Hadoop has continued to mature so updates like this from one of its key leaders are critically important.

Mike opened with a great overview of where the community has come in the last five years:

  • In 2008 the Big Data meme had not happened yet. And you probably hadn’t heard of Hadoop yet.
  • In 2009 the first Hadoop world was held and over 700 people showed up. This year, 3000 people attended in a sold out venue.
  • Now consider the many vendors in the ecosystem.
  • And consider the big trends.
  • When Hadoop was born was a compliment to traditional data processing. It was off on the side. Good for batch and for storage but could not handle real time. Much of the market did not pay much attention. But real time was always desired. Just a year ago Cloudera announced a real time platform, Impala, an open source real time SQL engine.

For context on where were are today and where we are going Mike reviewed that:

  • Other real time capabilities have been added including Cloudera search. In the single year since they have been announced over 5000 enterprises have added Impala and Search. Real time has always mattered.
  • Now that real time and search are both available, more work can be done on the real platform. Now other applications and uses can be supported on Hadoop. This platform is attracting work and attracting data. And it is attracting more and more users.
  • At enterprise deployments are showing a strong trend. Hadoop is emerging as an enterprise data hub. This meme is big in the industry now.

What is a data hub? A scale-out, affordable, reliable platform. Can hold any data in any format for as long as you want it. It is a storage layer with security built in that can do access control, auditing, logging, providence of data. And a secure storage substrate would also require a rich collection of engines for working with data. You want query, search, machine learning and analytics in place without moving it out. That collection of capabilities is hugely valuable and lets you work with the data where it  lives. But still this is not a hub. A hub needs to connect to the infrastructure you already rely on. That makes a hub and makes this concept very virtuous. This is something new. It is an enterprise data hub. This is a very big deal.

Cloudera announced, via Mike, the release of Cloudera 5, the industry’s first Enterprise Data Hub.

Bottom line of this new Enterprise Data Hub capability: Scale out storage, security, good data governance capability, a rich collection of engines for working on the data in place and delivering results to your systems and people.

For more see Mike expand on this concept here



Read the original blog entry...

More Stories By Bob Gourley

Bob Gourley writes on enterprise IT. He is a founder of Crucial Point and publisher of CTOvision.com

IoT & Smart Cities Stories
Business professionals no longer wonder if they'll migrate to the cloud; it's now a matter of when. The cloud environment has proved to be a major force in transitioning to an agile business model that enables quick decisions and fast implementation that solidify customer relationships. And when the cloud is combined with the power of cognitive computing, it drives innovation and transformation that achieves astounding competitive advantage.
Machine learning has taken residence at our cities' cores and now we can finally have "smart cities." Cities are a collection of buildings made to provide the structure and safety necessary for people to function, create and survive. Buildings are a pool of ever-changing performance data from large automated systems such as heating and cooling to the people that live and work within them. Through machine learning, buildings can optimize performance, reduce costs, and improve occupant comfort by ...
René Bostic is the Technical VP of the IBM Cloud Unit in North America. Enjoying her career with IBM during the modern millennial technological era, she is an expert in cloud computing, DevOps and emerging cloud technologies such as Blockchain. Her strengths and core competencies include a proven record of accomplishments in consensus building at all levels to assess, plan, and implement enterprise and cloud computing solutions. René is a member of the Society of Women Engineers (SWE) and a m...
Early Bird Registration Discount Expires on August 31, 2018 Conference Registration Link ▸ HERE. Pick from all 200 sessions in all 10 tracks, plus 22 Keynotes & General Sessions! Lunch is served two days. EXPIRES AUGUST 31, 2018. Ticket prices: ($1,295-Aug 31) ($1,495-Oct 31) ($1,995-Nov 12) ($2,500-Walk-in)
According to Forrester Research, every business will become either a digital predator or digital prey by 2020. To avoid demise, organizations must rapidly create new sources of value in their end-to-end customer experiences. True digital predators also must break down information and process silos and extend digital transformation initiatives to empower employees with the digital resources needed to win, serve, and retain customers.
IoT is rapidly becoming mainstream as more and more investments are made into the platforms and technology. As this movement continues to expand and gain momentum it creates a massive wall of noise that can be difficult to sift through. Unfortunately, this inevitably makes IoT less approachable for people to get started with and can hamper efforts to integrate this key technology into your own portfolio. There are so many connected products already in place today with many hundreds more on the h...
Digital Transformation: Preparing Cloud & IoT Security for the Age of Artificial Intelligence. As automation and artificial intelligence (AI) power solution development and delivery, many businesses need to build backend cloud capabilities. Well-poised organizations, marketing smart devices with AI and BlockChain capabilities prepare to refine compliance and regulatory capabilities in 2018. Volumes of health, financial, technical and privacy data, along with tightening compliance requirements by...
Charles Araujo is an industry analyst, internationally recognized authority on the Digital Enterprise and author of The Quantum Age of IT: Why Everything You Know About IT is About to Change. As Principal Analyst with Intellyx, he writes, speaks and advises organizations on how to navigate through this time of disruption. He is also the founder of The Institute for Digital Transformation and a sought after keynote speaker. He has been a regular contributor to both InformationWeek and CIO Insight...
Digital Transformation is much more than a buzzword. The radical shift to digital mechanisms for almost every process is evident across all industries and verticals. This is often especially true in financial services, where the legacy environment is many times unable to keep up with the rapidly shifting demands of the consumer. The constant pressure to provide complete, omnichannel delivery of customer-facing solutions to meet both regulatory and customer demands is putting enormous pressure on...
Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life settlement products to hedge funds and investment banks. After, he co-founded a revenue cycle management company where he learned about Bitcoin and eventually Ethereal. Andrew's role at ConsenSys Enterprise is a mul...