Welcome!

Eclipse Authors: Sematext Blog, Marcin Warpechowski, Trevor Parsons, Michael Meiner, Carmen Gonzalez

News Feed Item

Elasticsearch Now Certified on Cloudera Enterprise 5; Releases New Hadoop Connector

Elasticsearch Unlocks Potential for Businesses to Get Immediate Insights out of Data They Store in Hadoop

LOS ALTOS, CA and AMSTERDAM, THE NETHERLANDS -- (Marketwired) -- 06/19/14 -- Elasticsearch, Inc., the company on a mission to make data useful to businesses by delivering the world's most advanced search and analytics engine, today announced the 2.0 release of its Hadoop connector, Elasticsearch for Apache Hadoop, along with certification on Cloudera Enterprise 5. With Cloudera certification, Elasticsearch is now compatible across all Apache-based Hadoop distributions, including HortonWorks and MapR, helping businesses extract immediate insights regardless of where their hundreds of terabytes or even petabytes of data are stored.

Elasticsearch is the search and analytics engine behind the ELK stack, which also utilizes Logstash, a log management tool, and Kibana's powerful data visualization capabilities to help businesses pull vital information from their data stores. When used in conjunction with Hadoop, organizations no longer need to run a batch process and wait hours to analyze their data -- Elasticsearch for Apache Hadoop can pipe data to Elasticsearch for indexing as it's being generated, making it available for search and analysis in a matter of seconds. Kibana can also be used to explore massive amounts of data in Elasticsearch through easy-to-generate pie charts, bar graphs, scatter plots, histograms, and more.

How Businesses Leverage Elasticsearch and Hadoop
Elasticsearch is becoming the critical piece of pulling data from any environment and getting it into the hands of developers, engineering leads, CTOs, and CIOs who need insight into moving parts of their business at the rate they are happening. Customer examples include:

  • Klout, which stores petabytes of its 400 million+ users' data in a Hadoop Distributed File System and connects it to Elasticsearch. Klout query results, used to build targeted marketing campaigns, are delivered in seconds rather than minutes.
  • MutualMind, which enables customers like AT&T, Kraft, Nestle, and Starbucks to monitor their brands on social networks. After its Hadoop batches started taking 15+ minutes, MutualMind moved to Elasticsearch to power its real-time analytics, while utilizing Hadoop for statistical analysis.
  • An international financial services firm that uses Elasticsearch to analyze its access logs in just minutes instead of having to wait hours to run MapReduce jobs. Because Elasticsearch provided insights so quickly on the firm's large amounts of data, they've been able to increase the window of data they can analyze from one hour to a full week.

Key Features of Elasticsearch for Apache Hadoop

  • The ability to read and write data between Hadoop and Elasticsearch: Lets businesses get immediate, actionable insights by writing their data to Elasticsearch for real-time search and analysis. Complex jobs that would normally take minutes or hours to run in Hadoop can be handled quickly in Elasticsearch and read right back to Hadoop.
  • Native integration and support for popular Hadoop libraries: Lets users run queries natively on Hadoop through MapReduce, Hive, Pig, or Cascading APIs.
  • Snapshot/Restore: Makes it easy to take a snapshot of data within Elasticsearch -- perhaps a year's worth -- and archive it in Hadoop. At any time, the snapshot can be restored back to Elasticsearch for additional analysis.

Supporting Quotes from Cloudera, Elasticsearch, and Klout

Steven Schuurman, co-founder and CEO, Elasticsearch
"Hadoop was created to store and archive data at a massive scale, but businesses need to be able to ask, iterate, and extract actionable insights from this data -- which is what we designed our products for. With today's certification from Cloudera, Elasticsearch now works with all Apache-based Hadoop distributions, and with it, solves the last mile of big data Hadoop deployments by getting big insights, fast."

Tim Stevens, vice president of Business and Corporate Development at Cloudera
"Part of our mission at Cloudera is to support and promote an open architecture and allow customers to leverage their technology investments. Together, Cloudera and Elasticsearch provide businesses with a solution that allows them to get insight out of massive amounts of data."

Felipe Oliveria, director of Engineering, Backend for Klout
"Elasticsearch has a very good integration with Hadoop. It allows us to export a Hive table to an index on Elasticsearch very easily. HBase is a great data store, and it allows random access to the data, which Elasticsearch is perfect for. Elasticsearch fits very nicely into our data pipeline."

Because Elasticsearch works across distributed, diverse environments, engineers can search, extract, clean up and analyze data whether it comes from log events, social media activity, support tickets, website analytics or product interactions. Thousands of businesses worldwide continue to adopt Elasticsearch to store, search and analyze any type of data in real time, including Bloomberg, Comcast, eBay, Facebook, GitHub, Mayo Clinic, McGraw-Hill, Netflix, The New York Times, Target, Verizon, WordPress and Yelp.

To download Elasticsearch for Apache Hadoop, visit http://www.elasticsearch.org/overview/hadoop/. To find out more about Elasticsearch, visit www.elasticsearch.com.

Upcoming webinar: Real-time Analytics and Anomaly Detection using Elasticsearch and Apache Hadoop
On Wednesday, August 20, 2014 at 9:00am PT/12:00pm ET, Elasticsearch will host a webinar that goes over the features and benefits of Elasticsearch for Apache Hadoop, including a demonstration of how to use it as a platform to perform search and analytics, such as anomaly detection. To register, visit http://www.elasticsearch.org/webinars/elasticsearch-and-apache-hadoop.

About Elasticsearch, Inc.
Elasticsearch is on a mission to make massive amounts of data usable for businesses everywhere by delivering the world's most advanced search and analytics engine. With a laser focus on achieving the best user experience imaginable, the Elasticsearch ELK stack -- comprised of Elasticsearch, Logstash and Kibana -- has become one of the most popular and rapidly growing open source solutions in the market. Used by thousands of enterprises in virtually every industry today, Elasticsearch, Inc. provides production support, development support and training for the full ELK stack.

Elasticsearch, Inc. was founded in 2012 by the people behind the Elasticsearch and Apache Lucene open source projects. Since its initial release, Elasticsearch has more than 9 million cumulative downloads. Elasticsearch, Inc. is backed by Benchmark Capital, Index Ventures and NEA, with headquarters in Amsterdam and Los Altos, California, and offices around the world.

More Stories By Marketwired .

Copyright © 2009 Marketwired. All rights reserved. All the news releases provided by Marketwired are copyrighted. Any forms of copying other than an individual user's personal reference without express written permission is prohibited. Further distribution of these materials is strictly forbidden, including but not limited to, posting, emailing, faxing, archiving in a public database, redistributing via a computer network or in a printed form.

@ThingsExpo Stories

ARMONK, N.Y., Nov. 20, 2014 /PRNewswire/ --  IBM (NYSE: IBM) today announced that it is bringing a greater level of control, security and flexibility to cloud-based application development and delivery with a single-tenant version of Bluemix, IBM's platform-as-a-service. The new platform enables developers to build ap...

Building low-cost wearable devices can enhance the quality of our lives. In his session at Internet of @ThingsExpo, Sai Yamanoor, Embedded Software Engineer at Altschool, provided an example of putting together a small keychain within a $50 budget that educates the user about the air quality in their surroundings. He also provided examples such as building a wearable device that provides transit or recreational information. He then reviewed the resources available to build wearable devices at home including open source hardware, the raw materials required and the options available to power s...
The Internet of Things promises to transform businesses (and lives), but navigating the business and technical path to success can be difficult to understand. In his session at @ThingsExpo, Sean Lorenz, Technical Product Manager for Xively at LogMeIn, demonstrated how to approach creating broadly successful connected customer solutions using real world business transformation studies including New England BioLabs and more.
Since 2008 and for the first time in history, more than half of humans live in urban areas, urging cities to become “smart.” Today, cities can leverage the wide availability of smartphones combined with new technologies such as Beacons or NFC to connect their urban furniture and environment to create citizen-first services that improve transportation, way-finding and information delivery. In her session at @ThingsExpo, Laetitia Gazel-Anthoine, CEO of Connecthings, will focus on successful use cases.
Enthusiasm for the Internet of Things has reached an all-time high. In 2013 alone, venture capitalists spent more than $1 billion dollars investing in the IoT space. With "smart" appliances and devices, IoT covers wearable smart devices, cloud services to hardware companies. Nest, a Google company, detects temperatures inside homes and automatically adjusts it by tracking its user's habit. These technologies are quickly developing and with it come challenges such as bridging infrastructure gaps, abiding by privacy concerns and making the concept a reality. These challenges can't be addressed w...
The Domain Name Service (DNS) is one of the most important components in networking infrastructure, enabling users and services to access applications by translating URLs (names) into IP addresses (numbers). Because every icon and URL and all embedded content on a website requires a DNS lookup loading complex sites necessitates hundreds of DNS queries. In addition, as more internet-enabled ‘Things' get connected, people will rely on DNS to name and find their fridges, toasters and toilets. According to a recent IDG Research Services Survey this rate of traffic will only grow. What's driving t...
The Internet of Things is a misnomer. That implies that everything is on the Internet, and that simply should not be - especially for things that are blurring the line between medical devices that stimulate like a pacemaker and quantified self-sensors like a pedometer or pulse tracker. The mesh of things that we manage must be segmented into zones of trust for sensing data, transmitting data, receiving command and control administrative changes, and peer-to-peer mesh messaging. In his session at @ThingsExpo, Ryan Bagnulo, Solution Architect / Software Engineer at SOA Software, focused on desi...
"For over 25 years we have been working with a lot of enterprise customers and we have seen how companies create applications. And now that we have moved to cloud computing, mobile, social and the Internet of Things, we see that the market needs a new way of creating applications," stated Jesse Shiah, CEO, President and Co-Founder of AgilePoint Inc., in this SYS-CON.tv interview at 15th Cloud Expo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
The Industrial Internet revolution is now underway, enabled by connected machines and billions of devices that communicate and collaborate. The massive amounts of Big Data requiring real-time analysis is flooding legacy IT systems and giving way to cloud environments that can handle the unpredictable workloads. Yet many barriers remain until we can fully realize the opportunities and benefits from the convergence of machines and devices with Big Data and the cloud, including interoperability, data security and privacy.
The Internet of Things is tied together with a thin strand that is known as time. Coincidentally, at the core of nearly all data analytics is a timestamp. When working with time series data there are a few core principles that everyone should consider, especially across datasets where time is the common boundary. In his session at Internet of @ThingsExpo, Jim Scott, Director of Enterprise Strategy & Architecture at MapR Technologies, discussed single-value, geo-spatial, and log time series data. By focusing on enterprise applications and the data center, he will use OpenTSDB as an example t...
The industrial software market has treated data with the mentality of “collect everything now, worry about how to use it later.” We now find ourselves buried in data, with the pervasive connectivity of the (Industrial) Internet of Things only piling on more numbers. There’s too much data and not enough information. In his session at @ThingsExpo, Bob Gates, Global Marketing Director, GE’s Intelligent Platforms business, to discuss how realizing the power of IoT, software developers are now focused on understanding how industrial data can create intelligence for industrial operations. Imagine ...
Cultural, regulatory, environmental, political and economic (CREPE) conditions over the past decade are creating cross-industry solution spaces that require processes and technologies from both the Internet of Things (IoT), and Data Management and Analytics (DMA). These solution spaces are evolving into Sensor Analytics Ecosystems (SAE) that represent significant new opportunities for organizations of all types. Public Utilities throughout the world, providing electricity, natural gas and water, are pursuing SmartGrid initiatives that represent one of the more mature examples of SAE. We have s...
There is no doubt that Big Data is here and getting bigger every day. Building a Big Data infrastructure today is no easy task. There are an enormous number of choices for database engines and technologies. To make things even more challenging, requirements are getting more sophisticated, and the standard paradigm of supporting historical analytics queries is often just one facet of what is needed. As Big Data growth continues, organizations are demanding real-time access to data, allowing immediate and actionable interpretation of events as they happen. Another aspect concerns how to deliver ...
Scott Jenson leads a project called The Physical Web within the Chrome team at Google. Project members are working to take the scalability and openness of the web and use it to talk to the exponentially exploding range of smart devices. Nearly every company today working on the IoT comes up with the same basic solution: use my server and you'll be fine. But if we really believe there will be trillions of these devices, that just can't scale. We need a system that is open a scalable and by using the URL as a basic building block, we open this up and get the same resilience that the web enjoys.
The Internet of Things will greatly expand the opportunities for data collection and new business models driven off of that data. In her session at @ThingsExpo, Esmeralda Swartz, CMO of MetraTech, discussed how for this to be effective you not only need to have infrastructure and operational models capable of utilizing this new phenomenon, but increasingly service providers will need to convince a skeptical public to participate. Get ready to show them the money!
The 3rd International Internet of @ThingsExpo, co-located with the 16th International Cloud Expo - to be held June 9-11, 2015, at the Javits Center in New York City, NY - announces that its Call for Papers is now open. The Internet of Things (IoT) is the biggest idea since the creation of the Worldwide Web more than 20 years ago.
Cloud Expo 2014 TV commercials will feature @ThingsExpo, which was launched in June, 2014 at New York City's Javits Center as the largest 'Internet of Things' event in the world.
"People are a lot more knowledgeable about APIs now. There are two types of people who work with APIs - IT people who want to use APIs for something internal and the product managers who want to do something outside APIs for people to connect to them," explained Roberto Medrano, Executive Vice President at SOA Software, in this SYS-CON.tv interview at Cloud Expo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
SYS-CON Media announced that Splunk, a provider of the leading software platform for real-time Operational Intelligence, has launched an ad campaign on Big Data Journal. Splunk software and cloud services enable organizations to search, monitor, analyze and visualize machine-generated big data coming from websites, applications, servers, networks, sensors and mobile devices. The ads focus on delivering ROI - how improved uptime delivered $6M in annual ROI, improving customer operations by mining large volumes of unstructured data, and how data tracking delivers uptime when it matters most.
DevOps Summit 2015 New York, co-located with the 16th International Cloud Expo - to be held June 9-11, 2015, at the Javits Center in New York City, NY - announces that it is now accepting Keynote Proposals. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long development cycles that produce software that is obsolete at launch. DevOps may be disruptive, but it is essential.