Welcome!

Eclipse Authors: JP Morgenthal, Yeshim Deniz, Liz McMillan, Elizabeth White, XebiaLabs Blog

Related Topics: Java IoT, Industrial IoT, Weblogic, Eclipse, IoT User Interface, Recurring Revenue

Java IoT: Article

Column Store, In-Memory, MPP Databases and Oracle

How Oracle implements latest developments in database world

( For latest information on Oracle 12c database update please refer to the following article: Oracle 12c Database and How It Relates to SAP Hana )

RDBMSs are stable and mature products. While there is nothing radically new on horizon that would challenge Codd's relational theory and related advances in data processing there are some developments that force established vendors like Oracle to come up with new features and products.

Column Stores and Oracle
Column store concept has been around for quite a while. Vendors like HP Vertica grabbed some market share in data warehousing segment with their column store,  MPP databases. Oracle Exadata is offering Hybrid Columnar Compression -  solution that modifies Oracle standard row based storage (NSM, or row-major) into proprietary format that is probably closer to PAX (as opposed to DSM) classification. Rows of data are reorganized, broken down into columns, compressed and stored in Compression Units which consist of multiple Oracle data blocks. CU is physically implemented as a standard Oracle single column chained row. (Description of CU layout is based on marvelous article written by Oracle expert  Jonathan Lewis).

This is truly hybrid design, i.e., column store is implemented on top of a standard row store.

It is not in the scope of this article to discuss pros and cons of this implementation from performance, locking, compression and other points of view. I will just mention that HCC requires change in standard operating procedures and methods.

MPP (shared nothing) and Oracle
Column-based stores like HP Vertica use Multiple Parallel Processing, shared nothing design to enhance performance by bringing processing closer to data, i.e., data is processed in parallel on the node where it resides. Volume of data that is moved around is reduced with the additional benefit of CPU and data proximity.

Oracle's implementation of this idea could be classified as asymmetric MPP. Oracle Exadata uses offloading to storage layer, Smart Scan, Storage Indexes and other techniques to improve performance.

Storage layer (Exadata cells) are tasked with as much work as possible to reduce load on database server and network. Each Exadata storage cell has the ability to perform some parts of data processing operations as well as decompression.

IMDB and Oracle
SAP relatively recently released Hana - a fully functional in-memory RDBMS, targeted for both OLTP and OLAP applications. Hana operates on the premise that whole database is in memory and not on disk. Majority of data processing is now pointer based arithmetic, so whole sections of RDBMS code related to moving data back and forth between disk, RAM and CPU are not needed any more. This is all possible because memory is more affordable and abundant, so much so that most of modern OLTP databases can completely fit within modern server's RAM.

Oracle puchased Times Ten in-memory database, but marketed it mostly as caching layer to standard Oracle database. Times Ten is not marketed as stand alone IMDB the way SAP Hana is.

Oracle database can have Flash Cache devices configured as an extension of SGA for better performance (via database parameter), or for database logging purposes.

Exadata can be configured with terabytes of Flash Cache memory for database caching and to serve as solid state disk. This is not memory directly accessible by CPU though ( DRAM ), i.e., Oracle database accesses Flash Cache via PCI interface and IO operating system calls. In other words, Flash is treated the same as disk, with all negative consequences of such approach regarding code complexity and performance. The latest release of Exadata performs writes directly to flash cache first to improve performance. We should expect more optimizations that will try to better utilize abundance of various types of memory. Expected scenario could be similar to Microsoft Hekaton project is also about adding IMDB features to SQL Server ( tables can be loaded in memory and processed in IMDB fashion, with reduced latching and locking; perhaps choice between different storage engines will be possible).

Conclusion
Oracle will probably continue to execute on strategy that worked well in the past - gradual inclusion of new technologies into its core RDBMS product (like it did with programmable server, OODBMS, Internet database, XML, partitioning, etc.). None of the latest developments in server technologies and database world is as seismic as introduction of RDBMS, client-server and Internet computing was. Oracle was so far successful  in modifying its database engine to adjust to the changes in hardware and data processing methods. We expect this trend to continue, as pace of technological innovation is somewhat slowing down and no truly disruptive changes are on horizon. Oracle solutions are designed to introduce and take advantage of these new (old) technologies and avoid cannibalizing existing profits. Intent is to maximally protect existing legacy RDBMS software revenues and integrate and sell products that came with new hardware and software acquisitions. Oracle Exadata, for example, could be viewed just as intelligent, Oracle database aware and Oracle produced SAN, bundled with database server. It is perhaps safe strategy in an environment where even mediocre and lackluster repackaging, modifying and integrating acquisitions is not seriously challenged by radical new ideas or strong, competitive implementations of existing concepts and technologies. SAP Hana, for example, is also unification layer built on top of in-house built or acquired products.

More Stories By Ranko Mosic

Ranko Mosic, BScEng, is specializing in Big Data/Data Architecture consulting services ( database/data architecture, machine learning ). His clients are in finance, retail, telecommunications industries. Ranko is welcoming inquiries about his availability for consulting engagements and can be reached at 408-757-0053 or [email protected]

@ThingsExpo Stories
The many IoT deployments around the world are busy integrating smart devices and sensors into their enterprise IT infrastructures. Yet all of this technology – and there are an amazing number of choices – is of no use without the software to gather, communicate, and analyze the new data flows. Without software, there is no IT. In this power panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, Dave McCarthy, Director of Products at Bsquare Corporation; Alan Williamson, Principal...
Video experiences should be unique and exciting! But that doesn’t mean you need to patch all the pieces yourself. Users demand rich and engaging experiences and new ways to connect with you. But creating robust video applications at scale can be complicated, time-consuming and expensive. In his session at @ThingsExpo, Zohar Babin, Vice President of Platform, Ecosystem and Community at Kaltura, discussed how VPaaS enables you to move fast, creating scalable video experiences that reach your aud...
Amazon has gradually rolled out parts of its IoT offerings in the last year, but these are just the tip of the iceberg. In addition to optimizing their back-end AWS offerings, Amazon is laying the ground work to be a major force in IoT – especially in the connected home and office. Amazon is extending its reach by building on its dominant Cloud IoT platform, its Dash Button strategy, recently announced Replenishment Services, the Echo/Alexa voice recognition control platform, the 6-7 strategic...
Whether your IoT service is connecting cars, homes, appliances, wearable, cameras or other devices, one question hangs in the balance – how do you actually make money from this service? The ability to turn your IoT service into profit requires the ability to create a monetization strategy that is flexible, scalable and working for you in real-time. It must be a transparent, smoothly implemented strategy that all stakeholders – from customers to the board – will be able to understand and comprehe...
More and more brands have jumped on the IoT bandwagon. We have an excess of wearables – activity trackers, smartwatches, smart glasses and sneakers, and more that track seemingly endless datapoints. However, most consumers have no idea what “IoT” means. Creating more wearables that track data shouldn't be the aim of brands; delivering meaningful, tangible relevance to their users should be. We're in a period in which the IoT pendulum is still swinging. Initially, it swung toward "smart for smart...
Complete Internet of Things (IoT) embedded device security is not just about the device but involves the entire product’s identity, data and control integrity, and services traversing the cloud. A device can no longer be looked at as an island; it is a part of a system. In fact, given the cross-domain interactions enabled by IoT it could be a part of many systems. Also, depending on where the device is deployed, for example, in the office building versus a factory floor or oil field, security ha...
An IoT product’s log files speak volumes about what’s happening with your products in the field, pinpointing current and potential issues, and enabling you to predict failures and save millions of dollars in inventory. But until recently, no one knew how to listen. In his session at @ThingsExpo, Dan Gettens, Chief Research Officer at OnProcess, discussed recent research by Massachusetts Institute of Technology and OnProcess Technology, where MIT created a new, breakthrough analytics model for s...
SYS-CON Events has announced today that Roger Strukhoff has been named conference chair of Cloud Expo and @ThingsExpo 2017 New York. The 20th Cloud Expo and 7th @ThingsExpo will take place on June 6-8, 2017, at the Javits Center in New York City, NY. "The Internet of Things brings trillions of dollars of opportunity to developers and enterprise IT, no matter how you measure it," stated Roger Strukhoff. "More importantly, it leverages the power of devices and the Internet to enable us all to im...
Data is the fuel that drives the machine learning algorithmic engines and ultimately provides the business value. In his session at 20th Cloud Expo, Ed Featherston, director/senior enterprise architect at Collaborative Consulting, will discuss the key considerations around quality, volume, timeliness, and pedigree that must be dealt with in order to properly fuel that engine.
In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at D...
The Internet of Things will challenge the status quo of how IT and development organizations operate. Or will it? Certainly the fog layer of IoT requires special insights about data ontology, security and transactional integrity. But the developmental challenges are the same: People, Process and Platform and how we integrate our thinking to solve complicated problems. In his session at 19th Cloud Expo, Craig Sproule, CEO of Metavine, demonstrated how to move beyond today's coding paradigm and sh...
What happens when the different parts of a vehicle become smarter than the vehicle itself? As we move toward the era of smart everything, hundreds of entities in a vehicle that communicate with each other, the vehicle and external systems create a need for identity orchestration so that all entities work as a conglomerate. Much like an orchestra without a conductor, without the ability to secure, control, and connect the link between a vehicle’s head unit, devices, and systems and to manage the ...
Financial Technology has become a topic of intense interest throughout the cloud developer and enterprise IT communities. Accordingly, attendees at the upcoming 20th Cloud Expo at the Javits Center in New York, June 6-8, 2017, will find fresh new content in a new track called FinTech.
"We're a cybersecurity firm that specializes in engineering security solutions both at the software and hardware level. Security cannot be an after-the-fact afterthought, which is what it's become," stated Richard Blech, Chief Executive Officer at Secure Channels, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
"Once customers get a year into their IoT deployments, they start to realize that they may have been shortsighted in the ways they built out their deployment and the key thing I see a lot of people looking at is - how can I take equipment data, pull it back in an IoT solution and show it in a dashboard," stated Dave McCarthy, Director of Products at Bsquare Corporation, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
Everyone knows that truly innovative companies learn as they go along, pushing boundaries in response to market changes and demands. What's more of a mystery is how to balance innovation on a fresh platform built from scratch with the legacy tech stack, product suite and customers that continue to serve as the business' foundation. In his General Session at 19th Cloud Expo, Michael Chambliss, Head of Engineering at ReadyTalk, discussed why and how ReadyTalk diverted from healthy revenue and mor...
Whether your IoT service is connecting cars, homes, appliances, wearable, cameras or other devices, one question hangs in the balance – how do you actually make money from this service? The ability to turn your IoT service into profit requires the ability to create a monetization strategy that is flexible, scalable and working for you in real-time. It must be a transparent, smoothly implemented strategy that all stakeholders – from customers to the board – will be able to understand and comprehe...
The Internet of Things (IoT) promises to simplify and streamline our lives by automating routine tasks that distract us from our goals. This promise is based on the ubiquitous deployment of smart, connected devices that link everything from industrial control systems to automobiles to refrigerators. Unfortunately, comparatively few of the devices currently deployed have been developed with an eye toward security, and as the DDoS attacks of late October 2016 have demonstrated, this oversight can ...
You have great SaaS business app ideas. You want to turn your idea quickly into a functional and engaging proof of concept. You need to be able to modify it to meet customers' needs, and you need to deliver a complete and secure SaaS application. How could you achieve all the above and yet avoid unforeseen IT requirements that add unnecessary cost and complexity? You also want your app to be responsive in any device at any time. In his session at 19th Cloud Expo, Mark Allen, General Manager of...
"ReadyTalk is an audio and web video conferencing provider. We've really come to embrace WebRTC as the platform for our future of technology," explained Dan Cunningham, CTO of ReadyTalk, in this SYS-CON.tv interview at WebRTC Summit at 19th Cloud Expo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.