Hortonworks has announced the general availability of DataFlow 3.0, the newest version of its platform for streaming data analytics. The Hadoop distributor says the growth of IoT (Internet of Things) devices and related data will make streaming a crucial part of future analytic applications:
"To stay competitive in today's interconnected world, businesses must harness the insights from data everywhere," said Scott Gnau, chief technology officer at Hortonworks. "Increasingly, this means from point of creation on connected devices and it's crucial to make decisions as close as possible to the edge device."
The centerpiece of HDF 3.0 is Streaming Analytics manager, a drag-and-drop interface that even non-programmers can use to build streaming applications.
While HDF 3.0 leverages Apache NiFi under the hood, crucially it also supports other streaming engines, such as Apache Kafka and Apache Storm, a move that's supposed to add flexibility and save time when building streaming applications.
Hortonworks is also announcing that HDF 3.0 will be available on IBM Power Systems servers. The systems are geared for big data workloads through features such as multithreading, multi-level data caching and large amounts of main memory.
Analysis: Can Real-Time Be Manageable This Time?
There are many options but few default choices for handling streaming data processing and streaming analytics scenarios, says Constellation Research VP and principal analyst Doug Henschen. "Most organizations wade in slowly, cobbling together many technology piece parts and struggling with complexity. Some vendors do offer more comprehensive systems for streaming, but the proprietary complex event processing (CEP) systems of the past, for example, tended to be expensive overkill."
HDF provides a more modern, open-source option for streaming analytics and the new features in 3.0 fill gaps where users have struggled with complexity, Henschen adds.
The industry is still in the early days of streaming demand despite all the hype around IoT, and to date organizations have tended to experiment using one or two scenarios by adding a few piece-part technologies, he says: "Too often it’s only in hindsight, once streaming scenarios have proliferated, that companies recognize that they’re struggling with complexity, long development times and a lack for repeatability. HDF is akin to a factory, more for building out and supporting streaming analytics workflows. The best prospects for it are companies that see the promise of real-time performance but that have been burned by complexity."
24/7 Access to Constellation Insights
Subscribe today for unrestricted access to expert analyst views on breaking news.
