In recent weeks there have been announcements from many of the big names of the technology industry of ‘Event Engines’ in association with other parts of their Cloud based product sets. In addition there are a number of startups also offering Event Processing engines with a variety of different benefits that will run on an available Cloud service. Most are specifically announced as part of Internet of Things, IoT, solutions, or sometimes IoT suites, but what’s different from BPM business process engines?

The better question is why are these announced specifically for IoT solutions, and how do they fit with the development of IoT architectures as covered in the preceding two blogs in this series?

Research report now available: The Foundational Elements for the Internet of Things (IoT)

The sheer volume of devices and resulting data flows makes real time Analytics essential to ‘read’ and tag the valuable ‘Insights’, but real business value lies in using Event Hubs/Engines coupled with Analytical output to respond, or ‘react’, with sophisticated Business valuable actions.

Event Processing is not new, but Business valuable IoT deployments demand Complex Event Processing, CEP, to combine data from multiple sources into meaningful events then orchestrate process elements into optimized Process responses. Add the demand for real time, the unique combination of both ‘push’ and ‘pull’ data, frequent changes plus dynamic utilisation and a new generation of cloud based Event Engines is required.

Event Engines sit in a third layer over the Internet of Things, IoT, architecture that is becoming increasingly clearly defined; A base layer connectivity Infrastructure built on Fog Computing, or Edge based Clouds to localized the speed of ‘interactions’ between IoT Devices; Built over this is the store/search capabilities of Graph Databases with their unique capability to establishing relationships between data around connections in a manner similar to the manner that data is created by IoT Devices.

Event Engines are effectively the third layer of IoT architecture providing the higher levels of Business Value by managing responses to Event Trigger conditions, either as a complex event process from a single trigger event, or to draw a conclusion from a complex alignment of the flow of current data with stored data. The association of a Graph Database with IoT and Event Engines operating on Clouds has led to some Event Engine providers calling their associated (Graph)Database an ‘Event Cloud’.

Its not hard to imagine any number of examples that would be defined as simple event processes, classically the story of the towel dispenser running out of towels calling to be refilled. To some the definition of IoT remains that of a simple sensor on a machine reporting a certain value as changed, but to many it is now understood to be a wide range of data flowing from many different types of connected Devices. As IoT deployment scale increases both value and amount of data to be processed call for automation of ‘React’ outputs, and possibly integration with existing Enterprise Applications.

Complex Event Processing, CEP, is by definition complex to explain, but in simple terms is about finding new values from combinations of data and delivering an output that is Business Valuable. As is often the case with new Technology Event Engines and CEP is best understood by using examples of what can be done. The following two examples are edited versions of examples from the Wikipedia explanation of Complex Event Processing.

As an example of Complex Event Processing consider a car equipped with with just three simple IoT sensors that measure Tyre Pressure, Speed, and the presence of a Driver by a seat pressure. Individually each is able to offer a data flow and trigger condition. Combining the same data flows from the same three simple sensors using Complex Event Processing produces new data that is wholly different and of much higher value.

CEP Example 1: The speed sensor indicates car is moving when the tire pressure sensor data flow indicates the pressure in one tyre is dropping from 45 psi to 41 psi over 60 minutes. As the pressure in the tire is decreasing, a series of data events reporting the tyre pressure are being generated. In parallel a data flow is being generated indicating the car is being driven. (The presence of a driver and speed of movement). The car Event Processing Engine combines all three current and stored data flows to define the situation as gentle tyre deflation over a period of some time and outputs to the driver the display "loss Of Tire Pressure". This output may also be written into the structured database of the car maintenance log, and possibly in connected cars will even be sent out to seek the tyre puncture repair options.

CEP Example 2: Changing just one event reporting parameter in same situation produces an entirely different output and triggers appropriately different actions. If the Tyre pressure drops the same amount, but in 5 seconds, then the car Event Processing Engine will conclude the output to be “Tyre Punctured”, or “Blow Out”. This potentially catastrophic event will bring into play skid management control, hazard lights coming on, and possibly a warning being flashed externally to warn of a potential accident.

As with all forms of IoT deployment there are strong business management reasons to understand and define what outputs are required as much as Technology practitioners should know how to use IoT to deliver the requirement. The providers of Event Engines look to offer best practice drag and drop process design tools so, as with many Cloud based capabilities, the creation of Event Processes may move to becoming a business user activity.

The above two examples indicate the principles of both the two major forms of CEP, Aggregation and Detection, as well as the common combination of both into a Hybrid solution.

Aggregation Oriented CEP carry out processes by continuously calculating an ‘average value’ from multiple data flows to produce and trigger an output. Vibration increasing over a period taken in combination with speed in revs per minute, multiplied by hours run might indicate bearing wear; whereas rising, or constantly high temperature in the engine plus speed and hours running could be used to indicate when an oil change might be required.

Detection Oriented CEP seeks to find a required output trigger from a combination of event inputs in which a determined pattern or sequence can be found. Facebook’s search capability is an example using Detection Oriented CEP to looking for alignments and matches between apparently unrelated data held in Graph Databases.

Hybrid CEP is rapidly becoming a norm as the number of IoT sensors and Devices producing data flows increase Event Processing possibilities and informed users ask for a wider selection of output conditions.

It is tempting, and wrong, to draw parallels with event processing in Business Process Management, BPM, and therefore to consider BPM Rule Engines for IoT event processing. BPM Rule Engines are neither meant for continuous dynamic reprograming, nor can they combine IoT Device ‘push’ data with ‘pull’ data from APIs as two of the most obvious limitations.

Complex Event Processing, allied to Fog Computing and Graph Databases, makes for true game changing capabilities of the type that underpins many of the new high value Business disruptive capabilities. Simple Event trigger alarms may be enough to justify the first generation of IoT pilot deployment connected through small scale ‘Intranet’ deployments, but the real prize that a Global IoT environment brings delivers far higher levels of direct business value.

The Enterprises in each sector that are the first to learn how to connect, collate and process the new streams of Data in a massively connected IOT Device world will immediately gain the competitive advantage they seek. Its right to see that Data delivers the advantage, but to unlock that advantage requires new understanding of exactly what and how IoT works. The leading Enterprises are there already, think of Amazon, Facebook, or Google to understand how they collect and use data to monetize their new business models, and look around in your own Sector.