which way up?

As companies master big data and the ability to capture big data’s insights, they progress to wanting those insights faster. 

Enter continuous apps. Businesses have turned to continuous apps to deliver real-time insights. 

Continuous apps are already providing value for companies, so let's look at some examples to help identify where you might apply them in your business. 

Where You'll Find Continuous Apps

Even though continuous apps are relatively nascent, here are a few use cases that can be adapted to many situations. 

Predictive Maintenance - Early Warning of Big Problems

A relatively simple example is predictive maintenance. A continuous application recognizes streaming data that’s actionable and sends an alert so that someone can take action before a system failure happens. Batch analysis can often recognize the triggers for a chain of events leading to a bad outcome or process failure. The continuous app recognizes these triggers in time to preempt these problems.

Real-time Event Data and Content Assembly - Breaking Down Silo Walls

Continuous applications can also be used to support larger processes involving multiple departments that previously worked in silos. When a key event happens, the continuous application assembles disparate data with the necessary correlations and then delivers it in real time to the appropriate people. The app gives everyone access to more data than they previously had while also ensuring that everyone is operating off of the same data.

Global Real-time Data Infrastructure – Creating a Data Fabric for Support of Mission Critical Apps

Ad tech companies also have dipped into the continuous apps world. Ad companies collect streaming data in a data lake, which a continuous app performs aggregations and provides operational reporting and insights on, as well as updating predictive models for automated response. 

Such a continuous app correlates multiple feeds around events and directs them into multiple Hadoop environments for business continuity. The continuous app performs important computations, like stripping out personally identifiable information for regulatory compliance and joining events with model scores so that events can be merged on the fly with contextual information. 

As a result, the companies have rich information for data science to improve operational outcomes and deep analytics for efficacy in improving models and making strategic decisions. 

The Principles of Building a Continuous App

All of these use cases deliver value — and that’s key. Nobody’s in the business of building continuous apps for their own sake. To identify opportunities:

  • Identify where your apps already create value: Consider whether there’s an opportunity for that value to arrive faster and, if so, what impact it would have on the business. If having key information faster would increase revenue, reduce costs or improve customer satisfaction, then you have an opportunity for a continuous app 
  • Once you’ve chosen your starting point, the next step is data management: This involves combining real-time data such as sensor data with master data and other relevant data sources in a data lake. The data lake may be quite complex, with hundreds of tables joined together. Reporting may be updated every hour or every day, enabling people to see real-time data in a broader context and gain insights 
  • From there, look for opportunities to reduce latency: Instead of delivering insights every day or every hour, deliver them every five minutes. Reducing latency often means increasing automation, so processes that require human review and intervention will typically have higher latency (minutes not seconds or less).

Common Continuous App Mistakes

As exciting as apps that leverage real time data can be, they aren’t the end-all, be-all of data processing. 

Just because you can deliver data to people in real time doesn’t mean you should. They simply may not want it, or they may not be able to do anything with it. 

One company wanted to show second-by-second how a short-lived promotion was behaving on the internet. While it offered an opportunity to create interesting visuals, the people receiving the information wouldn’t have had enough time do anything with it. 

You have to consider how people will respond to the information. Again, look for business drivers, and beware of refreshing data more often than is actually valuable just because you can. There is a point of diminishing returns.

Another mistake that comes up is the failure to consider whether streaming is worth the cost. Real-time streaming is not free. It’s easy to get excited about streaming engines because they’re the shiny new thing, but people fail to consider whether the use case is worth the extra complexity and resource consumption in the architecture. 

If you have a fairly simple real-time response system where events come in and you need a little bit of state around a given entity, there are other architectures that will give you the value you seek without the complexity of a streaming engine. These are still continuous apps — it’s just about using the right architecture to support them.

It takes some time to get a working understanding of continuous apps and the options for implementation. But once you do, the payoff can be enormous. Just one or two key apps that leverage real-time information can transform your business. 

Editor's Note: This is the final in a three-part series. You can read all of the series here.

Title image "Detour" (CC BY-SA 2.0) by  bionicteaching