Dan Woods

Dan Woods, Contributor

I find technology that matters for early adopters.

8/25/2009 @ 6:00AM

Waking Up Multi-Core Processors

Why Apps Can’t Run Faster” explained that just putting more and more cores in chips doesn’t increase the processing speed of applications. This is because applications aren’t typically designed to take advantage of more than eight cores.

To be able to make the most of a multi-core processor, you can run lots of applications on one computer with lots of multi-core processors. Then the operating system balances the workload across all of the cores. But this is much less efficient than running applications that are multi-threaded that balance the work across cores inside the application. This is the art of parallel programming.

The problem is that parallel programming is extremely hard for a number of reasons. We generally don’t think about working in parallel algorithms because so many difficult issues arise. What happens when two parallel processes running on separate cores want to change the same piece of data? How does one of the parallel processes know when the other is finished so the process can continue? Furthermore, multi-threaded applications usually are written with a specific number of cores in mind at design time, and most of the time this cannot be changed without recompiling. And these are the easy problems. A host of other issues are deeply complex and do not yield to a simple solution. Parallel programming is rocket science, beyond the skill level of most developers.

Last week, in “Programming Innovation,” I pointed out how programming toolkits are becoming important levers of innovation because they convert rocket science like deep packet inspection and parallel programming into simpler forms that make the power of advanced techniques available to thousands of companies. This week we look at one toolkit that attacks the problem of parallel programming and shows both the power and limits of the process of simplification.

To succeed, toolkits must simplify a complex problem so that the really tricky parts are handled using a design that always works. A programming toolkit creates a set of Lego block-like components that can then be assembled to perform a programming task. In the case of parallel programming, this means that you must model a solution using the components. The toolkit translates the model into a program that is as parallel as the workflow allows, handling all of the tricky bits under the hood.

The DataRush platform from Pervasive Software attacks the problem of parallel programming by offering primitive building blocks for a data flow architecture. The Lego blocks are units of data transformation. The application development environment allows a programmer to use these building blocks to describe a specific program. Then, at run-time, the composition layer examines the hardware that the application is being run on and adjusts the program to take maximum advantage of the number of cores available. This avoids the problem of hard coding the number of cores.

Another advantage of a framework like the DataRush core system is the abstraction of parallel programming techniques. The platform allows Lego blocks to be added, hiding the details of lower level constructs such as threads or synchronization. This means programmers can avoid becoming entangled in parallelization details.

In order for the simplification of a programming toolkit to matter, it must deliver business results. Using the DataRush platform, Pervasive engineers were able to build an application for a client that processed health care claims 30 times faster than the existing system. This increase in performance was accomplished by parallelizing claims processing using standard parallelization techniques implemented in DataRush.

Another team built a fuzzy matching application for a business solutions provider in the Master Data Management (MDM) space. The solution was able to identify potential matches in an 18 million record dataset in under 20 minutes. The existing solution took over five hours. This substantial savings in run-time led to better quality matches by allowing the end user to experiment with different matching configurations. This type of design iteration is nearly impossible with run times measured in hours.

Pervasive’s DataRush and other programming toolkits don’t make all forms of parallel programming easy. They attack a specific subset that can be simplified if the program plays by certain rules and accepts certain limitations. Eventually programming toolkits will be developed for every different domain of complexity, serving different purposes like domain specific languages.

The speed of such calculations changes the game for what is possible in business processes. For Web site optimization, for example, instead of looking at Web traffic on a nightly basis, such techniques could allow analysis every hour or every 15 minutes. This dramatically changes the ability to respond.

Huge streams of data are flowing in from RFID, Twitter, sensor networks, e-commerce platforms, wireless devices and text messages. Extracting insights through data mining will be transformational if it can be done rapidly and cheaply. Multi-core processors provide the first step. Programming toolkits will complete the job. And waking up the power of multi-core processing to find meaning in these streams of data will provide a competitive advantage.


See Also:

Programming Innovation

Why Apps Can’t Run Faster

Closing The Gap With SaaS

Comments are turned off for this post.