Dan Woods

Dan Woods, Contributor

I find technology that matters for early adopters.

11/24/2009 @ 6:00AM

Multi-Core Slow Down

The widespread adoption of multi-core hardware is in many cases actually slowing down computing. A typical scenario: A company has a data-intensive software application and wants to make this software run as fast as possible. When its single-core machine reaches end of life, the company purchases multi-core hardware. To everyone’s surprise, the application starts to run slower.

For technical support staff, the first question to ask when someone calls with a performance problem is, Have you done a hardware upgrade? Often, when the application is switched back to single-core hardware, the problem disappears.

How can this be? Don’t more cores mean more computing power? Doesn’t more computing power mean software that runs faster? The simple answer is no. Unless the software was written specifically for the multi-core paradigm, multi-core hardware may be a waste of money for increasing performance.

The transition from a programming model based on one processor to one based on many processors involves a huge leap in complexity and skill. Never before has such a large technological transition been faced so casually. Forbes.com readers will be familiar with this topic through two articles. “Why Apps Can’t Run Faster” by Ed Sperling explained that many applications will not make use of the rapidly expanding number of cores. In “Waking Up Multi-Core Processors,” I explained how programming toolkits can make parallel programming easier for certain classes of applications. This week, we look at what happens when multi-core technology actually slows software down.

All previous transitions in computing entailed recompiling programs or other straightforward methods. When the address space of computers went from 16 to 32 to 64 bits, no heavy lifting was required. When processors, bus speeds or disks got faster, programs ran faster. When memory grew or special chip sets were added for graphics and input/output, it was the compiler or the operating system that had to change. Application developers could make tremendous strides forward by understanding the power of new mechanisms and writing their programs differently or by using more modern application programming interfaces, but even if they didn’t, much of the benefit would still be theirs.

But with the multi-core paradigm, the application itself must be aware that many processors are available. The application must then become the equivalent of a multi-tasking operating system and recognize that work can be performed in parallel and then parcel tasks out to many processors. Part of this programming model is keeping track of when each task is completed so that the next action can be taken. Another part is managing contention for resources shared among the processors. If one part of memory must be changed by many different processors, they must use queue mechanisms to get in line.

The mistakes you can make in single-core programming are numerous. In the multi-core model, you can to get into real trouble that is fiendishly hard to debug and optimize. Sometimes the work done to keep track of shared resources eats up the efficiencies. Other times, one process locks shared resource A, then wants shared resource B, which is locked by a process that has resource B and wants resource A. Its deadlock time. The number of programmers who can handle this sort of work is very small.

The sad truth for many applications is that the work performed really is not suited to parallel processing. You could have all the super geniuses in the world use Google’s new Go language or Microsoft’s PLINQ technology, both of which are primed for the multi-core world, but unless the application has many tasks that can be performed simultaneously, multi-core is not going to help.

There are several reasons that multi-core processors actually make many applications slower. First of all, the clock speed of each of the cores on a multi-core machine can be slower than those of the single core processors they are replacing. Second, some power of the multi-core processors is used to keep track of the sharing of resources among all the processors. For example, the more processors that are in play, the more work that must be done synchronizing caches used by each processor. Third, more processors may be contending for the same areas in memory. Fourth, the thundering herd problem occurs when lots of time is wasted figuring out which process is supposed to run next. These problems show up differently depending on the workload.

Keep in mind that I am not arguing that multi-core machines don’t do more work than single-core machines. If you have 10 separate programs running, a multi-core machine will be able to run many of these programs at once, keeping many cores going at the same time. This is clearly a benefit. Multi-core machines usually have newer memory that can be accessed faster and more of it as well. Other benefits of multi-core machines may include solid-state storage and faster network access and so on.

Even with all these advantages, multi-core machines are worse for particular applications and will be for many years–until software vendors and software developers become expert at parallel programming.

Of course, this represents a tremendous opportunity. Companies that use parallel programming properly will be able to make tremendous leaps forward over competitors. Here’s one example of what can be achieved. David Ehrlich, CEO of ParAccel, a company that makes a highly parallel database system, says that his technology is able to process large data sets 300 times faster in many cases.

Does this mean that you should avoid investing in multi-core hardware? Probably not. Will you even be able to avoid it? Probably not. What it does mean is that the race is on for those applications where speed matters and multi-core will help. The starting gun has fired for software vendors. For some, the race doesn’t matter. But for many, their applications may be left in the dust while competitors who have mastered parallel programming race ahead.


See Also:

Organic IT

How Far Will Google Wave Go?

The Myth Of Crowdsourcing

Comments are turned off for this post.