RC

-LRB-

-RRB-

May

Computer

Science/Mathematics

IBM

Research

Report

Graph

Partitioning

Based

Sparse

Matrix

Orderings

for

Interior-Point

Algorithms

Anshul

Gupta

IBM

Research

Division

T.J.

Watson

Research

Center

P.

O.

Box

Yorktown

Heights

New

York

anshul@watson.ibm.com

LIMITED

DISTRIBUTION

NOTICE

This

report

has

been

submitted

for

publication

outside

of

IBM

and

will

probably

be

copyrighted

if

accepted

for

publication

It

has

been

issued

as

a

Research

Report

for

early

dissemination

of

its

contents

In

view

of

the

transfer

of

copyright

to

the

outside

publisher

its

distribution

outside

of

IBM

prior

to

publication

should

be

limited

to

peer

communications

and

specific

requests

After

outside

publication

requests

should

be

filled

only

by

reprints

or

legally

obtained

copies

of

the

article

-LRB-

e.g.

payment

of

royalties

-RRB-

IBM

Research

Division

Almaden

Austin

China

Haifa

Tokyo

Watson

Zurich

Graph

Partitioning

Based

Sparse

Matrix

Orderings

for

Interior-Point

Algorithms

Anshul

Gupta

IBM

T.

J.

Watson

Research

Center

P.

O.

Box

Yorktown

Heights

NY

anshul@watson.ibm.com

IBM

Research

Report

RC

May

Abstract

The

key

computational

step

in

interior-point

-LRB-

IP

-RRB-

methods

for

solving

linear

programming

-LRB-

LP

-RRB-

problems

is

the

solution

of

a

sparse

symmetric

system

of

linear

equations

-LRB-

-RRB-

y

b

The

time

and

the

memory

requirements

of

this

step

depend

on

the

initial

ordering

of

rows

and

columns

in

the

symmetric

matrix

Computing

an

optimal

ordering

for

sparse

matrix

factorization

is

an

NP-complete

problem

and

developing

fast

and

effective

ordering

heuristics

has

been

a

subject

of

research

for

almost

three

decades

Currently

a

heuristic

known

as

multiple

minimum

degree

-LRB-

MMD

-RRB-

-LRB-

or

one

of

its

variants

-RRB-

is

almost

universally

employed

by

the

LP

community

while

using

IP

methods

In

this

paper

we

show

that

a

completely

different

approach

to

sparse

matrix

ordering

that

is

based

on

graph

partitioning

is

significantly

more

suitable

for

the

matrices

arising

in

an

IP

computation

than

MMD

or

its

variants

Experiments

with

our

ordering

algorithm

show

a

cumulative

speedup

by

a

factor

of

and

an

average

speedup

of

over

a

minimum-degree

based

ordering

for

solving

a

comprehensive

suite

of

real

optimization

problems

In

addition

our

graph-partitioning

based

ordering

algorithm

is

more

parallelizable

than

minimum-degree

based

orderings

algorithm

and

it

renders

the

ordered

matrix

more

amenable

to

parallel

factorization

Introduction

Over

the

past

decade

interior-point

-LRB-

IP

-RRB-

or

barrier

methods

-LSB-

-RSB-

have

become

increasingly

popular

for

solving

medium

to

large-size

linear

programming

-LRB-

LP

-RRB-

problems

IP

algorithms

are

iterative

in

nature

and

the

most

time-consuming

computational

step

in

each

iteration

is

solving

a

-LRB-

typically

-RRB-

sparse

symmetric

positive

definite

system

of

linear

equations

of

the

form

-LRB-

-RRB-

y

b.

Solving

this

system

using

direct

methods

involves

four

steps

-LSB-

-RSB-

namely

ordering

symbolic

factorization

numerical

factorization

and

triangular

solutions

Sparse

matrix

factorization

derives

its

efficiency

from

avoiding

arithmetic

operations

with

zeros

which

constitute

the

bulk

of

the

matrix

During

the

process

of

factorization

some

of

the

original

zero

entries

in

the

matrix

are

overwritten

by

a

nonzero

value

This

phenomenon

is

know

as

fill-in

The

time

and

memory

requirements

of

numerical

factorization

and

triangular

solutions

depend

on

the

amount

of

fill-in

which

is

a

function

of

the

initial

ordering

of

row

and

columns

of

the

sparse

symmetric

matrix

The

structure

of

the

coefficient

matrix

remains

the

same

in

each

iteration

of

an

IP

algorithm

Therefore

a

single

ordering

and

symbolic

factorization

suffices

for

numerical

factorization

in

each

iteration

For

a

majority

of

sparse

matrices

numerical

factorization

is

the

most

time-consuming

of

the

four

steps

However

for

some

LP

problems

the

ordering

time

using

a

conventional

ordering

method

can

be

comparable

to

or

exceed

the

total

factorization

time

of

all

iterations

Therefore

depending

on

the

problem

at

hand

the

overall

run

time

of

an

IP

algorithm

is

dominated

by

either

ordering

or

numerical

factorization

or

both

In

any

case

the

choice

of

a

good

ordering

technique

is

crucial

to

the

efficiency

of

the

entire

solution

process

In

this

paper

we

describe

ordering

heuristics

that

offer

significant

improvement

over

traditional

ordering

heuristics

for

LP

problems

both

in

terms

of

ordering

time

and

quality

In

addition

to

IP

methods

computing

a

fill-reducing

ordering

or

permutation

of

the

rows

and

columns

of

a

sparse

matrix

is

an

important

problem

in

any

application

that

uses

a

direct

method

for

solving

large

sparse

systems

of

linear

equations

Finding

an

optimal

ordering

is

an

NP-complete

problem

-LSB-

-RSB-

and

heuristics

must

be

used

to

obtain

an

acceptable

non-optimal

solution

Improving

the

run

time

and

quality

of

ordering

heuristics

has

been

a

subject

of

research

for

almost

three

decades

Two

main

classes

of

successful

heuristics

have

evolved

over

the

years

-LRB-

-RRB-

Minimum-Degree

-LRB-

MD

-RRB-

based

heuristics

and

-LRB-

-RRB-

Graph-Partitioning

-LRB-

GP

-RRB-

based

heuristics

MD

based

heuristics

are

local

greedy

heuristics

that

reorder

the

columns

of

a

symmetric

sparse

matrix

such

that

the

column

with

the

fewest

nonzeros

at

a

given

stage

of

factorization

is

the

next

one

to

be

eliminated

at

that

stage

GP

based

heuristics

regard

the

symmetric

sparse

matrix

as

the

adjacency

matrix

of

a

graph

and

follow

a

divide-and-conquer

strategy

to

label

the

nodes

of

the

graph

by

partitioning

it

into

smaller

subgraphs

The

initial

success

of

MD

based

heuristics

prompted

intense

research

-LSB-

-RSB-

to

improve

their

run

time

and

quality

and

they

have

been

the

methods

of

choice

among

practitioners

The

Multiple

Minimum

Degree

-LRB-

MMD

-RRB-

algorithm

by

George

and

Liu

-LSB-

-RSB-

and

the

Approximate

Minimum

Degree

-LRB-

AMD

-RRB-

algorithm

by

Davis

Amestoy

and

Duff

-LSB-

-RSB-

represent

the

state

of

the

art

in

MD

based

heuristics

Until

now

with

the

exception

of

Rothberg

and

Hendrickson

-LSB-

-RSB-

most

researchers

have

focused

on

ordering

sparse

matrices

arising

in

finite-element

applications

and

these

applications

have

guided

the

development

of

the

ordering

heuristics

Extensive

use

of

the

interior-point

methods

for

solving

LP

problems

is

relatively

recent

As

a

result

the

linear

programming

community

has

been

using

these

well

established

ordering

heuristics

that

were

not

originally

developed

for

their

applications

This

paper

shows

that

MD

based

heuristics

can

be

quite

unsuitable

for

many

linear

programming

problems

Recent

work

by

the

author

-LSB-

-RSB-

Hendrickson

and

Rothberg

-LSB-

-RSB-

Ashcraft

and

Liu

-LSB-

-RSB-

and

Karypis

and

Kumar

-LSB-

-RSB-

suggests

that

GP

based

heuristics

are

capable

of

producing

better

quality

orderings

than

MD

based

heuristics

for

finite-element

problems

while

staying

within

a

small

constant

factor

of

the

run

time

of

MD

based

heuristics

In

this

paper

we

present

a

suite

of

algorithms

that

enhance

the

GP

based

ordering

approach

to

generate

robust

orderings

of

sparse

matrices

arising

in

LP

problem

In

-LSB-

-RSB-

Rothberg

and

Hendrickson

independently

presented

evidence

of

the

superiority

of

GP

based

ordering

methods

for

LP

problems

using

quite

different

graph

partitioning

methods

We

show

that

with

our

modifications

the

benefits

of

GP

based

ordering

over

MD

based

ordering

are

even

more

pronounced

for

LP

problems

than

for

finite-element

problems

The

algorithms

presented

in

this

paper

are

included

in

the

graph

partitioning

and

sparse

matrix

ordering

package

-LRB-

WGPP

-RRB-

-LSB-

-RSB-

developed

by

the

author

In

-LSB-

-RSB-

we

present

experimental

results

of

using

WGPP

for

graph

partitioning

and

for

ordering

sparse

matrices

arising

in

different

applications

In

this

paper

we

show

that

WGPP

offers

significant

advantages

over

MD

based

heuristics

for

the

matrices

arising

in

IP

computations

For

such

matrices

WGGP

produces

better

quality

orderings

-LRB-

on

an

average

-RRB-

much

faster

than

AMD

or

MMD

-LRB-

unlike

finite

element

matrices

where

the

run

time

of

WGPP

is

higher

than

that

of

its

MD

based

counterparts

-RRB-

Section

shows

that

replacing

an

MD

based

ordering

by

the

WGPP

ordering

reduced

the

total

solution

time

of

a

suite

of

medium

to

large

LP

problems

by

a

factor

of

Both

MMD

and

AMD

algorithms

rely

heavily

on

the

concepts

of

mass

elimination

and

indistinguishable

nodes

-LSB-

-RSB-

in

order

to

achieve

their

run

time

efficiency

These

concepts

work

very

well

for

finite-element

graphs

but

may

not

be

very

effective

for

many

relatively

unstructured

LP

problems

We

have

observed

that

for

sparse

matrices

of

similar

size

-LRB-

in

terms

of

number

of

nonzero

entries

-RRB-

the

run

time

of

MMD

and

AMD

is

a

decreasing

function

of

the

average

supernode

size

in

the

factor

-LRB-

larger

supernodes

mean

more

successful

mass

elimination

-RRB-

A

supernode

is

a

set

of

consecutive

columns

in

a

triangular

factor

of

a

sparse

matrix

with

identical

off-diagonal

nonzero

structure

LP

matrices

typically

have

much

smaller

supernodes

than

finiteelement

matrices

The

run

time

of

WGPP

is

independent

of

the

size

of

the

supernodes

in

the

factor

and

depends

solely

on

the

number

of

nonzeros

in

the

original

sparse

matrix

As

a

result

WGPP

has

been

observed

to

be

faster

than

AMD

by

a

factor

of

several

hundred

on

some

of

our

test

problems

with

very

small

or

no

supernodes

-LRB-

Table

-RRB-

WGPP

achieves

this

improvement

in

ordering

time

without

compromising

the

quality

of

ordering

In

fact

our

experiments

show

that

WGPP

produces

orderings

that

require

less

work

in

factorization

on

an

average

on

a

large

suite

of

randomly

selected

LP

matrices

-LRB-

Tables

and

-RRB-

In

addition

to

generating

better

quality

orderings

faster

GP

based

methods

are

more

suitable

for

distributedmemory

parallel

computers

than

MD

based

methods

The

only

attempt

to

perform

a

minimum

degree

ordering

in

parallel

that

we

are

aware

of

-LSB-

-RSB-

was

not

successful

in

reducing

the

ordering

time

over

a

serial

implementation

On

the

other

hand

there

is

a

strong

theoretical

and

experimental

evidence

of

efficient

parallel

algorithms

for

graph

partitioning

and

sparse

matrix

orderings

based

on

graph

partitioning

-LSB-

-RSB-

GP

based

ordering

also

aids

the

parallelization

of

the

factorization

and

triangular

solution

phases

of

a

direct

solver

Gupta

Karypis

and

Kumar

-LSB-

-RSB-

have

proposed

a

highly

scalable

parallel

formulation

of

sparse

Cholesky

factorization

that

derives

a

significant

part

of

its

parallelism

from

the

underlying

partitioning

of

the

graph

of

the

sparse

matrix

In

-LSB-

-RSB-

Gupta

and

Kumar

present

efficient

parallel

algorithms

for

solving

lower

and

upper-triangular

systems

resulting

from

sparse

factorization

In

both

parallel

factorization

and

triangular

solutions

a

part

of

the

parallelism

would

be

lost

if

an

MD

based

heuristic

is

used

to

preorder

the

sparse

matrix

The

remainder

of

the

paper

contains

two

major

sections

In

Section

we

describe

the

overall

ordering

process

and

the

heuristics

used

in

its

various

stages

In

Section

we

present

experimental

results

comparing

the

WGPP

orderings

with

the

orderings

produced

by

the

state-of-the-art

MD

based

heuristics

The

Multiple

Multilevel

Recursive

Bisections

Heuristic

The

complete

ordering

algorithm

involves

several

stages

a

preprocessing

step

and

a

postprocessing

step

In

Sections

and

we

describe

the

core

algorithm

and

its

various

stages

Section

describes

the

preprocessing

step

that

has

a

significant

impact

on

the

quality

and

the

run

time

of

ordering

Section

describes

a

postprocessing

step

that

may

aid

certain

factorization

implementations

Figure

illustrates

the

overall

ordering

procedure

for

LP

matrices

by

means

of

a

flowchart

Graph

Bisection

Any

GP

based

ordering

algorithm

regards

the

matrix

as

the

adjacency

matrix

of

a

graph

and

assigns

labels

to

the

nodes

of

the

graph

These

labels

specify

the

sequence

in

which

the

matrix

columns

corresponding

to

the

package

is

available

to

the

users

in

the

form

of

a

linkable

module

nodes

are

eliminated

during

numerical

factorization

The

overall

approach

of

our

ordering

algorithm

follows

the

fundamental

technique

of

generalized

nested

dissection

-LSB-

-RSB-

The

graph

is

bisected

by

finding

and

removing

a

node-separator

the

nodes

of

the

two

resulting

disconnected

subgraphs

are

labeled

by

applying

the

same

technique

recursively

and

the

nodes

of

the

separator

are

labeled

after

the

nodes

of

the

subgraphs

have

been

labeled

-LRB-

i.e.

the

separator

nodes

get

higher

labels

than

any

of

the

nodes

of

the

two

subgraphs

-RRB-

The

recursion

terminates

when

the

subgraphs

become

too

small

at

which

stage

they

are

labeled

using

a

minimum

degree

heuristic

The

reason

why

this

technique

works

is

that

upon

elimination

a

column

corresponding

to

any

of

the

nodes

in

the

two

subgraphs

can

create

a

fill

only

in

the

columns

corresponding

to

the

separator

nodes

and

not

any

node

of

the

other

subgraph

Intuitively

smaller

node-separators

mean

less

fill

and

less

work

during

factorization

For

finite-element

graphs

with

certain

nice

properties

it

can

be

proved

that

this

technique

yields

orderings

within

a

constant

factor

of

the

optimal

-LSB-

-RSB-

Such

bounds

can

not

be

proved

for

the

unstructured

LP

matrices

However

as

we

show

in

Section

with

suitable

modifications

the

strategy

works

quite

well

in

practice

even

for

matrices

with

arbitrary

sparsity

patterns

In

addition

graph

bisection

yields

two

submatrices

of

the

original

matrix

that

can

be

factored

independently

in

parallel

This

is

the

basis

of

many

efficient

parallel

algorithms

for

sparse

matrix

factorization

-LSB-

-RSB-

A

key

step

in

our

ordering

algorithm

is

finding

a

small

node

bisector

of

a

graph

Recent

research

-LSB-

-RSB-

has

shown

multilevel

algorithms

to

be

fast

and

effective

in

computing

graph

partitions

A

typical

multilevel

graph

partitioning

algorithm

has

four

components

namely

coarsening

initial

partitioning

uncoarsening

and

refining

In

-LSB-

-RSB-

we

describe

in

detail

the

general

heuristic

to

partition

a

graph

into

k

parts

In

the

following

subsections

we

briefly

discuss

the

components

of

graph

partitioning

in

the

context

of

bisection

-LRB-

partioning

into

two

parts

-RRB-

with

a

node-separator

Coarsening

The

goal

of

the

coarsening

phase

is

to

reduce

the

size

of

a

graph

while

preserving

its

properties

that

are

essential

to

finding

good

a

partition

Using

the

appropriate

coarsening

algorithm

is

very

important

for

robust

and

good

quality

graph

partitioning

For

coarsening

the

original

graph

is

regarded

as

a

weighted

graph

with

a

unit

weight

assigned

to

each

edge

and

each

node

In

a

coarsening

step

a

maximal

set

of

edges

of

the

graph

is

identified

such

that

no

two

edges

in

this

set

have

a

node

in

common

This

set

of

edges

is

known

as

a

matching

The

edges

in

this

set

are

then

removed

The

two

nodes

that

an

edge

in

the

matching

connects

are

collapsed

into

a

single

node

This

composite

node

is

assigned

a

weight

that

is

the

sum

of

the

weights

of

the

two

component

nodes

A

coarsening

step

usually

reduces

the

number

of

nodes

in

the

graph

by

nearly

half

Figure

illustrates

the

coarsening

steps

Note

that

coarsening

also

results

in

some

edges

being

collapsed

into

one

in

which

case

the

collapsed

edge

is

assigned

a

weight

equal

to

the

sum

of

weights

of

the

component

edges

Given

a

weighted

graph

after

any

stage

of

coarsening

there

are

several

choices

of

matchings

for

the

next

coarsening

step

A

simple

matching

scheme

-LSB-

-RSB-

known

as

random

matching

-LRB-

RM

-RRB-

randomly

chooses

pairs

of

connected

unmatched

nodes

to

include

in

the

matching

In

-LSB-

-RSB-

Karypis

and

Kumar

describe

a

heuristic

known

as

heavy-edge

matching

-LRB-

HEM

-RRB-

HEM

not

only

helps

reduce

the

run

time

of

the

refinement

component

of

graph

partitioning

but

also

tends

to

generate

partitions

with

small

separators

The

HEM

strategy

randomly

picks

an

unmatched

node

selects

the

edge

with

the

highest

weight

among

the

edges

incident

on

this

node

that

connect

it

to

other

unmatched

nodes

and

marks

both

nodes

connected

by

this

edge

as

matched

Note

that

the

weight

of

an

edge

connecting

two

nodes

in

a

coarsened

version

of

the

graph

is

the

number

of

edges

in

the

original

graph

that

connect

the

two

sets

of

original

nodes

collapsed

into

the

two

coarse

nodes

HEM

by

absorbing

the

heavier

edges

generates

coarse

graphs

whose

nodes

are

loosely

connected

-LRB-

by

the

lighter

remaining

edges

-RRB-

This

ensures

that

a

partition

of

the

coarse

graph

corresponds

to

a

good

partition

of

the

original

graph

HEM

can

miss

some

heavy

edges

in

the

graph

because

the

nodes

are

visited

in

random

order

for

matching

For

example

consider

a

node

i

the

heaviest

edge

incident

on

which

connects

it

to

a

node

j

If

i

is

visited

before

j

and

both

i

and

j

are

unmatched

the

edge

-LRB-

i

j

-RRB-

will

be

included

in

the

matching

If

there

exists

-LRB-

c

-RRB-

A

matching

of

the

coarse

graph

-LRB-

a

-RRB-

A

matching

of

the

original

graph

-LRB-

d

-RRB-

After

steps

of

coarsening

-LRB-

b

-RRB-

Graph

after

coarsening

Figure

Illustration

of

graph

coarsening

an

edge

-LRB-

j

k

-RRB-

such

that

i

and

k

are

not

connected

it

will

be

excluded

from

the

matching

even

if

it

is

much

heavier

than

-LRB-

i

j

-RRB-

because

j

is

no

longer

available

for

matching

HEM

was

originally

devised

for

finite-element

graphs

for

which

it

works

reasonably

well

because

the

range

of

edge

weights

is

usually

small

However

in

the

graphs

of

LP

matrices

there

is

a

large

variation

among

the

edge

weights

-LRB-

especially

in

the

later

stages

of

coarsening

-RRB-

and

heavy

edges

left

out

from

the

matching

adversely

affect

the

quality

of

coarsening

To

overcome

this

problem

after

the

first

few

coarsening

steps

we

switch

to

what

we

call

the

heaviest-edge

matching

We

sort

the

edges

by

their

weights

and

visit

them

in

the

decreasing

order

of

their

weights

to

inspect

them

for

their

possible

inclusion

in

the

matching

Ties

are

broken

randomly

The

benefit

of

heaviest-edge

matching

over

Karypis

and

Kumar

's

HEM

are

more

pronounced

in

the

later

stages

of

coarsening

and

that

is

where

sorting

is

not

too

expensive

because

the

graph

has

shrunk

considerably

from

its

original

size

Even

the

modified

HEM

may

occasionally

fail

to

satisfactorily

coarsen

the

graph

of

an

LP

matrix

An

example

of

such

a

graph

is

one

that

has

a

star

like

structure

-LRB-

Figure

-RRB-

Since

RM

and

HEM

disregard

the

possibility

of

matching

disconnected

nodes

they

fail

to

preserve

the

properties

of

such

a

graph

upon

coarsening

and

may

result

in

a

very

unbalanced

coarse

graph

Moreover

RM

and

HEM

reduce

the

size

of

a

star

shaped

graph

by

only

one

node

in

one

coarsening

step

In

order

to

overcome

these

problems

conceptually

WGPP

regards

every

graph

as

a

completely

connected

graph

with

a

weight

zero

assigned

to

every

edge

-LRB-

i

j

-RRB-

that

does

not

really

exist

in

the

physical

graph

It

then

attempts

to

maximize

a

modified

weight

function

of

any

two

unmatched

nodes

i

and

j

when

considering

them

to

include

in

the

matching

as

a

pair

The

modified

edge

weight

-LRB-

i

j

-RRB-

between

two

nodes

i

and

j

with

node

weights

nw

-LRB-

i

-RRB-

and

nw

-LRB-

j

-RRB-

respectively

and

connected

by

an

edge

of

-LRB-

possibly

zero

-RRB-

weight

ew

-LRB-

i

j

-RRB-

is

given

by

-LRB-

i

j

-RRB-

ew

-LRB-

i

j

-RRB-

nw

-LRB-

i

-RRB-

nw

-LRB-

j

-RRB-

Here

and

are

constants

defined

at

the

beginning

of

the

given

level

of

coarsening

WGPP

uses

-LRB-

i

j

-RRB-

-LRB-

coarsening

steps

-RRB-

WGPP

matching

RM/HEM

-LRB-

coarsening

steps

-RRB-

Figure

Difference

between

the

coarse

graph

produced

by

RM

or

HEM

and

that

generated

by

the

WGPP

matching

algorithms

when

the

original

graph

has

a

star

structure

instead

of

the

real

edge

weight

ew

-LRB-

i

j

-RRB-

within

the

HEM

framework

It

is

clear

from

the

above

equation

that

the

modified

edge

weight

criterion

yields

a

balanced

coarsening

in

fewer

steps

while

preserving

the

benefits

of

HEM

To

save

coarsening

time

in

the

actual

implementation

WGPP

switches

to

using

the

modified

edge

weights

only

if

heavy

and

heaviest-edge

matchings

fail

to

reduce

the

size

of

the

graph

sufficiently

Initial

partitioning

In

-LSB-

-RSB-

we

show

that

the

initial

partitioning

phase

can

be

completely

eliminated

thereby

simplifying

and

speeding

up

the

overall

partitioning

algorithm

All

multilevel

graph

partitioning

schemes

described

in

the

literature

stop

the

coarsening

phase

when

the

graph

has

been

reduced

to

a

few

hundred

to

a

few

thousand

nodes

and

use

some

heuristic

to

compute

an

initial

partition

of

the

coarse

graph

at

a

substantial

run

time

cost

One

of

the

effective

heuristics

for

initial

partitioning

is

graph

growing

-LSB-

-RSB-

In

order

to

bisect

a

graph

the

graph

growing

heuristic

would

pick

up

a

node

at

random

tag

it

and

keep

tagging

its

neighbors

and

neighbors

neighbors

in

a

breadth-first

manner

until

the

cumulative

weight

of

the

tagged

nodes

is

roughly

equal

to

the

cumulative

weight

of

the

untagged

nodes

The

set

of

tagged

and

untagged

nodes

now

form

the

two

partitions

of

the

graph

In

-LSB-

-RSB-

we

discuss

that

graph

growing

is

the

bottom-up

equivalent

of

coarsening

We

continue

to

coarsen

the

graph

until

it

contains

only

two

nodes

which

constitute

a

natural

initial

partition

Uncoarsening

and

refinement

The

uncoarsening

and

refining

components

of

graph

partitioning

work

together

The

uncoarsening

and

refining

phase

of

WGPP

is

quite

different

from

most

other

multilevel

graph

partitioning

algorithms

Initially

the

two

nodes

of

the

coarsest

graph

are

assigned

different

tags

indicating

that

they

belong

to

different

initial

partitions

They

are

then

split

into

the

nodes

that

were

collapsed

to

form

them

during

the

coarsening

phase

This

reversal

of

coarsening

is

carried

out

one

step

at

a

time

The

nodes

of

the

uncoarsened

graph

inherit

their

tags

from

their

parent

nodes

in

the

coarser

graph

At

any

stage

of

uncoarsening

the

edges

connecting

a

pair

of

nodes

belonging

to

different

partitions

constitute

an

edge-separator

of

the

graph

A

subset

of

the

nodes

which

the

separator

edges

are

incident

on

is

a

node-separator

if

-LRB-

and

only

if

-RRB-

this

subset

covers

all

the

edges

of

the

separator

The

removal

of

an

edge

or

a

node-separator

from

the

graph

breaks

it

into

two

disconnected

subgraphs

After

each

step

of

uncoarsening

the

separator

is

refined

While

refining

an

edge-separator

an

attempt

is

made

to

l

small

node

separators

SubgraphSubgraph

smallest

edge

separator

a

b

c

d

e

f

g

h

i

j

k

Figure

Drawbacks

of

choosing

a

node-separator

from

the

minimum

edge-separator

minimize

its

total

weight

by

switching

the

partitions

of

some

nodes

if

this

switching

reduces

the

separator

size

and

does

not

make

the

weights

of

the

two

parts

too

imbalanced

While

refining

a

node-separator

nodes

are

moved

between

the

separator

and

the

two

subgraphs

partitioned

by

it

in

an

attempt

to

minimize

the

total

weight

of

the

separator

nodes

The

ultimate

goal

of

the

graph

bisection

phase

of

ordering

is

to

find

a

small

node-separator

Current

GP

based

ordering

algorithms

follow

two

different

approaches

to

finding

a

small

node-separator

A

conventional

approach

-LSB-

-RSB-

refines

the

edge-separator

between

the

two

subgraphs

after

each

step

of

uncoarsening

so

that

few

edges

connect

nodes

of

different

subgraphs

in

the

final

partitioning

of

the

original

graph

Then

an

algorithm

for

finding

a

minimum

cover

-LSB-

-RSB-

is

used

to

compute

a

node-separator

from

the

edge-separator

This

approach

relies

heavily

on

the

assumption

that

the

size

of

a

node-separator

is

proportional

to

the

size

of

the

edge-separator

containing

it

This

assumption

is

not

too

unreasonable

for

the

relatively

structured

finite-element

meshes

but

usually

does

not

work

well

for

LP

matrices

For

example

consider

the

graph

in

Figure

Minimizing

the

edge-separator

will

partition

the

graph

between

nodes

c

d

e

f

and

nodes

g

h

i

j

The

smallest

possible

node-separator

that

can

be

extracted

from

the

edge-separator

-LRB-

c

g

-RRB-

-LRB-

d

h

-RRB-

-LRB-

e

i

-RRB-

-LRB-

f

j

-RRB-

contains

at

least

four

nodes

On

the

other

hand

node-separators

of

size

two

-LRB-

i.e.

a

b

or

k

l

-RRB-

are

available

with

almost

the

same

degree

of

imbalance

between

the

two

partitions

Another

approach

-LSB-

-RSB-

is

to

find

a

node-separator

within

the

coarse

graph

and

refine

it

into

a

nodeseparator

of

the

original

graph

in

order

to

overcome

the

drawbacks

of

working

with

an

edge-separator

However

even

this

approach

is

also

not

completely

free

of

pitfalls

In

Figure

we

show

that

the

heavier

node-separator

in

the

coarse

graph

may

result

in

the

smaller

node-separator

in

the

original

graph

after

uncoarsening

-LRB-

and

vice

versa

-RRB-

The

weight

of

a

node

in

the

coarse

graph

is

the

number

of

nodes

of

the

original

graph

that

are

collapsed

during

the

coarsening

steps

to

form

the

coarse

node

Not

all

the

component

nodes

of

a

coarse

separator

node

may

have

edges

crossing

partition

boundaries

As

the

graph

is

uncoarsened

such

nodes

are

eliminated

from

the

node-separator

by

the

refining

process

How

many

such

nodes

are

eliminated

from

a

node-separator

depends

on

the

connectivity

of

these

nodes

to

the

nodes

outside

the

separator

For

example

in

Figure

the

smaller

coarse

node-separator

consisting

of

nodes

b

and

c

has

a

total

weight

of

This

coarse

separator

results

in

a

final

node-separator

of

size

This

is

because

all

six

nodes

have

inter-partition

connections

and

of

them

can

be

removed

from

the

node-separator

by

refining

On

the

other

hand

the

coarse

separator

of

weight

consisting

of

nodes

e

and

g

results

in

a

final

separator

containing

only

nodes

This

is

because

two

of

the

four

component

nodes

of

both

coarse

nodes

c

and

g

have

no

inter-partition

edges

and

are

eliminated

from

the

separator

by

refinement

In

this

example

choosing

the

smaller

of

the

two

edge-separators

in

the

coarse

separator

a

b

c

d

e

f

g

h

large

edge

large

node

small

node

separator

separator

small

edge

separator

Figure

Drawbacks

of

choosing

a

small

node-separator

for

refining

in

a

coarse

graph

graph

yields

the

smaller

node-separator

in

the

original

graph

Unlike

in

the

original

graph

the

size

of

the

edge-separator

is

not

completely

irrelevant

in

coarse

versions

of

the

graph

With

these

motivations

we

now

present

our

uncoarsening

and

refining

scheme

which

is

illustrated

in

Figure

We

first

perform

as

many

steps

of

coarsening

on

the

original

n-node

graph

as

required

to

coarsen

it

down

to

two

nodes

We

then

uncoarsen

the

graph

to

q

nodes

where

q

n

while

refining

the

edgeseparator

after

each

step

of

uncoarsening

At

this

stage

we

save

the

partition

coarsen

the

q-node

graph

back

to

two

nodes

and

repeat

the

uncoarsening

and

refining

process

This

process

is

repeated

a

few

times

and

of

all

the

partitions

of

the

q-node

graph

generated

we

chose

the

best

for

further

uncoarsening

to

obtain

a

partition

of

the

original

n-node

graph

The

best

partition

is

selected

on

the

basis

of

a

weighted

average

of

the

size

of

edge-separator

the

size

of

the

node-separator

and

the

inverse

of

the

imbalance

between

the

two

parts

Unlike

finite-element

graphs

the

graphs

from

the

unstructured

LP

problems

usually

show

large

variations

in

the

quality

of

different

partitions

of

the

same

graph

The

randomization

in

the

coarsening

process

ensures

that

the

different

trials

result

in

sufficiently

different

partitions

Selecting

the

best

amongst

many

partitions

goes

a

long

way

in

finding

a

good

separator

for

the

original

graph

Most

of

the

partitioning

cost

is

incurred

in

the

first

few

coarsening

steps

of

the

original

graph

and

the

last

few

uncoarsening

and

refining

steps

when

the

size

of

the

graph

is

large

Therefore

multiple

cycles

of

coarsening

and

uncoarsening

on

the

smaller

coarser

q-node

graph

have

little

impact

on

the

overall

run

time

of

the

entire

algorithm

Finally

the

selected

partition

of

the

q-node

graph

is

uncoarsened

to

the

original

n-node

graph

one

step

at

a

time

After

each

uncoarsening

step

we

refine

the

node-separator

i.e.

in

the

final

steps

of

uncoarsening

the

subject

of

refinement

is

changed

from

the

edge-separator

to

a

node-separator

For

refining

edge-separators

we

use

the

popular

Fiduccia-Mattheyses

-LSB-

-RSB-

heuristic

which

is

a

linear

time

version

of

the

Kernighan-Lin

-LSB-

-RSB-

heuristic

In

-LSB-

-RSB-

Ashcraft

and

Liu

extended

the

Fiduccia-Mattheyses

framework

to

refine

node-separators

as

well

We

use

a

similar

approach

when

node-separators

are

refined

in

the

final

steps

of

uncoarsening

Recursive

Bisection

and

Ordering

Once

a

node-separator

of

the

original

graph

is

found

it

is

removed

and

the

entire

procedure

is

repeated

recursively

on

the

two

disconnected

subgraphs

The

recursion

terminates

if

a

subgraph

is

too

small

in

which

q

nodes

nodes

q

n

nodes

n

fs

Finest

Level

Coarsest

Level

fs

min

-LRB-

ft

-RRB-

ft

Refine

edge-separator

Refine

node-separator

Figure

An

overview

of

our

uncoarsening

and

refining

scheme

for

finding

a

node-separator

from

the

best

amongst

multiple

coarse

bisections

fi

is

a

weighted

average

of

the

node-separator

size

edge-separator

size

and

the

inverse

of

imbalance

between

the

two

parts

in

the

i-th

bisection

case

it

is

not

partitioned

any

further

but

ordered

using

a

minimum

degree

heuristic

In

our

implementation

the

size

of

the

terminal

subgraphs

ranges

from

a

few

dozen

nodes

to

a

few

hundred

nodes

depending

on

the

size

of

the

original

graph

We

use

the

AMD

variation

-LRB-

due

to

Davis

Amestoy

and

Duff

-LSB-

-RSB-

-RRB-

of

the

minimum

degree

heuristic

for

ordering

these

small

subgraphs

After

the

nodes

of

the

two

subgraphs

at

any

level

of

recursion

have

been

labeled

the

nodes

of

the

separator

at

that

level

are

labeled

and

the

algorithm

returns

to

the

previous

level

if

any

This

procedure

ensures

that

the

labels

assigned

to

the

nodes

of

any

separator

are

higher

than

any

label

in

the

two

disconnected

subgraphs

separated

by

the

separator

Therefore

during

numerical

factorization

the

columns

corresponding

to

the

separator

nodes

are

eliminated

after

all

columns

corresponding

to

the

nodes

of

the

two

subgraphs

have

been

eliminated

Preprocessing

Sections

and

describe

the

process

of

labeling

the

nodes

of

the

graph

corresponding

to

a

sparse

matrix

In

this

section

we

describe

a

simple

preprocessing

step

that

if

performed

before

subjecting

the

graph

to

the

labeling

process

described

previously

improves

both

the

run

time

and

the

quality

of

the

ordering

heuristic

Most

matrices

of

the

form

that

need

to

be

factored

while

solving

an

LP

problem

using

a

barrier

method

have

a

few

-LRB-

typically

about

-RRB-

the

columns

that

are

much

denser

than

the

remaining

columns

The

number

of

nonzeros

in

these

few

columns

is

typically

several

times

more

than

the

average

number

of

nonzeros

per

column

of

the

sparse

matrix

The

preprocessing

step

illustrated

in

Figure

simply

removes

the

nodes

with

excessively

high

degrees

from

the

graph

The

remainder

of

the

graph

is

labeled

using

the

heuristics

described

in

Sections

and

and

then

the

previously

removed

high-degree

nodes

are

assigned

the

highest

labels

Typically

the

removal

of

these

nodes

either

breaks

the

graph

up

into

several

disconnected

components

or

renders

it

much

loosely

connected

than

the

original

graph

This

aids

the

partitioning

algorithm

in

quickly

finding

node-separators

of

a

small

-LRB-

or

zero

-RRB-

size

Postprocessing

As

the

factorization

of

a

sparse

symmetric

matrix

proceeds

from

top

left

to

bottom

right

the

columns

that

are

eliminated

create

nonzero

entries

or

fill-in

in

the

remainder

of

the

matrix

Towards

the

end

of

the

factorization

process

the

lower

right

corner

of

the

matrix

often

becomes

fairly

dense

Many

implementations

of

sparse

removal

of

the

three

nodes

high-degree

Original

Graph

high-degree

nodes

Graph

after

the

Figure

Illustration

of

the

preprocessing

step

performed

in

WGPP

before

ordering

an

LP

matrix

factorization

switch

to

dense

matrix

factorization

at

this

stage

because

the

cost

of

additional

arithmetic

operations

is

more

than

compensated

for

by

the

advantages

of

dense

factorization

such

as

absence

of

indirect

addressing

and

better

utilization

of

memory

hierarchy

With

a

GP

based

ordering

the

matrix

columns

corresponding

to

the

nodes

of

a

separator

usually

tend

to

become

denser

during

factorization

than

the

columns

corresponding

to

the

nodes

of

the

subgraphs

that

the

separator

separates

This

is

because

the

separator

columns

receive

fill-in

from

the

columns

of

both

the

subgraphs

that

they

separate

On

the

other

hand

the

columns

of

each

subgraph

receive

fill-in

from

the

nodes

of

only

that

subgraph

In

addition

typically

separators

have

fewer

nodes

than

the

separated

subgraphs

A

large

number

of

columns

contributing

fill-in

to

a

small

number

of

separator

columns

results

in

the

separator

columns

becoming

relatively

dense

during

factorization

An

implementation

of

sparse

matrix

factorization

that

switches

to

dense

factorization

would

benefit

from

a

larger

and

denser

bottom

right

leftover

portion

of

the

matrix

during

factorization

In

order

to

aid

such

implementations

we

collect

all

separators

larger

than

a

certain

size

and

relabel

their

nodes

so

that

they

receive

the

highest

labels

In

other

words

we

regard

the

top

m

levels

of

recursive

bisection

as

a

single

partition

into

parts

and

all

the

bisectors

-LRB-

a

separator

that

bisects

-RRB-

as

a

single

composite

separator

whose

removal

results

in

disconnected

subgraphs

The

nodes

of

these

subgraphs

are

labeled

before

the

nodes

of

the

composite

separator

While

labeling

the

nodes

of

the

composite

separator

we

make

sure

that

the

nodes

are

labeled

according

to

the

depth

of

the

level

of

recursion

in

which

they

were

included

in

the

separator

For

example

the

nodes

of

the

first

bisector

of

the

original

graph

receives

higher

labels

than

the

nodes

of

the

two

bisectors

of

the

two

resulting

subgraphs

This

relabeling

of

separator

nodes

does

not

effect

the

total

amount

of

fill-in

during

factorization

or

its

operation

count

It

only

effects

the

location

of

fill-in

and

attempts

to

direct

most

of

the

fill-in

towards

the

high-index

columns

of

the

matrix

Experimental

Results

In

this

section

we

present

detailed

experimental

comparison

of

the

WGPP

ordering

based

on

multiple

multilevel

recursive

bisection

with

the

best

minimum-degree

based

ordering

algorithms

currently

in

use

for

solving

LP

problems

using

barrier

methods

The

other

two

graph-partitioning

based

ordering

softwares

that

we

are

Enough

partitions

tried

no

First

partition

yes

yes

no

Coarsen

graph

to

two

nodes

Start

with

the

original

graph

Preprocess

high

degree

nodes

yes

splitting

the

graph

into

two

or

more

subgraphs

For

each

subgraph

do

the

following

no

yes

Is

subgraph

small

enough

yes

End

All

subgraphs

ordered

at

all

levels

saved

partition

Better

than

the

no

Save

current

partition

Uncoarsen

the

saved

partition

while

refining

Partially

uncoarse

while

refining

the

edge-separator

the

node-separator

Remove

the

node-separator

no

Order

subgraph

using

minimum

degree

Figure

An

overview

of

the

ordering

algorithm

for

LP

matrices

in

WGPP

aware

of

are

Metis

-LSB-

-RSB-

and

DDSEP

-LSB-

-RSB-

Both

of

these

were

designed

primarily

for

finite-element

problems

Of

these

only

Metis

is

available

at

the

time

of

writing

this

paper

and

it

does

not

work

for

a

majority

of

LP

problems

Comparison

with

MMD

and

AMD

In

Table

we

compare

three

codes

namely

MMD

by

George

and

Liu

-LSB-

-RSB-

AMD

by

Davis

Amestoy

and

Duff

-LSB-

-RSB-

and

WGPP

-LSB-

-RSB-

described

in

this

paper

on

a

dozen

large

LP

problems

The

table

reports

the

run

times

of

the

three

ordering

codes

on

an

IBM

workstation

All

three

were

compiled

with

the

highest

level

of

optimization

-LRB-

option

-RRB-

of

XLF

Fortran

compiler

The

table

also

reports

the

quality

of

ordering

in

terms

of

the

operation

count

in

Cholesky

factorization

and

the

average

supernode

size

From

Table

it

can

be

observed

that

on

an

average

MMD

produces

slightly

better

ordering

than

AMD

but

takes

significantly

more

time

The

ratios

of

the

AMD

operation

count

to

WGPP

operation

count

in

the

last

row

of

the

table

show

that

WGPP

produces

much

better

ordering

than

its

MD

based

counterparts

for

these

problems

The

run

time

of

WGPP

relative

to

AMD

depends

on

the

average

supernode

size

For

matrices

with

larger

supernodes

-LRB-

e.g.

GRADE

GSMND

PILOT

-RRB-

AMD

is

more

than

twice

as

fast

as

WGPP

However

for

matrices

with

very

small

or

no

supernodes

-LRB-

e.g.

-RRB-

WGPP

can

be

several

hundred

times

faster

than

AMD

Although

it

so

happens

that

WGPP

ordering

reduces

the

factorization

operation

count

over

AMD

for

all

the

matrices

in

Table

more

extensive

experiments

-LRB-

Table

-RRB-

show

that

there

are

some

case

in

which

WGPP

results

in

a

higher

fill-in

and

operation

count

that

an

MD

based

ordering

The

dependence

of

the

run

time

of

MD

based

heuristics

on

the

average

supernode

size

can

be

exploited

to

improve

the

ordering

in

many

such

cases

From

Table

one

can

observe

that

the

average

supernode

size

is

a

property

of

the

matrix

and

it

shows

only

marginal

variations

for

different

orderings

of

the

same

matrix

Therefore

after

performing

the

GP

based

ordering

and

computing

the

resulting

average

supernode

size

it

is

possible

to

predict

whether

or

not

an

MD

based

heuristic

will

be

fast

for

that

matrix

If

the

average

supernode

size

is

large

then

an

MD

based

ordering

can

be

attempted

and

the

better

of

the

two

orderings

can

be

used

for

factorization

This

would

result

in

only

a

slight

increase

in

the

ordering

time

because

the

second

ordering

-LRB-

minimum

degree

-RRB-

is

attempted

only

when

its

run

time

is

known

to

be

small

WGPP

has

a

parameter

by

which

the

user

can

switch

the

option

of

attempting

the

second

ordering

on

or

off

When

this

option

is

on

WGPP

attempts

the

second

ordering

if

the

average

supernode

size

is

large

enough

Thus

for

the

class

of

problems

for

which

minimum

degree

is

both

fast

and

better

than

the

graph

partitioning

based

ordering

WGPP

returns

the

permutation

corresponding

to

the

former

Impact

on

IBM

Optimization

Subroutine

Library

The

IBM

Optimization

Subroutine

Library

-LRB-

OSL

-RRB-

-LSB-

-RSB-

is

a

collection

of

high-performance

mathematical

subroutines

for

use

by

application

programs

that

solve

optimization

problems

In

addition

to

the

traditional

Simplex

method

-LSB-

-RSB-

OSL

includes

subroutines

that

implement

all

three

popular

algorithmic

approaches

for

the

interior-point

-LRB-

barrier

-RRB-

methods

namely

the

primal

barrier

method

-LSB-

-RSB-

the

primal-dual

method

-LSB-

-RSB-

and

the

predictor-corrector

method

-LSB-

-RSB-

WGPP

contains

subroutines

that

replace

OSL

's

internal

minimumdegree

based

ordering

subroutines

-LRB-

see

-LSB-

-RSB-

for

details

-RRB-

if

an

application

program

is

linked

with

both

WGPP

and

OSL

libraries

In

Table

we

compare

OSL

's

run

times

for

solving

a

large

randomly

chosen

suite

of

LP

problems

using

IP

methods

with

and

without

linking

WGPP

The

problems

range

in

size

from

a

a

few

seconds

to

several

hours

of

execution

time

The

experiments

shown

in

this

table

were

conducted

on

an

IBM

workstation

To

facilitate

a

comparison

between

MD

and

GP

based

orderings

the

results

in

Table

were

generated

with

the

option

of

attempting

the

second

ordering

turned

off

The

problems

that

have

a

before

their

serial

numbers

OSL

users

can

use

WGPP

in

this

manner

and

may

get

enhancement

performance

from

OSL

for

interior-point

methods

Matrix

GRADE

GSMND

PILOT

USAIR

Dimension

-LRB-

N

-RRB-

Nonzeros

-LRB-

NNZA

-RRB-

Density

-LRB-

NNZA

N

-RRB-

M

Runtime

-LRB-

sec

-RRB-

M

Avg

supnode

size

D

Opcount

-LRB-

-RRB-

1438 2676 6294 30377

2153 186 270 1872

A

Runtime

-LRB-

sec

-RRB-

M

Avg

supnode

size

D

Opcount

-LRB-

-RRB-

1334 2753 4926 30493

2460 204 318 2054

W

Runtime

-LRB-

sec

-RRB-

G

Avg

supnode

size

P

Opcount

-LRB-

-RRB-

1226 2181 3307 12843

492 204 286 1881

P

OpcountAMD

OpcountWGPP

Table

Ordering

time

and

quality

comparisons

between

MMD

AMD

and

WGPP

Matrix

OSL

Ordering

WGPP

Ordering

NNZOSL

nfOSL

TOSL

NNZWGPP

nfWGPP

TWGPP

TOSL

TWGPP

-LRB-

-RRB-

-LRB-

seconds

-RRB-

-LRB-

-RRB-

-LRB-

seconds

-RRB-

AAMAT

ALLGRADE

CAPLOAD

CRED

DBTMIN

EUROPCAR

GISMONDI

GREENBEA

GREENBEB

MICHELIN

MPS

SLOW

PEUGEOT

PILOTS

TMPROB

UNILEVER

UNOCAL

USAIRORIG

WORK

Table

Experimental

comparison

between

the

minimum-degree

based

OSL

ordering

and

graph-partitioning

based

WGPP

ordering

For

the

problems

with

a

the

actual

orderings

generated

by

WGPP

are

based

on

minimum

degree

and

are

better

than

the

ones

shown

in

the

table

TOSL

TWGPP

TOSL

TWGPP

-LRB-

seconds

-RRB-

-LRB-

seconds

-RRB-

-LRB-

ratio

-RRB-

Total

of

all

problems

excluding

GISMONDI

Total

of

largest

problems

Total

of

smallest

problems

Total

of

OSL

's

best

-LRB-

lowest

ratios

-RRB-

Total

of

WGPP

's

best

-LRB-

highest

ratios

-RRB-

Average

of

the

ratio

TOSL

TWGPP

Median

of

the

ratio

TOSL

TWGPP

Table

Some

statistics

on

total

run

time

with

OSL

and

WGPP

orderings

on

the

problems

in

Table

-LRB-

excluding

GISMONDI

-RRB-

actually

ran

faster

and

with

less

fill-in

when

this

option

was

on

The

table

shows

a

set

of

three

columns

of

numbers

for

OSL

's

internal

ordering

and

WGPP

ordering

The

first

column

-LRB-

NNZ

-RRB-

for

each

ordering

is

the

number

of

nonzeros

in

the

lower

triangular

factor

and

is

indicative

of

the

quality

of

ordering

The

second

column

-LRB-

nf

-RRB-

shows

the

number

of

iterations

required

by

the

barrier

method

used

to

solve

the

problem

This

is

the

number

of

numerical

factorizations

that

were

performed

on

a

symmetric

matrix

whose

row

and

column

permutation

was

computed

once

The

third

column

-LRB-

T

-RRB-

gives

the

total

run

time

Depending

on

the

problem

the

total

run

time

for

OSL

's

ordering

is

dominated

either

by

the

ordering

time

or

numerical

factorization

time

or

both

The

total

run

time

for

WGPP

ordering

which

is

uniformly

fast

is

almost

always

dominated

by

the

numerical

factorization

time

Symbolic

factorization

triangular

solution

and

computing

the

product

consume

only

a

small

fraction

of

the

total

run

time

for

both

orderings

-LRB-

unless

A

is

a

very

narrow

rectangular

matrix

in

which

case

computing

the

product

can

be

relatively

costly

-RRB-

Note

that

besides

ordering

time

and

ordering

quality

the

number

of

iterations

also

affects

the

overall

solution

time

The

number

of

iterations

-LRB-

nf

-RRB-

often

varies

slightly

between

the

two

orderings

but

this

variation

is

completely

random

and

it

is

safe

to

assume

that

the

effects

of

this

variation

cancel

out

over

a

large

set

of

test

problems

The

last

column

of

Table

is

the

ratio

of

the

run

times

with

OSL

's

internal

MD

based

ordering

-LRB-

TOSL

-RRB-

and

WGPP

ordering

-LRB-

TWGPP

-RRB-

An

entry

greater

than

in

the

last

columns

indicates

that

that

the

problem

was

solved

faster

with

the

WGPP

ordering

Table

abstracts

some

statistics

from

the

run

times

of

the

LP

problems

listed

in

Table

for

the

two

orderings

The

entire

suite

of

problems

-LRB-

excluding

-RRB-

was

solved

about

times

faster

with

WGPP

ordering

than

with

OSL

's

internal

ordering

The

advantage

of

WGPP

was

more

pronounced

for

the

larger

problems

its

use

reduced

the

total

run

time

for

the

ten

most

time-consuming

problems

by

a

factor

of

almost

At

the

other

end

of

the

spectrum

WGPP

resulted

in

only

a

marginal

speedup

by

a

factor

of

for

the

ten

smallest

problems

Since

all

ordering

methods

are

heuristics

no

single

one

can

be

guaranteed

to

be

better

than

others

for

all

problems

There

were

several

problems

for

which

WGPP

resulted

in

a

slower

solution

However

an

average

problem

was

speeded

up

by

more

than

and

half

of

the

problems

experienced

close

to

speedup

with

the

use

of

WGPP

WGPP

increased

the

cumulative

run

time

of

the

ten

worst

cases

by

but

reduced

the

cumulative

run

time

by

in

the

ten

best

cases

Out

of

there

were

eight

problems

for

which

WGPP

cut

the

solution

time

in

less

than

half

while

there

were

for

which

the

solution

time

increased

by

more

than

a

factor

of

two

The

run

time

reduced

for

problems

and

increased

for

The

results

in

Table

capture

the

effect

of

both

ordering

time

and

ordering

quality

In

Table

we

present

similar

statistics

on

the

fill-in

or

the

number

of

nonzeros

in

the

triangular

factor

resulting

from

the

two

orderings

This

is

solely

an

indication

of

ordering

quality

The

reader

must

note

that

typically

a

small

largest

problem

of

the

suite

GISMONDI

could

not

proceed

beyond

symbolic

factorization

-LRB-

which

computes

the

number

of

nonzeros

in

the

factor

before

actual

numerical

factorization

-RRB-

because

there

was

n't

enough

memory

to

store

the

factor

NNZOSL

NNZWGPP

NNZOSL/NNZWGPP

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

ratio

-RRB-

Total

of

all

problems

Total

of

largest

problems

Total

of

smallest

problems

Total

of

OSL

's

best

-LRB-

smallest

ratios

-RRB-

Total

of

WGPP

's

best

-LRB-

highest

ratios

-RRB-

Average

of

the

ratio

NNZOSL/NNZWGPP

Median

of

the

ratio

NNZOSL/NNZWGPP

Table

Some

statistics

on

the

number

of

nonzeros

in

the

triangular

factor

with

OSL

and

WGPP

orderings

on

the

problems

in

Table

reduction

-LRB-

increase

-RRB-

in

fill-in

results

in

a

large

reduction

-LRB-

increase

-RRB-

in

the

operation

count

and

hence

the

run

time

of

numerical

factorization

-LSB-

-RSB-

Table

shows

that

WGPP

reduced

the

fill-in

in

almost

as

many

problems

as

the

number

of

problems

in

which

it

increased

the

fill-in

However

most

increases

were

much

smaller

than

most

decreases

This

is

indicated

by

the

fact

that

the

original

OSL

ordering

produced

more

average

and

more

cumulative

fill-in

than

WGPP

over

the

entire

set

of

test

problems

It

is

also

important

that

the

improvements

were

more

significant

for

larger

problems

than

for

smaller

ones

For

the

largest

ten

problems

the

original

ordering

was

worse

in

terms

of

fill-in

whereas

for

the

smallest

ten

problems

WGPP

was

off

by

The

averages

shown

in

Tables

and

improved

further

when

the

option

to

attempt

a

second

ordering

was

turned

on

This

option

reduced

fill-in

and

the

overall

run

time

for

the

problems

whose

serial

numbers

are

prefixed

by

a

in

Table

There

was

no

noticeable

increase

in

the

overall

run

time

of

the

problems

for

which

this

option

attempted

a

second

ordering

-LRB-

minimum

degree

-RRB-

but

did

not

succeed

in

reducing

the

fill-in

and

used

the

first

-LRB-

the

original

WGPP

-RRB-

ordering

The

overall

run

time

of

such

problems

remained

practically

unchanged

Tables

and

show

the

statistics

on

the

total

solution

time

and

fill-in

with

this

option

turned

on

TOSL

TWGPP

TOSL

TWGPP

-LRB-

seconds

-RRB-

-LRB-

seconds

-RRB-

-LRB-

ratio

-RRB-

Total

of

all

problems

excluding

GISMONDI

Total

of

largest

problems

Total

of

smallest

problems

Total

of

OSL

's

best

-LRB-

lowest

ratios

-RRB-

Total

of

WGPP

's

best

-LRB-

highest

ratios

-RRB-

Average

of

the

ratio

TOSL

TWGPP

Median

of

the

ratio

TOSL

TWGPP

Table

The

statistics

on

total

run

time

with

OSL

and

WGPP

orderings

on

the

problems

in

Table

-LRB-

excluding

GISMONDI

-RRB-

with

the

WGPP

option

of

attempting

a

second

ordering

based

on

minimum

degree

turned

on

NNZOSL

NNZWGPP

NNZOSL/NNZWGPP

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

ratio

-RRB-

Total

of

all

problems

Total

of

largest

problems

Total

of

smallest

problems

Total

of

OSL

's

best

-LRB-

smallest

ratios

-RRB-

Total

of

WGPP

's

best

-LRB-

highest

ratios

-RRB-

Average

of

the

ratio

NNZOSL/NNZWGPP

Median

of

the

ratio

NNZOSL/NNZWGPP

Table

The

statistics

on

the

number

of

nonzeros

in

the

triangular

factor

with

OSL

and

WGPP

orderings

for

the

problems

in

Table

with

the

WGPP

option

of

attempting

a

second

ordering

based

on

minimum

degree

turned

on

Concluding

Remarks

In

this

paper

we

present

strong

empirical

evidence

that

graph-partitioning

-LRB-

GP

-RRB-

based

sparse

matrix

ordering

is

a

suitable

alternative

to

the

traditional

minimum-degree

-LRB-

MD

-RRB-

based

ordering

techniques

for

interior-point

methods

We

describe

a

GP

based

ordering

heuristic

for

LP

problems

that

is

a

part

of

the

more

comprehensive

WGPP

library

-LSB-

-RSB-

developed

at

IBM

Research

for

graph

partitioning

and

ordering

different

types

of

sparse

matrices

By

means

of

experiments

over

a

large

suite

of

real-life

LP

problems

we

demonstrate

the

superiority

of

WGPP

ordering

over

other

state-of-the-art

ordering

softwares

in

use

in

terms

of

both

ordering

time

and

quality

Our

experiments

show

that

WGPP

ordering

is

particularly

useful

for

solving

large

problems

Therefore

we

hope

that

GP

based

orering

will

not

only

help

reduce

the

solution

time

of

large

LP

problems

but

will

also

render

larger

problems

solvable

on

today

's

workstation

class

computers

by

reducing

the

run

time

and

memory

requirements

of

these

problems

GP

based

ordering

also

aids

in

effective

parallelization

of

some

of

the

key

computations

in

an

interior-point

algorithm

-LSB-

-RSB-

Acknowledgements

I

am

grateful

to

John

Forrest

for

providing

the

test

problems

for

providing

access

to

his

workstation

to

run

the

experiments

and

for

his

help

in

creating

the

experimental

set

up

by

exposing

OSL

's

internal

ordering

routines

I

am

also

thankful

to

George

Karypis

and

Vipin

Kumar

for

useful

discussions

during

the

earlier

part

of

this

work

and

to

Edward

Rothberg

for

his

comments

on

an

earlier

draft

of

this

paper

The

sparse

matrix

ordering

subroutine

of

WGPP

uses

the

AMD

-LSB-

-RSB-

code

written

by

Tim

Davis

Patrick

Amestoy

and

Iain

Duff

to

label

small

subgraphs

with

a

few

hundred

nodes

References

-LSB-

-RSB-

Cleve

Ashcraft

and

Joseph

W.-H

Liu

Robust

ordering

of

sparse

matrices

using

multisection

Technical

Report

CS

Department

of

Computer

Science

York

University

Ontario

Canada

-LSB-

-RSB-

Cleve

Ashcraft

and

Joseph

W.-H

Liu

Generalized

nested

dissection

Some

recent

progress

In

Proceedings

of

Fifth

SIAM

Conference

on

Applied

Linear

Algebra

Snowbird

Utah

June

-LSB-

-RSB-

T.

Bui

and

C.

Jones

A

heuristic

for

reducing

fill

in

sparse

matrix

factorization

In

Proceedings

of

the

Sixth

SIAM

Conference

on

Parallel

Processing

for

Scientific

Computing

pages

-LCB-

-LSB-

-RSB-

G.

B.

Dantzig

Linear

Programming

and

Extensions

Princeton

University

Press

Princeton

NJ

-LSB-

-RSB-

Timothy

A.

Davis

Patrick

Amestoy

and

Iain

S.

Duff

An

approximate

minimum

degree

ordering

algorithm

Technical

Report

Computer

and

Information

Sciences

Department

University

of

Florida

Gainesville

FL

-LSB-

-RSB-

Iain

S.

Duff

and

Torbjorn

Wiberg

Remarks

on

implementations

of

O

-LRB-

-RRB-

assignment

algorithms

ACM

Transactions

on

Mathematical

Software

-LCB-

-LSB-

-RSB-

C.

M.

Fiduccia

and

R.

M.

Mattheyses

A

linear

time

heuristic

for

improving

network

partitions

In

Proceedings

of

the

IEEE

Design

Automation

Conference

pages

-LCB-

-LSB-

-RSB-

Alan

George

Nested

dissection

of

a

regular

finite-element

mesh

SIAM

Journal

on

Numerical

Ananlysis

-LCB-

-LSB-

-RSB-

Alan

George

and

Joseph

W.-H

Liu

Computer

Solution

of

Large

Sparse

Positive

Definite

Systems

Prentice-Hall

Englewood

Cliffs

NJ

-LSB-

-RSB-

Alan

George

and

Joseph

W.-H

Liu

The

evolution

of

the

minimum

degree

ordering

algorithm

SIAM

Review

-LRB-

-RRB-

-LCB-

March

-LSB-

-RSB-

Alan

George

Joseph

W.-H

Liu

and

Esmond

G.-Y

Ng

Communication

reduction

in

parallel

sparse

Cholesky

factorization

on

a

hypercube

In

M.

T.

Heath

editor

Hypercube

Multiprocessors

pages

-LCB-

SIAM

Philadelphia

PA

-LSB-

-RSB-

Madhurima

Ghose

and

Edward

Rothberg

A

parallel

implementtaion

of

the

multiple

minimum

degree

ordering

heuristic

Technical

report

Old

Dominion

University

Norfolk

VA

-LSB-

-RSB-

P.

E.

Gill

W.

Murray

M.

A.

Saunders

J.

A.

Tomlin

and

M.

H.

Wright

On

projected

Newton

barrier

methods

for

linear

programming

and

an

equivalence

to

Karmarkar

's

projective

method

Mathematical

Programming

-LCB-

-LSB-

-RSB-

Anshul

Gupta

Fast

and

effective

algorithms

for

graph

partitioning

and

sparse

matrix

reordering

Technical

Report

RC

-LRB-

-RRB-

IBM

T.

J.

Watson

Research

Center

Yorktown

Heights

NY

July

Available

on

the

WWW

at

the

IBM

Research

CyberJournal

site

at

-LSB-

-RSB-

Anshul

Gupta

WGPP

Watson

graph

partitioning

-LRB-

and

sparse

matrix

ordering

-RRB-

package

Users

manual

Technical

Report

RC

-LRB-

-RRB-

IBM

T.

J.

Watson

Research

Center

Yorktown

Heights

NY

May

-LSB-

-RSB-

Anshul

Gupta

George

Karypis

and

Vipin

Kumar

Highly

scalable

parallel

algorithms

for

sparse

matrix

factorization

Technical

Report

Department

of

Computer

Science

University

of

Minnesota

Minneapolis

MN

To

appear

in

IEEE

Transactions

on

Parallel

and

Distributed

Systems

Postscript

file

available

via

anonymous

FTP

from

the

site

ftp

-LSB-

-RSB-

Anshul

Gupta

and

Vipin

Kumar

Parallel

algorithms

for

forward

and

back

substitution

in

direct

solution

of

sparse

linear

systems

In

Supercomputing

Proceedings

December

-LSB-

-RSB-

Bruce

Hendrickson

and

Robert

Leland

A

multilevel

algorithm

for

partitioning

graphs

In

Supercomputing

Proceedings

Also

available

a

Technical

Report

Sandia

National

Laboratories

Albuquerque

NM

-LSB-

-RSB-

Bruce

Hendrickson

and

Edward

Rothberg

Improving

the

runtime

and

quality

of

nested

dissection

ordering

Technical

Report

SGI

manuscript

Silicon

Graphics

Inc.

Mountain

View

CA

-LSB-

-RSB-

Narendra

Karmarkar

A

new

polynomial-time

algorithm

for

linear

programming

Combinatorica

-LRB-

-RRB-

-LCB-

-LSB-

-RSB-

George

Karypis

Anshul

Gupta

and

Vipin

Kumar

Parallel

formulation

of

interior

point

algorithms

Technical

Report

Department

of

Computer

Science

University

of

Minnesota

Minneapolis

MN

April

A

short

version

appears

in

Supercomputing

Proceedings

-LSB-

-RSB-

George

Karypis

and

Vipin

Kumar

A

fast

and

high

quality

multilevel

scheme

for

partitioning

irregular

graphs

Technical

Report

TR

Department

of

Computer

Science

University

of

Minnesota

-LSB-

-RSB-

George

Karypis

and

Vipin

Kumar

METIS

Unstructured

graph

partitioning

and

sparse

matrix

ordering

system

Technical

report

Department

of

Computer

Science

University

of

Minnesota

-LSB-

-RSB-

George

Karypis

and

Vipin

Kumar

Parallel

multilevel

graph

partitioning

Technical

Report

TR

Department

of

Computer

Science

University

of

Minnesota

-LSB-

-RSB-

B.

W.

Kernighan

and

S.

Lin

An

efficient

heuristic

procedure

for

partitioning

graphs

The

Bell

System

Technical

Journal

-LSB-

-RSB-

Optimization

Subroutine

Library

Guide

and

Reference

IBM

Corporation

Release

Fifth

Edition

February

Publication

number

-LSB-

-RSB-

R.

J.

Lipton

D.

J.

Rose

and

R.

E.

Tarjan

Generalized

nested

dissection

SIAM

Journal

on

Numerical

Analysis

-LCB-

-LSB-

-RSB-

R.

J.

Lipton

and

R.

E.

Tarjan

A

separator

theorem

for

planar

graphs

SIAM

Journal

on

Applied

Mathematics

-LCB-

-LSB-

-RSB-

Irvin

J.

Lustig

Roy

E.

Marsten

and

David

F.

Shanno

Computational

experience

with

a

primal-dual

interior-point

method

for

linear

programming

Linear

Algebra

and

Its

Applications

-LCB-

-LSB-

-RSB-

Irvin

J.

Lustig

Roy

E.

Marsten

and

David

F.

Shanno

On

implementing

Mehrotra

's

predictor-corrector

interior-point

method

for

linear

programming

SIAM

Journal

on

Optimization

-LRB-

-RRB-

-LCB-

August

-LSB-

-RSB-

N.

Megiddo

Pathways

to

the

optimal

set

in

linear

programming

In

N.

Megiddo

editor

Progress

in

Mathematical

Programming

pages

-LCB-

Springer-Verlag

New

York

-LSB-

-RSB-

Sanjay

Mehrotra

On

the

implementation

of

a

primal-dual

interior

point

method

SIAM

Journal

on

Optimization

-LRB-

-RRB-

-LCB-

November

-LSB-

-RSB-

Alex

Pothen

and

C.-J

Fan

Computing

the

block

triangular

form

of

a

sparse

matrix

ACM

Transactions

on

Mathematical

Software

-LSB-

-RSB-

Edward

Rothberg

and

Bruce

Hendrickson

Sparse

matrix

ordering

methods

for

interior

point

linear

programming

Technical

Report

SGI

manuscript

Silicon

Graphics

Inc.

Mountain

View

CA

-LSB-

-RSB-

M.

Yannakakis

Computing

the

minimum

fill-in

is

NP-complete

SIAM

Journal

on

Algebraic

Discrete

Methods

-LCB-

graph partitioning
linear programming
ordering
sparse linear systems
sparse matrices