McGill

University

School

of

Computer

Science

On

the

Limits

of

Program

Parallelism

and

its

Smoothability

ACAPS

Technical

Memo

June

Kevin

B.

Theobald

Guang

R.

Gao

Laurie

J.

Hendren

Advanced

Compilers

Architectures

and

Parallel

Systems

Group

ACAPS

ffl

School

of

Computer

Science

ffl

University

St.

ffl

Montr?eal

ffl

Canada

ffl

Some

recent

studies

of

the

so-called

limits

on

instruction

parallelism

in

application

programs

have

reported

limits

that

are

surprisingly

low

-LRB-

instructions

-RRB-

for

well-known

benchmark

programs

In

this

paper

we

report

results

of

a

new

study

of

instruction-level

parallelism

and

the

smoothability

of

this

parallelism

In

addition

to

showing

a

strikingly

high

limit

of

parallelism

for

an

oracle

machine

model

we

also

study

the

following

new

aspects

of

parallelism

and

smoothability

Parallelism

Limits

In

addition

to

confirming

some

results

recently

reported

-LRB-

i.e.

by

Wilson

and

Lam

-LSB-

-RSB-

-RRB-

our

work

also

provides

answers

to

the

following

important

questions

for

architects

and

compiler

writters

which

were

left

open

ffl

What

are

the

most

important

characteristics

of

the

oracle

machine

models

ffl

What

happens

if

we

allow

each

test

program

to

run

to

full

completion

instead

of

stopping

after

a

limited

number

of

instructions

ffl

Do

these

results

apply

to

other

real

programs

-LRB-

run

to

completion

-RRB-

in

addition

to

the

selected

benchmark

programs

ffl

How

do

various

restrictions

on

the

use

and

reuse

of

memory

impact

the

potential

parallelism

Smoothability

In

our

study

smoothability

is

measured

quantitatively

and

compared

for

a

number

of

programs

Our

results

indicate

that

the

parallelism

obtained

has

a

relatively

smooth

temporal

profile

although

different

programs

may

have

different

smoothabilities

In

particular

our

results

indicate

that

machine

models

with

speculative

multithreading

exhibit

impressive

levels

of

instruction-level

parallelism

with

high

smoothability

Contents

Introduction

Methodology

and

Models

Our

Trace

Simulation

Methodology

Memory

Disambiguation

Register

and

Memory

Renaming

Control

Barrier

Elimination

Finite

Windows

Unrolling

Experimental

Models

Used

Smoothability

Experimental

Results

Effects

of

Branch

and

Jump

Prediction

Finite

Scheduling

Windows

Frugal

Oracles

Unrolling

Results

Smoothability

Results

Compiler

Optimization

Previous

Studies

Conclusions

A

Experimental

Testbed

List

of

Figures

Packing

parallelizable

instructions

An

example

Standard

smoothability

profile

Standard

processor

utilization

profile

List

of

Tables

Oracle

and

Machine

Models

Benchmarks

and

test

cases

used

A

summary

of

results

The

effects

of

finite

window

sizes

The

effect

of

frugal

use

of

memory

The

effects

of

unrolling

The

effect

of

finite

processors

Introduction

At

the

current

rate

of

progress

in

VLSI

technology

it

will

be

possible

by

the

turn

of

the

century

to

produce

a

single

chip

with

tens

to

hundreds

of

millions

of

transistors

This

level

of

integration

can

be

used

to

implement

a

high-density

high-performance

processor

on

a

single

chip

perhaps

one

based

on

advanced

architectural

ideas

such

as

multithreaded

architectures

-LSB-

A

DCC

-RSB-

Such

processors

can

then

become

the

building

blocks

of

an

advanced

massively-parallel

computer

Those

contemplating

the

design

of

these

highly-parallel

machines

must

ask

two

fundamental

questions

Question

-LRB-

Limits

of

Parallelism

-RRB-

How

much

parallelism

exists

in

the

target

applications

Question

-LRB-

Smoothness

of

Parallelism

-RRB-

Is

the

available

parallelism

smooth

enough

to

be

effectively

exploited

In

order

to

justify

having

machines

supporting

extensive

parallelism

there

must

be

enough

useful

work

to

do

at

any

point

in

time

to

keep

most

of

the

processors

usefully

busy

This

means

that

not

only

must

there

be

enough

parallelism

in

the

application

to

be

exploited

by

these

processors

but

the

parallelism

must

also

be

smoothable

to

be

fairly

evenly

distributed

over

time

Otherwise

Amdahl

's

Law

will

take

effect

and

prevent

the

computer

from

achieving

a

good

speed-up

There

have

been

many

experiments

involving

a

wide

variety

of

machine

models

and

target

applications

to

measure

the

limits

of

parallelism

that

may

be

exploited

in

a

program

-LSB-

BYP

-RSB-

Some

have

reported

limits

that

are

surprisingly

low

-LRB-

instructions

-RRB-

for

well-known

benchmark

programs

even

under

their

best

machine

model

-LSB-

-RSB-

In

-LSB-

-RSB-

it

has

been

demonstrated

that

such

limits

can

become

substantially

higher

if

more

powerful

machine

models

are

considered

which

can

exploit

parallelism

by

eliminating

constraints

due

to

sequential

control

flow

Most

previous

studies

are

interested

in

Question

above

In

this

paper

we

are

interested

in

answers

to

both

Questions

and

and

we

report

results

of

our

study

which

are

unique

in

the

following

aspects

Limits

of

Parallelism

In

answering

Question

our

results

have

shown

a

strikingly

high

limit

of

parallelism

for

oracle

machine

models

In

addition

to

SPEC

benchmark

programs

we

are

also

interested

in

a

wider

variety

of

real

programs

and

to

that

end

we

have

added

an

interesting

program

for

large-vocabulary

speech

recognition

which

we

received

from

industry

For

our

experiments

we

allow

each

test

program

to

run

to

its

completion

in

some

cases

the

traces

have

lengths

of

over

billion

instructions

This

may

turn

out

to

be

critical

in

providing

measurement

of

smoothness

over

the

entire

execution

of

a

program

-LRB-

See

Section

-RRB-

We

also

study

the

impact

of

various

restrictions

on

the

use

of

memory

to

the

potential

parallelism

Thus

our

results

provide

answers

to

some

unanswered

questions

in

previous

studies

Smoothability

of

Parallelism

We

introduce

smoothability

as

a

measure

to

characterize

instructionlevel

parallelism

Smoothability

is

measured

quantitatively

and

compared

for

a

number

of

programs

Our

results

indicate

that

the

parallelism

obtained

has

a

relatively

smooth

temporal

profile

In

particular

under

speculative

multithreading

our

collection

of

both

scientific

and

non-scientific

programs

not

only

exhibit

impressive

instruction-level

parallelism

but

also

exhibit

a

high

degree

of

uniformity

in

the

parallelism

To

search

for

the

limits

of

parallelism

and

measure

their

smoothability

under

different

architectural

ideas

we

have

developed

an

experimental

testbed

This

testbed

is

based

on

a

trace

simulation

method

similar

to

previous

experiments

-LSB-

-RSB-

The

tool

allows

us

to

analyze

the

execution

of

real

code

and

calculates

under

various

architecture

models

how

much

parallelism

could

potentially

be

attained

for

a

given

model

Using

this

tool

we

begin

with

a

true

Omniscient

Oracle

machine

model

and

measure

the

idealized

limit

of

parallelism

We

then

examine

the

degradation

in

performance

as

we

selectively

remove

key

features

from

the

Omniscient

Oracle

and

move

it

towards

more

realistic

machine

models

at

different

levels

A

total

of

models

are

used

in

the

study

including

the

oracle

Parallelism

smoothness

was

measured

for

some

models

in

order

to

study

the

smoothability

aspect

described

later

in

the

paper

Our

oracle

machine

models

ignore

control

dependencies

entirely

-LRB-

in

this

sense

similar

to

the

oracle

proposed

in

-LSB-

-RSB-

-RRB-

While

the

results

obtained

from

the

oracle

are

an

upper

bound

and

may

be

unrealistically

high

we

believe

that

they

give

a

good

indication

of

the

parallelism

that

may

be

achieved

with

suitably-designed

parallel

algorithms

It

is

not

the

goal

of

this

research

to

design

a

machine

which

can

execute

the

benchmarks

in

a

manner

corresponding

to

the

analysis

in

our

experiments

Rather

we

are

interested

in

studying

the

upper

bounds

of

potential

parallelism

inherent

in

the

application

Our

objective

in

this

paper

is

not

to

provide

a

recipe

for

the

design

of

specific

architectures

and

measure

the

possible

parallelism

achievable

Rather

we

study

what

effects

of

the

limitations

of

each

architecture

model

make

it

impossible

to

achieve

the

parallelism

limits

In

the

next

section

we

describe

how

trace

analysis

works

and

develop

the

models

used

in

our

study

Smoothability

is

defined

and

discussed

in

Section

The

experimental

results

and

their

ramifications

are

presented

in

In

Section

we

compare

our

work

to

previous

studies

The

final

section

Section

summarizes

the

paper

discusses

the

conclusions

that

may

be

drawn

from

this

study

and

describes

how

we

plan

to

extend

this

analysis

tool

in

the

future

We

briefly

describe

implementation

of

the

tool

in

an

appendix

Methodology

and

Models

This

section

describes

the

trace

simulation

techniques

and

machine

models

used

in

our

study

First

we

show

conceptually

how

our

trace

analysis

reveals

parallelism

in

a

program

and

how

various

features

added

to

a

model

affect

potential

parallelism

These

effects

are

illustrated

using

a

simple

loop

Next

we

present

the

basic

machine

models

used

in

this

study

and

discuss

what

the

use

of

these

models

to

measure

parallelism

can

tell

us

about

exploiting

parallelism

in

real

computers

Our

Trace

Simulation

Methodology

Trace

analysis

begins

with

a

trace

of

the

execution

of

a

program

This

trace

consists

of

a

stream

of

operations

representing

the

actual

order

of

instructions

executed

-LRB-

not

the

static

object

code

-RRB-

For

each

executed

operation

the

trace

gives

the

opcode

PC

address

memory-access

address

-LRB-

if

any

-RRB-

and

the

destination

of

a

branch

or

jump

The

analyzer

reads

the

operations

from

the

stream

and

packs

them

into

parallel

instructions

-LRB-

PI

-RRB-

As

the

analyzer

reads

each

operation

in

the

trace

it

inserts

the

operation

into

the

earliest

PI

possible

while

simultaneously

respecting

the

dependencies

between

that

operation

and

all

previous

operations

The

following

types

of

dependency

between

operations

may

exist

ffl

Data

dependency

if

an

operation

reads

a

storage

which

was

most

recently

written

by

operation

and

if

was

packed

into

PI

p

then

can

be

scheduled

no

earlier

than

PI

p

ffl

Anti-dependency

if

an

operation

writes

a

storage

location

which

was

most

recently

read

by

operation

and

if

was

packed

into

PI

p

then

can

be

scheduled

no

earlier

than

PI

p.

-LRB-

It

is

assumed

that

the

write

and

read

can

occur

simultaneously

and

will

read

the

proper

value

-RRB-

ffl

Output

dependency

if

an

operation

writes

a

storage

location

which

was

most

recently

written

by

operation

and

if

was

packed

into

PI

p

then

can

be

scheduled

no

earlier

than

PI

p

ffl

Control

dependency

if

operation

is

the

most

recent

conditional

branch

prior

to

a

given

operation

in

the

trace

and

if

was

packed

into

PI

p

then

can

be

scheduled

no

earlier

than

PI

p

In

this

study

we

assume

that

all

operations

-LRB-

including

memory

references

-RRB-

take

a

single

cycle

to

execute

Memory

is

global

with

unlimited

access

and

there

are

no

caches

In

our

basic

models

infinitely

many

functional

units

are

available

to

execute

all

operations

which

can

be

packed

into

one

PI

-LRB-

we

limit

the

number

of

processors

later

when

we

study

smoothability

discussed

in

the

next

section

-RRB-

In

most

cases

we

also

allow

operations

arbitrarily

far

apart

in

the

dynamic

trace

to

be

packed

into

the

same

PI

so

long

as

all

other

dependency

constraints

are

obeyed

-LRB-

Real

machines

may

not

be

so

generous

but

this

study

aims

to

determine

how

much

parallelism

is

truly

inherent

in

a

program

independent

of

any

particular

hardware

limitations

-RRB-

To

illustrate

the

basic

principle

consider

this

simple

loop

fragment

for

multiplying

two

vectors

componentwise

ld

-LSB-

-RSB-

ld

-LSB-

-RSB-

fmuls

st

-LSB-

-RSB-

subcc

bg

nop

A

dynamic

trace

would

have

the

sequence

repeated

many

times

The

left

half

of

Figure

shows

how

our

trace

analyzer

would

pack

these

operations

into

PI

's

if

all

four

dependency

types

listed

above

were

obeyed

Arcs

are

drawn

to

show

the

data

dependencies

between

operations

The

arc

with

a

small

tick

mark

drawn

from

to

represents

an

anti-dependency

The

dashed

line

indicates

a

barrier

caused

by

the

conditional

branch

at

which

prevents

all

future

operations

from

being

scheduled

before

that

barrier

One

iteration

of

the

loop

can

be

initiated

every

cycles

so

the

parallelism

is

Other

models

described

later

relax

the

dependency

constraints

or

tighten

the

resource

limitations

The

following

subsections

describe

how

these

constraints

may

be

modified

through

such

means

as

register

renaming

and

branch

prediction

and

the

effects

such

changes

will

have

on

overall

parallelism

storage

location

means

either

a

register

or

a

memory

cell

P

Ii

P

Ii

P

Ii

P

Ii

P

Ii

R

P

Ii

P

Ii

P

Ii

P

Ii

P

Ii

Figure

Packing

parallelizable

instructions

An

example

Memory

Disambiguation

Memory

references

ca

n't

always

be

determined

at

compile-time

due

to

indirect

addressing

A

conservative

analysis

would

assume

that

any

two

memory

references

could

refer

to

the

same

memory

location

so

that

a

dependency

would

exist

between

them

Thus

a

conservative

scheduler

would

have

to

assume

that

an

anti-dependency

might

exist

between

and

making

it

impossible

to

overlap

separate

iterations

of

the

loop

However

since

the

actual

addresses

of

memory

accesses

appear

in

the

trace

the

analyzer

can

determine

at

run-time

whether

two

memory

references

really

conflict

This

would

model

the

potential

effects

of

perfect

compiler

alias-analysis

and/or

special

hardware

to

check

for

memory

conflicts

at

runtime

It

can

also

model

an

architecture

with

a

more

disciplined

use

of

memory

that

makes

such

conflicts

more

explicit

-LRB-

hence

easier

to

detect

-RRB-

Our

analyzer

supports

this

feature

-LRB-

run-time

memory

disambiguation

-RRB-

as

an

option

Register

and

Memory

Renaming

A

false

dependency

-LRB-

anti

or

output

-RRB-

exists

between

two

operations

when

one

must

follow

the

other

not

because

the

latter

requires

data

produced

by

the

former

but

merely

because

the

latter

needs

to

reuse

a

storage

location

-LRB-

register

or

memory

cell

-RRB-

used

by

the

former

An

example

of

this

in

the

code

sample

is

the

use

of

register

for

the

loop

index

which

can

not

be

updated

-LRB-

by

-RRB-

before

being

used

to

construct

a

memory

address

-LRB-

by

-RRB-

Thus

overlapping

the

iterations

of

the

loop

body

as

in

software

pipelining

is

impossible

False

dependencies

also

exist

in

references

to

main

memory

This

can

limit

parallelism

in

two

ways

Many

sequential

programs

reuse

data

structures

-LRB-

such

as

arrays

-RRB-

to

optimize

the

use

of

memory

Such

optimization

may

be

a

good

idea

in

programs

with

a

single

thread

of

control

In

parallel

machines

however

reusing

memory

in

this

way

means

that

available

processors

can

not

start

operating

on

an

updated

version

of

a

data

structure

while

other

processors

continue

to

read

the

old

version

because

the

update

must

wait

until

all

processors

are

finished

with

the

old

copy

Under

the

conventional

stack

architecture

procedure

calls

at

the

same

level

in

a

program

reuse

the

same

portion

of

the

stack

leading

to

contention

for

that

part

of

the

stack

For

instance

when

executing

object

code

corresponding

to

x

sin

-LRB-

y

-RRB-

cos

-LRB-

z

-RRB-

the

processor

can

not

push

z

onto

the

stack

and

call

cos

until

it

has

popped

the

return

value

of

sin

-LRB-

y

-RRB-

off

the

stack

even

though

these

two

function

calls

are

independent

and

could

run

in

parallel

The

inhibiting

effects

of

false

dependencies

can

be

eliminated

by

ensuring

that

each

register

or

memory

location

is

written

only

once

In

a

real

processor

false

dependencies

can

be

reduced

by

creating

a

sufficiently

large

register

file

coupled

with

other

techniques

such

as

register

renaming

The

architecture

provides

additional

registers

that

are

not

programmer-visible

and

dynamically

allocates

them

to

store

each

new

value

generated

This

will

enforce

a

single-assignment

rule

and

hence

resolves

the

anti-

and

output-dependences

False

dependencies

in

the

heap

can

in

principle

be

eliminated

by

adhering

to

the

single-assignment

rule

either

explicitly

in

the

programming

language

or

by

using

memory

renaming

Conventional

architectures

do

not

presently

perform

memory

renaming

although

dynamic

dataflow

machines

with

colored

tokens

employ

a

form

of

renaming

False

dependencies

in

the

stack

can

be

caused

by

the

linear

stack

model

the

standard

runtime

memory

organization

for

supporting

nested

procedure

invocations

on

sequential

machines

Such

false

dependencies

can

be

eliminated

for

example

by

organizing

the

memory

frames

for

procedure

invocation

in

a

tree-like

structure

as

proposed

in

several

multithreaded

architecture

models

-LSB-

CSS

-RSB-

Infinite

renaming

in

which

a

register

or

memory

location

can

be

renamed

any

number

of

times

is

equivalent

to

ignoring

all

false

dependencies

between

objects

of

a

particular

type

For

instance

if

register

renaming

is

applied

to

the

dependence

graph

in

Figure

then

can

be

executed

in

parallel

with

and

of

the

same

iteration

as

the

anti-dependency

with

no

longer

exists

This

moves

operation

up

as

well

so

the

conditional-branch

barrier

has

moved

up

by

PI

's

If

perfect

alias

analysis

is

also

used

then

the

iteration

issue

rate

has

increased

to

once

every

other

cycle

raising

parallelism

to

as

the

right

half

of

Figure

shows

Control

Barrier

Elimination

example

The

IBM

RISC

superscalar

machine

implements

a

form

of

register

renaming

in

its

floating

point

unit

-LRB-

FPU

-RRB-

assumes

the

input

and

output

arrays

do

not

overlap

If

memory

disambiguation

is

not

included

in

the

model

then

and

can

not

execute

until

of

the

previous

iteration

has

finished

because

there

might

be

a

data

dependence

between

them

Previous

parallelism

experiments

in

which

conditional

branches

were

barriers

to

upward

motion

of

operations

generally

had

low

parallelism

especially

in

non-numerical

programs

This

is

because

conditional

branches

occur

quite

frequently

in

most

programs

which

limits

the

search

for

parallelism

to

small

basic

blocks

in

the

code

To

get

better

parallelism

results

one

must

consider

models

representing

possible

architectures

which

reduce

the

deleterious

effects

of

conditional

branches

The

oracle

introduced

in

-LSB-

-RSB-

makes

the

most

optimistic

assumptions

by

ignoring

control

dependencies

entirely

Our

oracles

like

most

others

also

ignore

output

and

anti-dependencies

so

that

parallelism

is

only

limited

by

true

data

dependencies

in

the

program

It

is

possible

to

use

more

conservative

models

between

the

extremes

of

having

an

oracle

and

forcing

barriers

at

every

conditional

branch

One

method

is

branch

prediction

in

which

the

processor

tries

to

decide

which

branch

is

most

likely

to

be

taken

and

takes

that

branch

When

a

predicted

branch

is

taken

and

it

later

turns

out

that

that

branch

should

not

have

been

taken

then

all

instructions

executed

after

the

incorrect

branch

must

be

aborted

and

their

side-effects

reversed

There

can

be

more

branches

in

between

the

time

when

a

mis-predicted

branch

is

taken

and

the

time

at

which

this

error

is

discovered

which

means

that

at

any

time

there

may

be

multiple

active

machine

states

only

one

of

which

is

valid

Branch

prediction

is

modeled

by

the

trace

analyzer

by

ignoring

barriers

created

by

correctlypredicted

branches

and

keeping

barriers

created

by

incorrectly-predicted

branches

Wall

describes

a

method

of

dynamic

branch

prediction

in

which

the

prediction

of

which

way

to

branch

comes

from

a

count

which

is

based

on

the

history

of

previous

executions

of

that

branch

Jump

prediction

is

done

simply

by

predicting

that

the

processor

will

jump

to

the

previous

jump

destination

Branch

prediction

counts

and

jump

prediction

addresses

are

stored

in

tables

hashed

by

the

lower

bits

of

the

PC

address

of

the

branch

or

jump

In

our

experiments

the

hash

tables

are

large

enough

to

give

each

PC

address

a

unique

table

entry

A

more

conservative

assumption

which

does

not

involve

much

speculation

is

to

assume

a

form

of

coarse-grain

parallelism

in

which

the

barriers

produced

by

branches

in

one

procedure

do

not

affect

the

scheduling

of

operations

within

other

procedures

We

call

this

feature

procedure

separation

Procedure

separation

allows

procedures

with

no

data

dependencies

to

execute

in

parallel

which

they

probably

would

be

unable

to

do

otherwise

if

no

branch

prediction

were

used

-LRB-

Some

previous

studies

-LSB-

-RSB-

extract

more

parallelism

by

analyzing

the

code

to

look

for

instructions

that

are

really

not

control-dependent

on

one

another

even

though

they

are

separated

by

conditional

branches

This

would

allow

for

instance

two

consecutive

but

independent

DO-loops

to

run

concurrently

We

have

not

implemented

this

feature

at

present

-RRB-

Finite

Windows

So

far

we

have

assumed

that

an

operation

from

anywhere

in

the

trace

can

be

placed

in

the

earliest

PI

possible

subject

to

data

and

control

dependencies

In

theory

the

last

operation

in

the

sequential

trace

could

be

packed

into

the

first

PI

However

we

may

place

limitations

on

how

far

apart

two

operations

can

be

packed

One

possible

limitation

is

to

assume

that

the

parallel

machine

can

only

look

so

far

ahead

of

the

program

counter

when

looking

for

operations

that

are

ready

to

be

executed

With

a

limited

window

size

the

analyzer

conceptually

keeps

future

operations

from

the

trace

stream

in

the

window

and

packs

them

into

PI

's

from

the

window

When

the

trace

stream

fills

the

window

the

analyzer

must

issue

the

lowest-numbered

unissued

PI

thereby

making

it

unavailable

for

further

packing

and

remove

the

operations

in

that

PI

from

the

window

This

limitation

was

adopted

by

Wall

who

used

windows

This

models

how

a

real

superscalar

could

schedule

instructions

at

run-time

though

in

practice

Model

Rename

Rename

Branch

Jump

Proc

Window

Unroll

Name

Regs

Memory

Predict

Predict

Sep.

Length

Omniscient

Oracle

yes

all

perfect

perfect

yes

no

Unrolling

Omni

Or

yes

all

perfect

perfect

yes

yes

Myopic

Oracle

yes

all

perfect

perfect

yes

pre

no

Short

Oracle

yes

all

perfect

perfect

yes

post

no

Tree

Oracle

yes

stack

perfect

perfect

yes

no

Linear

Oracle

yes

heap

perfect

perfect

yes

no

Frugal

Oracle

yes

disambig

perfect

perfect

yes

no

Speculative

Multithreaded

yes

all

table

table

yes

no

Unrolling

Spec

Mult

yes

all

table

table

yes

yes

Multithreaded

yes

all

yes

no

Unrolling

Multithreaded

yes

all

yes

yes

Smart

Superscalar

yes

all

no

no

Stupid

Superscalar

no

no

no

Table

Oracle

and

Machine

Models

the

window

size

of

current

superscalars

is

very

small

Another

limitation

is

to

place

a

finite

window

after

the

scheduling

In

this

case

the

analyzer

keeps

a

finite

queue

of

PI

's

available

for

packing

An

arbitrary

number

of

operations

from

the

trace

can

be

packed

into

these

PI

's

but

if

a

new

PI

is

initiated

and

this

causes

the

PI

's

to

exceed

the

window

length

the

analyzer

must

issue

the

lowest-numbered

unissued

PI

again

making

it

unavailable

for

further

packing

Unrolling

Even

when

false

dependencies

and

control

dependencies

are

ignored

parallelism

in

loops

may

be

restricted

because

there

are

true

data

dependencies

between

successive

versions

of

the

loop

index

variables

In

other

words

in

a

FOR

loop

only

one

iteration

of

the

loop

can

be

initiated

every

cycle

since

the

index

variable

can

only

be

incremented

once

per

cycle

It

is

conceivable

that

a

processor

could

initiate

all

iterations

of

a

loop

simultaneously

by

generating

the

entire

set

of

indexes

simultaneously

provided

they

were

related

in

a

simple

way

such

as

a

constant

difference

To

measure

how

such

a

feature

could

affect

parallelism

we

provide

an

option

which

models

this

feature

by

eliminating

the

normal

execution

time

of

any

operation

that

increments

or

decrements

a

register

by

a

constant

Experimental

Models

Used

We

used

models

in

our

study

We

chose

the

models

by

deciding

which

architectural

features

we

believe

are

important

for

a

high-performance

parallel

architecture

to

have

and

created

models

in

which

these

features

were

adjusted

in

order

to

measure

how

the

given

feature

-LRB-

or

lack

thereof

-RRB-

changed

the

amount

of

available

parallelism

The

models

are

summarized

in

Table

In

general

we

chose

a

top-down

approach

From

the

previous

discussion

it

is

clear

that

the

most

ideal

oracle

model

which

we

call

the

Omniscient

Oracle

should

have

the

following

features

ffl

Infinite

processors

ffl

No

barriers

due

to

control

dependences

ffl

Perfect

memory

disambiguation

ffl

Infinite

register

renaming

ffl

Infinite

memory

renaming

ffl

No

window

limitations

We

then

created

models

in

which

one

or

more

features

are

restricted

to

measure

their

importance

Thus

five

additional

oracles

which

retain

the

first

four

features

in

the

list

above

but

are

limited

in

some

other

way

were

added

The

Myopic

Oracle

and

Short

Oracle

measure

the

effect

of

allowing

only

local

parallelism

to

be

used

by

imposing

a

size

limit

of

on

one

of

the

two

scheduling

windows

-LRB-

see

Section

-RRB-

The

Myopic

Oracle

has

a

limit

on

the

pre-scheduling

window

and

the

Short

Oracle

has

a

limit

on

the

post-scheduling

window

The

Omniscient

Myopic

and

Short

Oracles

have

unlimited

memory

renaming

The

other

limited

oracles

measure

what

happens

when

memory

renaming

is

selectively

removed

The

Tree

Oracle

allows

renaming

of

stack

variables

to

measure

the

limits

of

parallelism

exploitable

by

a

machine

using

a

tree

of

stacks

or

some

equivalent

implementation

but

does

not

allow

renaming

in

the

heap

The

Linear

Oracle

retains

the

linear

stack

model

by

not

allowing

stack

elements

to

be

renamed

but

allows

renaming

in

the

heap

The

Frugal

Oracle

has

no

memory

renaming

The

other

machines

do

not

have

perfect

branch

prediction

The

Multithreaded

Machine

has

procedure

separation

but

no

speculation

This

machine

is

so

named

because

its

behavior

models

what

might

occur

on

an

idealized

multithreaded

machine

-LRB-

with

infinite

renaming

-RRB-

executing

a

sequential

benchmark

without

the

algorithm

modifications

that

would

be

necessary

to

achieve

the

oracles

levels

of

performance

The

Speculative

Multithreaded

Machine

has

procedure

separation

and

also

has

branch

and

jump

prediction

capabilities

using

a

separate

table

for

each

as

described

in

Section

The

Smart

Superscalar

has

neither

but

still

has

unlimited

memory/register

renaming

and

disambiguation

The

Stupid

Superscalar

has

of

these

features

The

two

superscalar

machines

can

only

find

parallelism

within

basic

blocks

Finally

we

have

unrolling

versions

-LRB-

see

Section

-RRB-

of

the

Omniscient

Oracle

Speculative

Multithreaded

Machine

and

Multithreaded

Machine

Smoothability

As

highlighted

in

the

introduction

an

important

question

to

ask

about

programs

with

high

degrees

of

parallelism

is

how

evenly

can

this

parallelism

be

distributed

If

a

program

's

parallelism

is

concentrated

in

short

bursts

of

massive

parallelism

separated

by

long

sequential

sections

then

Amdahl

's

Law

will

take

over

and

the

machine

will

need

many

more

processors

than

the

average

to

achieve

the

theoretical

model

is

similar

to

Wall

's

Stupid

model

-LSB-

-RSB-

except

that

Wall

's

model

was

limited

to

processors

and

could

only

schedule

operations

from

within

a

window

of

instructions

These

limitations

however

have

almost

no

impact

on

the

Stupid

model

since

parallelism

is

limited

to

that

within

basic

blocks

n

P

-LRB-

n

-RRB-

P

-LRB-

-RRB-

dP

-LRB-

-RRB-

e

npeak

nmax

P

-LRB-

dP

-LRB-

-RRB-

e

-RRB-

Figure

Standard

smoothability

profile

parallelism

limits

Furthermore

the

utilization

of

these

processors

is

poor

during

the

sequential

sections

of

the

execution

If

the

number

of

processors

is

limited

then

some

operations

will

be

delayed

but

this

delay

will

add

to

the

total

execution

time

only

if

the

delayed

operations

are

in

critical

paths

In

this

section

we

describe

how

our

analyzer

can

measure

this

effect

and

we

define

a

value

called

smoothability

which

quantifies

this

property

Our

analyzer

has

the

option

of

limiting

the

number

of

operations

which

can

be

packed

into

one

PI

This

limitation

puts

an

upper

bound

on

the

maximum

attainable

parallelism

and

may

cause

some

operations

to

be

delayed

-LRB-

scheduled

into

later

PI

's

-RRB-

because

the

PI

's

in

which

they

first

could

be

packed

are

already

full

Because

the

analyzer

's

scheduler

processes

operations

in

the

order

in

which

they

appear

in

the

dynamic

trace

it

is

always

the

later

instructions

which

will

be

delayed

For

a

given

architectural

model

and

a

given

benchmark

program

we

can

define

P

-LRB-

n

-RRB-

the

parallelism

function

which

gives

the

average

parallelism

when

the

width

of

each

PI

is

limited

to

n

operations

-LRB-

i.e.

there

are

n

processors

-RRB-

We

use

P

-LRB-

-RRB-

to

denote

the

parallelism

when

the

number

of

processors

is

infinite

A

plot

of

P

-LRB-

n

-RRB-

will

look

something

like

Figure

This

curve

is

bounded

above

by

two

factors

For

small

values

of

n

P

-LRB-

n

-RRB-

is

bounded

by

n

since

we

ca

n't

have

greater

than

linear

speedup

This

is

represented

by

the

diagonal

dashed

line

in

Figure

Once

n

reaches

P

-LRB-

-RRB-

that

value

becomes

the

new

bound

since

P

-LRB-

n

-RRB-

can

never

be

greater

than

P

-LRB-

-RRB-

as

shown

by

the

horizontal

dashed

line

P

-LRB-

n

-RRB-

is

guaranteed

to

reach

P

-LRB-

-RRB-

if

n

equals

or

exceeds

npeak

the

largest

number

of

operations

packed

into

any

single

PI

i.e.

the

peak

instantaneous

parallelism

because

above

that

point

all

operations

are

executed

immediately

and

there

are

no

delays

Under

the

more

optimistic

models

npeak

may

be

quite

high

since

many

operations

not

dependent

on

previous

operations

will

be

packed

into

the

first

PI

Usually

P

-LRB-

n

-RRB-

will

reach

P

-LRB-

-RRB-

at

some

lower

value

of

n

called

nmax

When

n

is

between

nmax

and

npeak

some

operations

are

delayed

but

the

delays

do

not

affect

critical

paths

so

the

total

number

of

PI

's

does

n't

increase

If

a

program

is

ideally

-LRB-

evenly

-RRB-

distributed

over

time

then

the

number

of

operations

performed

in

every

cycle

is

either

bP

-LRB-

-RRB-

c

or

dP

-LRB-

-RRB-

e

-LRB-

since

P

-LRB-

-RRB-

may

be

non-integral

and

each

PI

has

an

integral

number

of

operations

-RRB-

Thus

dP

-LRB-

-RRB-

e

processors

are

sufficient

to

guarantee

that

every

operation

in

the

trace

is

executed

as

early

as

possible

A

program

that

is

not

as

smooth

however

will

have

-LRB-

with

infinite

processors

-RRB-

some

PI

's

with

more

than

dP

-LRB-

-RRB-

e

operations

With

only

dP

-LRB-

-RRB-

e

processors

some

of

these

operations

will

need

to

be

deferred

until

a

later

PI

in

which

not

so

many

operations

are

packed

If

the

deferred

operation

is

not

on

a

critical

path

and

its

results

are

not

immediately

needed

then

this

delay

wo

n't

increase

the

total

number

of

PI

's

needed

to

execute

the

program

However

if

the

operation

is

in

n

U

-LRB-

n

-RRB-

dP

-LRB-

-RRB-

e

npeak

nmax

U

-LRB-

dP

-LRB-

-RRB-

e

-RRB-

Figure

Standard

processor

utilization

profile

a

critical

path

and

it

is

delayed

by

k

cycles

it

is

possible

that

every

future

operation

dependent

on

that

operation

will

be

delayed

by

k

cycles

increasing

the

total

number

of

PI

's

by

k

Since

P

-LRB-

n

-RRB-

is

bounded

above

in

the

manner

previously

described

we

can

define

another

function

to

represent

how

well

the

processors

are

utilized

compared

to

the

ideal

case

U

-LRB-

n

-RRB-

P

-LRB-

n

-RRB-

n

if

n

P

-LRB-

-RRB-

P

-LRB-

n

-RRB-

P

-LRB-

-RRB-

if

n

P

-LRB-

-RRB-

A

utilization

curve

corresponding

to

the

curve

in

Figure

is

shown

in

Figure

We

define

smoothability

to

be

the

processor

utilization

at

the

critical

value

n

dP

-LRB-

-RRB-

e

i.e.

S

U

-LRB-

dP

-LRB-

-RRB-

e

-RRB-

P

-LRB-

dP

-LRB-

-RRB-

e

-RRB-

P

-LRB-

-RRB-

Our

analyzer

can

keep

a

record

of

how

many

operations

are

in

each

PI

and

can

delay

operations

whenever

a

PI

becomes

full

To

measure

the

smoothability

for

a

given

model

we

first

compute

P

-LRB-

-RRB-

for

that

model

round

it

to

the

next

integer

and

rerun

the

program

with

the

resulting

value

as

the

PI

width

limit

We

compute

smoothability

for

both

the

Omniscient

Oracle

and

the

Speculative

Multithreaded

Machine

We

also

take

parallelism

profiles

for

both

and

record

the

average

number

of

cycles

each

operation

must

be

deferred

This

gives

us

an

idea

whether

the

smoothability

is

mostly

due

to

a

genuinely

even

distribution

of

operations

over

time

or

to

the

ability

to

defer

operations

without

excessively

slowing

down

the

execution

of

critical

paths

Experimental

Results

To

measure

the

parallelism

available

under

the

various

models

a

testbed

was

developed

to

implement

the

trace

analyzer

Since

programs

can

show

significant

differences

in

behavior

when

run

on

short

inputs

and

on

larger

inputs

we

required

our

testbed

to

handle

traces

for

realistic

input

sizes

Also

we

stipulated

that

all

programs

tested

should

run

to

completion

giving

the

analyzer

enough

opportunity

to

extract

parallelism

from

all

parts

of

the

code

Implementation

details

are

provided

in

Appendix

A

We

decided

to

base

the

analysis

on

traces

of

programs

executed

on

Sun

SparcStations

The

Sparc

architecture

was

chosen

for

the

testbed

because

it

is

representative

of

present-day

RISC

processors

Also

the

Sparc

processor

has

a

special

feature

facilitating

procedure

calls

the

partitioning

of

registers

into

sets

which

are

selectively

visible

to

the

processor

via

register

windows

The

mundane

task

of

saving

and

restoring

local

registers

and

passing

parameters

is

much

more

efficient

This

avoids

inflating

the

parallelism

numbers

with

the

pushes

and

pops

which

the

analyzer

would

schedule

in

parallel

Finally

some

of

the

recent

trace-driven

experiments

have

used

MIPS

workstations

and

we

decided

to

confirm

the

generality

of

the

results

by

using

a

different

RISC

processor

We

ran

our

analysis

tool

on

benchmarks

comprising

a

total

of

test

cases

under

each

of

the

models

We

chose

problems

that

we

felt

are

representative

of

the

types

of

computations

that

are

likely

to

be

performed

on

future

high-performance

architectures

We

looked

for

programs

large

enough

to

give

figures

suggestive

of

what

can

be

achieved

with

grand

challenge

problems

without

being

so

large

as

to

overwhelm

our

analyzer

The

benchmarks

consist

of

three

regular

FP-intensive

scientific

programs

written

in

FORTRAN

one

irregular

FP-intensive

FORTRAN

program

and

five

symbolic

applications

written

in

C.

All

are

from

standard

benchmark

test

suites

except

for

a

speech-recognition

program

taken

from

an

actual

industrial

application

The

tests

are

summarized

in

Table

Source

Program

Description

Test

Case

Useful

Call

of

ops

Operations

Dep

FP

Ld

St

DLX

tex

Text

formatting

man.tex

-LRB-

p.

-RRB-

suite

draft.tex

-LRB-

-RRB-

Indust

speech

Speech

recognit

li

Lisp

interpreter

-LRB-

queens

-RRB-

SPEC

espresso

Boolean

bca.in

test

minimization

cps.in

suite

ti.in

tial.in

eqntott

Truth-table

gen.

int

pri

eqn

DLX

spice

Analog

circuit

small

suite

simulation

large

tomcatv

Mesh

N

generation

N

N

SPEC

N

test

doduc

Hydrocode

tiny

suite

simulation

small

ref

fpppp

Quantum

NATOMS

chemistry

NATOMS

Table

Benchmarks

and

test

cases

used

In

cases

where

benchmarks

had

more

than

one

input

case

we

ran

all

available

cases

to

see

how

parallelism

might

vary

for

different

sizes

and

types

of

inputs

We

used

the

standard

input

cases

listed

except

that

we

reduced

the

size

of

the

Queens

problem

run

on

the

Lisp

interpreter

from

to

queens

and

the

number

of

atoms

in

fpppp

from

to

One

of

the

goals

of

our

study

was

to

analyze

complete

program

executions

rather

than

stopping

at

some

point

as

in

some

previous

studies

Our

experiments

confirmed

the

wisdom

of

this

choice

as

some

of

the

programs

showed

long-term

changes

in

parallelism

over

time

By

programming

our

analyzer

to

print

intermediate

results

at

regular

intervals

we

found

for

instance

that

parallelism

of

eqntott

under

the

Multithreaded

model

was

over

after

million

operations

but

dropped

below

after

half

a

billion

operations

Each

of

the

next

three

subsections

discusses

the

observed

effects

of

one

type

of

architectural

limitation

on

parallelism

This

is

followed

by

a

presentation

of

the

smoothability

results

and

a

discussion

of

the

role

of

compiler

optimization

Effects

of

Branch

and

Jump

Prediction

Benchmark

Omniscient

Speculative

Multithreaded

Smart

Stupid

Oracle

Multithreaded

Superscalar

Superscalar

tex

-LRB-

man

-RRB-

tex

-LRB-

draft

-RRB-

speech

li

-LRB-

queens

-RRB-

espresso

-LRB-

bca

-RRB-

espresso

-LRB-

cps

-RRB-

espresso

-LRB-

ti

-RRB-

espresso

-LRB-

tial

-RRB-

eqntott

spice

-LRB-

small

-RRB-

spice

-LRB-

large

-RRB-

tomcatv

-LRB-

N

-RRB-

tomcatv

-LRB-

N

-RRB-

tomcatv

-LRB-

N

-RRB-

tomcatv

-LRB-

N

-RRB-

doduc

-LRB-

tiny

-RRB-

doduc

-LRB-

small

-RRB-

doduc

-LRB-

ref

-RRB-

fpppp

-LRB-

NATOMS

-RRB-

fpppp

-LRB-

NATOMS

-RRB-

Table

A

summary

of

results

A

summary

of

the

parallelism

results

for

base

models

is

given

in

Table

Examining

this

table

we

note

that

the

results

vary

from

surprisingly

high

parallelism

results

for

the

Omniscient

Oracle

to

the

expectedly

poor

parallelism

results

for

the

stupid

superscalar

model

In

the

case

of

the

Omniscient

Oracle

we

find

impressive

parallelism

for

speech

-LRB-

-RRB-

eqntott

-LRB-

-RRB-

tomcatv

-LRB-

-LCB-

-RRB-

and

fpppp

-LRB-

-LCB-

-RRB-

In

addition

all

other

programs

but

spice

and

tex

show

parallelism

results

greater

than

A

comparison

of

the

first

four

models

shows

the

importance

of

branch

and

jump

prediction

The

Speculative

Multithreaded

machine

is

just

like

the

Omniscient

Oracle

except

it

does

not

know

the

outcomes

of

branches

and

indirect

jumps

Therefore

it

must

predict

them

and

sometimes

makes

mistakes

The

Multithreaded

model

is

a

further

restriction

that

has

no

jump

or

branch

prediction

It

does

however

allow

instructions

separated

by

a

conditional

branch

to

execute

concurrently

if

they

are

in

different

procedure

instances

The

Smart

Superscalar

does

n't

even

make

this

allowance

From

Table

we

can

make

the

following

observations

ffl

For

some

programs

like

li

espresso

eqntott

and

tomcatv

the

power

to

predict

jumps

and

branches

perfectly

is

extremely

important

As

expected

for

scientific

programs

that

have

regular

data-independent

behavior

like

doduc

branch

predication

is

much

less

important

ffl

There

is

a

substantial

parallelism

loss

between

the

Speculative

Multithreaded

model

and

the

Multithreaded

model

This

result

indicates

that

for

programs

with

irregular

control

flow

speculative

execution

may

be

very

important

Finite

Scheduling

Windows

In

order

to

study

what

characteristics

of

the

Omniscient

Oracle

give

such

good

parallelism

results

we

performed

a

series

of

experiments

where

we

constrained

other

selected

features

of

the

oracle

In

Table

we

compare

the

parallelism

results

of

the

Omniscient

Oracle

and

Myopic

Oracle

These

two

oracles

are

identical

except

the

Myopic

Oracle

schedules

operations

from

a

window

whose

maximum

length

is

The

comparison

is

quite

startling

and

leads

to

the

following

observations

ffl

For

all

programs

but

spice

the

Myopic

Oracle

performs

significantly

worse

In

many

cases

part

of

the

loss

is

due

to

the

fact

that

the

size

of

the

window

-LRB-

-RRB-

is

an

upper

bound

on

the

maximum

parallelism

However

in

no

case

did

the

parallelism

on

the

Myopic

Oracle

even

come

close

to

this

bound

ffl

For

symbolic

programs

like

eqntott

and

speech

and

numeric

programs

like

fpppp

the

Myopic

Oracle

gives

less

than

of

the

Omniscient

Oracle

's

parallelism

Thus

we

can

conclude

that

parallel

operations

in

these

programs

may

be

separated

by

large

distances

in

the

trace

ffl

For

most

applications

the

Myopic

Oracle

's

performance

on

a

given

program

declined

as

the

problem

size

increased

As

the

problem

size

increases

many

loops

become

longer

pushing

parallel

operations

further

apart

This

effect

is

most

noticable

with

tomcatv

and

fpppp

We

tried

another

variation

of

a

fixed-window

oracle

by

placing

the

after

the

operation

scheduler

The

Short

Oracle

can

see

operations

across

an

infinite

distance

but

can

only

reach

a

short

distance

when

scheduling

them

This

removes

the

upper

bound

on

parallelism

and

the

Short

Oracle

will

always

perform

at

least

as

well

as

the

Myopic

Oracle

However

this

model

still

shows

the

extent

to

which

concurrent

operations

are

separated

in

the

instruction

trace

We

found

ffl

For

all

numerical

programs

except

fpppp

and

for

spice

the

finite

window

limit

has

little

or

no

impact

on

parallelism

This

suggests

that

most

of

the

parallelism

comes

from

loops

which

ca

n't

be

parallelized

so

effectively

if

the

window

comes

before

the

scheduler

ffl

For

symbolic

programs

like

speech

li

and

eqntott

the

Short

Oracle

is

almost

as

bad

as

the

Myopic

Oracle

suggesting

that

most

parallelism

comes

from

widely-separated

locations

in

the

program

and

not

from

within

loops

Benchmark

Omniscient

Myopic

of

Short

of

Oracle

Oracle

Omni

Oracle

Omni

tex

-LRB-

man

-RRB-

tex

-LRB-

draft

-RRB-

speech

li

-LRB-

queens

-RRB-

espresso

-LRB-

bca

-RRB-

espresso

-LRB-

cps

-RRB-

espresso

-LRB-

ti

-RRB-

espresso

-LRB-

tial

-RRB-

eqntott

spice

-LRB-

small

-RRB-

spice

-LRB-

large

-RRB-

tomcatv

-LRB-

N

-RRB-

tomcatv

-LRB-

N

-RRB-

tomcatv

-LRB-

N

-RRB-

tomcatv

-LRB-

N

-RRB-

doduc

-LRB-

tiny

-RRB-

doduc

-LRB-

small

-RRB-

doduc

-LRB-

ref

-RRB-

fpppp

-LRB-

NATOMS

-RRB-

fpppp

-LRB-

NATOMS

-RRB-

Table

The

effects

of

finite

window

sizes

Frugal

Oracles

Having

established

that

window

limitations

are

usually

bad

we

next

experimented

with

frugal

oracles

The

results

of

these

experiments

are

given

in

Table

In

this

table

we

give

the

parallelism

measurements

for

the

Tree

Oracle

-LRB-

stack

renaming

but

no

heap

renaming

-RRB-

the

Linear

Oracle

-LRB-

heap

renaming

but

no

stack

renaming

-RRB-

and

the

truly

Frugal

Oracle

-LRB-

no

stack

or

heap

renaming

-RRB-

For

each

result

we

give

the

percentage

of

parallelism

relative

to

the

Omniscient

Oracle

Studying

these

results

we

can

make

the

following

observations

ffl

Stack

renaming

is

very

important

for

numeric

programs

and

for

symbolic

programs

like

speech

and

eqntott

This

is

probably

due

to

overlapping

of

successive

iterations

that

call

the

same

procedure

in

numeric

programs

and

concurrent

execution

of

separate

procedure

calls

in

symbolic

programs

ffl

Heap

renaming

is

extremely

important

for

li

speech

and

eqntott

irrelevant

to

fpppp

and

almost

irrelevant

to

doduc

In

the

case

of

the

symbolic

programs

this

reflects

the

fact

that

the

programs

are

computing

with

large

dynamically-allocated

data

structures

espresso

and

tomcatv

show

variations

in

performance

with

problem

size

but

in

opposite

directions

ffl

Every

program

shows

marked

loss

of

parallelism

with

the

Frugal

Oracle

Thus

it

appears

that

renaming

of

both

stack-allocated

data

structures

and

heap-allocated

data

structures

is

an

important

characteristic

of

the

Omniscient

Oracle

Usually

one

or

the

other

is

the

limiting

case

for

a

given

application

In

all

cases

the

Frugal

Oracle

performs

as

well

or

almost

as

well

as

the

worst

of

the

Tree

Oracle

and

Linear

Oracle

Benchmark

Omniscient

Tree

of

Linear

of

Frugal

of

Oracle

Oracle

Omni

Oracle

Omni

Oracle

Omni

tex

-LRB-

man

-RRB-

tex

-LRB-

draft

-RRB-

speech

li

-LRB-

queens

-RRB-

espresso

-LRB-

bca

-RRB-

espresso

-LRB-

cps

-RRB-

espresso

-LRB-

ti

-RRB-

espresso

-LRB-

tial

-RRB-

eqntott

spice

-LRB-

small

-RRB-

spice

-LRB-

large

-RRB-

tomcatv

-LRB-

N

-RRB-

tomcatv

-LRB-

N

-RRB-

tomcatv

-LRB-

N

-RRB-

tomcatv

-LRB-

N

-RRB-

doduc

-LRB-

tiny

-RRB-

doduc

-LRB-

small

-RRB-

doduc

-LRB-

ref

-RRB-

fpppp

-LRB-

NATOMS

-RRB-

fpppp

-LRB-

NATOMS

-RRB-

Table

The

effect

of

frugal

use

of

memory

Our

studies

show

however

that

for

irregular

code

memory

renaming

by

itself

produces

little

benefit

Our

experiments

showed

little

difference

between

the

Smart

Superscalar

and

the

Stupid

Superscalar

for

such

applications

-LRB-

refer

to

Table

-RRB-

Thus

to

achieve

further

improvements

we

must

go

to

more

aggressive

models

like

the

Multithreading

and

Speculative

Multithreading

models

Unrolling

Results

We

performed

a

final

set

of

experiments

to

see

how

more

parallelism

could

be

extracted

from

programs

through

static

unrolling

as

described

in

Section

We

reran

the

programs

on

the

Omniscient

Oracle

Speculative

Multithreading

Machine

and

Multithreading

Machine

allowing

the

instructions

that

add

constants

to

registers

to

execute

instantaneously

rather

than

taking

the

usual

one-cycle

delay

The

results

are

in

Table

From

this

we

can

see

ffl

With

the

exception

of

speech

unrolling

had

little

impact

on

parallelism

under

the

Omniscient

Oracle

model

While

statically

unrolling

a

loop

may

allow

all

iterations

to

commence

simultaneously

this

apparently

does

n't

decrease

the

lengths

of

critical

paths

very

much

ffl

Unrolling

was

n't

a

much

greater

benefit

to

the

Multithreaded

Machine

Only

tomcatv

had

more

than

a

improvement

ffl

Unrolling

was

a

significant

benefit

to

the

Speculative

Multithreading

Machine

Many

programs

saw

improvements

of

one-third

and

tomcatv

programs

nearly

doubled

in

speed

This

is

because

the

elimination

of

the

one-cycle

delay

between

successive

loop

iterations

shortens

the

total

time

needed

to

execute

an

entire

loop

Since

the

last

iteration

of

a

loop

is

typically

followed

by

a

mispredicted

branch

which

creates

a

barrier

for

all

succeeding

operations

inside

the

procedure

containing

the

loop

packing

the

loop

in

fewer

PI

's

reduces

the

total

number

of

PI

's

needed

for

the

whole

procedure

Benchmark

Omniscient

Oracle

Speculative

Multithr

Multithreaded

Orig

Unr

Inc.

Orig

Unr

Inc.

Orig

Unr

Inc.

tex

-LRB-

man

-RRB-

tex

-LRB-

draft

-RRB-

speech

li

-LRB-

queens

-RRB-

espresso

-LRB-

bca

-RRB-

espresso

-LRB-

cps

-RRB-

espresso

-LRB-

ti

-RRB-

espresso

-LRB-

tial

-RRB-

eqntott

spice

-LRB-

small

-RRB-

spice

-LRB-

large

-RRB-

tomcatv

-LRB-

N

-RRB-

tomcatv

-LRB-

N

-RRB-

tomcatv

-LRB-

N

-RRB-

tomcatv

-LRB-

N

-RRB-

doduc

-LRB-

tiny

-RRB-

doduc

-LRB-

small

-RRB-

doduc

-LRB-

ref

-RRB-

fpppp

-LRB-

NATOMS

-RRB-

fpppp

-LRB-

NATOMS

-RRB-

Table

The

effects

of

unrolling

Smoothability

Results

In

all

of

the

models

that

we

have

presented

so

far

we

have

assumed

an

infinite

number

of

processors

It

is

therefore

interesting

to

study

the

effect

of

limiting

the

number

of

processors

That

is

if

we

really

had

a

limited

number

of

processors

is

the

parallelism

profile

smooth

enough

to

be

effectively

mapped

to

those

processors

In

Table

we

list

the

parallelism

results

for

the

Omniscient

Oracle

and

the

Speculative

Multithreaded

Machine

For

each

model

we

present

the

smoothability

-LRB-

defined

in

Section

-RRB-

In

all

cases

under

both

models

the

smoothability

is

better

than

We

also

list

the

average

operation

delay

i.e.

the

average

number

of

PI

's

by

which

each

operation

is

delayed

before

it

can

be

executed

-LRB-

The

average

includes

instructions

which

are

executed

as

soon

as

they

are

ready

which

are

counted

as

-RRB-

Low

numbers

as

with

li

and

fpppp

indicate

that

the

application

is

already

fairly

evenly

distributed

in

time

so

not

much

smoothing

is

needed

High

numbers

as

with

doduc

-LRB-

Speculative

Multithreaded

Machine

-RRB-

suggest

that

many

operations

are

being

delayed

a

long

time

before

being

executed

The

high

smoothability

numbers

however

suggest

that

these

delays

do

not

significantly

increase

the

lengths

of

critical

paths

Benchmark

Omni

Smooth

Average

Speculative

Smooth

Average

Oracle

ability

Delay

Multithread

ability

Delay

tex

-LRB-

man

-RRB-

tex

-LRB-

draft

-RRB-

speech

li

-LRB-

queens

-RRB-

espresso

-LRB-

bca

-RRB-

espresso

-LRB-

cps

-RRB-

espresso

-LRB-

ti

-RRB-

espresso

-LRB-

tial

-RRB-

eqntott

spice

-LRB-

small

-RRB-

spice

-LRB-

large

-RRB-

tomcatv

-LRB-

N

-RRB-

tomcatv

-LRB-

N

-RRB-

tomcatv

-LRB-

N

-RRB-

tomcatv

-LRB-

N

-RRB-

doduc

-LRB-

tiny

-RRB-

doduc

-LRB-

small

-RRB-

doduc

-LRB-

ref

-RRB-

fpppp

-LRB-

NATOMS

-RRB-

fpppp

-LRB-

NATOMS

-RRB-

Table

The

effect

of

finite

processors

Compiler

Optimization

We

wish

to

make

one

final

point

concerning

optimization

All

programs

were

compiled

at

maximum

optimization

except

for

three

of

the

FORTRAN

programs

In

these

cases

optimization

level

had

a

significant

impact

on

program

parallelism

Optimization

level

-LCB-

was

used

on

tomcatv

because

that

yielded

the

best

results

for

executing

the

sequential

program

in

terms

of

total

instruction

count

and

the

numbers

of

memory

accesses

However

what

is

more

significant

is

that

the

Omniscient

Oracle

parallelism

dropped

from

to

when

optimization

was

increased

to

This

is

surprising

since

the

Omniscient

Oracle

only

accounts

for

true

data

dependencies

and

we

would

not

expect

automatic

compiler

optimizations

to

change

the

overall

pattern

of

data

dependences

in

a

program

We

presently

have

no

explanation

for

this

anomaly

Two

other

FORTRAN

programs

were

compiled

at

sub-maximum

levels

because

the

increase

in

parallelism

was

far

greater

than

the

sequential

instruction

count

When

-LCB-

was

used

for

spice

the

total

number

of

PI

's

decreased

by

even

though

there

were

more

sequential

instructions

Using

-LCB-

for

doduc

reduced

total

PI

's

by

more

than

a

factor

of

even

though

there

were

more

sequential

instructions

In

these

two

cases

one

may

wish

to

multiply

the

parallelism

figures

reported

by

the

reciprocal

factors

-LRB-

spice

-RRB-

and

-LRB-

doduc

-RRB-

for

a

fair

comparison

with

the

optimized

sequential

program

Previous

Studies

Most

previous

studies

on

the

limits

of

instruction-level

parallelism

have

taken

one

of

two

approaches

One

approach

is

to

analyze

the

selected

benchmark

at

either

the

source-code

or

object-code

level

usually

with

a

special

interpreter

that

is

based

on

a

certain

parallel

machine

model

The

other

approach

used

in

our

study

is

to

schedule

machine

instructions

from

a

trace

generated

by

an

actual

execution

of

the

object

code

The

following

presents

some

of

the

previous

work

along

both

paths

An

early

experimental

tool

designed

by

Kuck

et

al

-LSB-

-RSB-

compiled

small

Fortran

programs

to

run

on

an

abstract

parallel

machine

By

recognizing

parallelism

at

the

source

code

level

-LRB-

e.g.

parallel

iterations

of

DO

loops

-RRB-

and

by

aggressively

reordering

complex

expressions

they

were

able

to

speed

up

most

programs

by

a

factor

of

and

speed

up

some

by

as

much

as

Tjaden

and

Flynn

built

a

simulator

based

on

the

superscalar

model

-LSB-

-RSB-

Their

simulator

could

execute

ordinary

IBM

machine

code

but

attempted

to

reschedule

operations

dynamically

using

a

look-ahead

window

of

between

and

operations

They

obtained

speedups

of

up

to

on

a

suite

of

library

routines

Both

studies

got

modest

results

because

they

only

looked

for

parallelism

within

small

blocks

and

could

not

execute

independent

instructions

from

separate

blocks

These

blocks

are

typically

separated

by

conditional

branches

and

if

the

decision

whether

to

execute

one

block

is

controlled

by

the

outcome

of

a

conditional

branch

in

another

block

the

first

block

ca

n't

begin

execution

until

the

branch

has

been

resolved

Nicolau

and

Fisher

wrote

an

interpreter

for

intermediate

code

generated

by

a

compiler

front-end

and

a

tool

to

analyze

the

instruction

trace

generated

by

the

interpreter

-LSB-

-RSB-

They

addressed

the

shortcomings

of

the

previous

studies

by

allowing

operations

from

many

parts

of

the

program

to

execute

concurrently

They

introduced

the

oracle

model

which

they

added

to

their

analysis

tool

The

analyzer

also

ignored

any

dependency

which

was

not

a

true

data

dependency

They

were

able

to

obtain

respectable

speedups

from

small

scientific

routines

The

previous

work

discussed

so

far

only

studied

small

programs

primarily

small

numerical

routines

To

give

a

better

idea

of

how

large

programs

will

fare

on

parallel

machines

there

have

been

several

recent

studies

on

this

subjects

-LSB-

BYP

-RSB-

Below

we

briefly

survey

work

which

are

most

relevant

to

ours

Kumar

-LSB-

-RSB-

developed

a

tool

that

automatically

added

statistics-gathering

statements

to

FORTRAN

source-code

By

running

the

modified

FORTRAN

programs

he

was

able

to

measure

parallelism

figures

for

scientific

applications

The

tool

also

performed

control-dependence

analysis

so

that

independent

FORTRAN

statements

separated

by

barriers

could

still

run

concurrently

Kunar

found

the

potential

for

high

levels

of

parallelism

in

regular

numerical

code

Wall

-LSB-

-RSB-

analyzed

benchmarks

-LRB-

SPEC

programs

plus

some

real

programs

and

toy

benchmarks

-RRB-

in

which

the

number

of

executed

operations

ranged

from

million

to

billion

He

developed

a

series

of

models

representing

various

levels

of

optimism

about

what

the

hardware

and

compiler

could

do

He

then

ran

the

benchmarks

on

a

MIPS

processor

and

analyzed

the

instruction

traces

in

a

manner

similar

to

Tjaden

and

Flynn

and

Nicolau

and

Fisher

Even

under

his

perfect

model

with

oracle-like

branch

prediction

parallelism

rarely

exceeded

This

is

because

his

schedule

used

a

finite

-LRB-

-RRB-

input

window

from

which

to

fill

parallel

instructions

and

because

he

did

not

allow

for

memory

renaming

Recently

Wilson

and

Lam

have

reported

their

study

of

the

way

control

flow

limits

parallelism

They

have

demonstrated

that

substantially

higher

parallelism

can

be

achieved

by

relaxing

the

constraints

imposed

by

control

flow

on

parallelism

using

control-dependence

analysis

executing

multiple

flows

of

control

simultaneously

and

speculative

execution

-LSB-

-RSB-

They

used

six

SPEC

programs

and

other

programs

as

the

test

suite

and

each

program

was

simulated

for

million

instructions

Conclusions

In

this

paper

we

have

presented

a

study

of

the

instruction

level

parallelism

and

provided

some

answers

to

questions

regarding

the

limits

of

parallelism

and

its

smoothability

We

have

reported

on

the

construction

of

a

testbed

that

was

used

to

work

with

a

wide

variety

of

architecture

models

This

testbed

allowed

us

to

experiment

with

both

standard

benchmark

programs

and

a

real

speech

application

The

testbed

was

implemented

in

such

a

way

as

to

allow

us

to

perform

experiments

on

complete

traces

for

reasonably

large

input

data

sets

By

examining

an

Omniscient

Oracle

and

specializations

of

this

oracle

we

were

able

to

study

the

key

features

that

are

important

in

finding

large

amounts

of

fine-grain

parallelism

These

results

showed

that

restricting

the

oracle

by

making

it

myopic

-LRB-

finite

instruction

window

-RRB-

or

frugal

-LRB-

no

memory

renaming

-RRB-

resulted

in

reducing

the

parallelism

dramatically

As

part

of

the

Frugal

Oracle

test

we

looked

at

the

effect

of

allowing

only

stack

renaming

and

showed

that

this

is

a

useful

feature

for

many

programs

However

for

other

programs

stack

renaming

is

not

enough

and

heap

renaming

is

also

required

This

indicates

that

further

work

at

both

the

compiler

and

architecture

level

must

be

undertaken

to

remove

the

memory

reuse

bottleneck

We

also

showed

that

the

power

of

branch

and

jump

prediction

is

very

important

especially

for

programs

with

irregular

control

flow

This

confirms

the

results

of

previous

experiments

In

particular

we

found

disappointingly

low

amounts

of

parallelism

-LRB-

-RRB-

for

the

irregular

programs

under

the

Multithreaded

model

Other

studies

have

seen

the

same

results

even

when

they

performed

control-dependence

analysis

-LRB-

see

Section

-RRB-

to

increase

parallelism

within

a

procedure

However

some

applications

like

tex

intuitively

seem

to

have

much

more

potential

parallelism

than

found

by

our

Multithreaded

model

-LRB-

-RRB-

or

Lam

and

Wilson

's

model

with

control-dependence

analysis

-LRB-

-RRB-

This

has

important

implications

for

future

directions

in

architecture

It

is

sometimes

believed

that

to

exploit

maximum

parallelism

we

only

need

to

rewrite

existing

algorithms

in

a

better

language

such

as

a

functional

language

so

that

an

efficient

compiler

can

extract

all

the

available

parallelism

without

being

burdened

by

such

sequential

artifacts

as

memory

reuse

But

the

Multithreaded

model

gets

poor

results

even

though

it

eliminates

false

data

dependencies

and

represents

what

could

be

achieved

by

an

efficient

dataflow

machine

running

well-compiled

programs

This

would

suggest

that

in

many

cases

large-scale

parallelism

will

not

be

achieved

merely

by

rewriting

existing

imperative-language

programs

in

different

languages

They

must

be

redesigned

completely

starting

at

the

algorithmic

level

Whether

this

will

require

the

programmer

to

write

programs

with

an

explicit

parallel

machine

model

in

mind

or

to

use

a

more

abstract

programming

paradigm

such

as

logic

programming

is

an

interesting

but

entirely

different

area

of

research

If

such

changes

are

impractical

then

one

should

consider

some

sort

of

speculation

for

programs

with

irregular

control

flow

We

showed

that

this

can

significantly

improve

program

performance

We

also

showed

that

if

such

speculation

is

used

simultaneous

initiation

of

multiple

loop

iterations

usually

can

boost

program

execution

speed

even

more

Smoothability

has

been

demonstrated

to

be

an

interesting

measure

of

the

program

parallelism

We

reported

the

observation

that

all

programs

under

test

demonstrated

smoothability

better

than

Furthermore

different

programs

may

exhibit

different

smoothabilities

Thus

there

is

good

hope

for

mapping

the

available

parallelism

onto

finite

resources

Our

analyzer

has

proven

to

be

a

valuable

experimental

tool

and

many

useful

extensions

are

possible

For

future

work

we

plan

to

extend

our

existing

testbed

in

the

following

directions

ffl

Add

control-dependence

analysis

-LRB-

see

Section

to

extract

more

parallelism

from

the

nonoracle

models

ffl

Add

a

memory

latency

model

to

the

various

experimental

architectures

ffl

Connect

the

testbed

with

our

compiler

so

that

we

can

add

measurements

relative

to

the

real

alias

analyses

that

can

be

performed

by

a

parallelizing

compiler

ffl

Add

more

programs

to

the

test

suite

ffl

Experiment

with

various

multithreading

scheduling

strategies

References

-LSB-

A

-RSB-

Robert

Alverson

et

al.

The

Tera

computer

system

In

Proceedings

of

the

Annual

International

Symposium

on

Computer

Architecture

pages

-LCB-

Seattle

Washington

May

-LCB-

IEEE

Computer

Society

and

ACM

SIGARCH

Also

in

Computer

Architecture

News

-LRB-

-RRB-

June

-LSB-

-RSB-

A.

Agarwal

B.

H.

Lim

D.

Kranz

and

J.

Kubiatowicz

APRIL

A

processor

architecture

for

multiprocessing

In

Proceedings

of

the

Annual

International

Symposium

on

Computer

Architecture

pages

-LCB-

Seattle

Washington

May

-LCB-

IEEE

Computer

Society

and

ACM

SIGARCH

Also

in

Computer

Architecture

News

-LRB-

-RRB-

June

-LSB-

-RSB-

T.

M.

Austin

and

G.

S.

Sohi

Dynamic

dependence

analysis

of

ordinary

programs

In

Proceedings

of

the

Annual

International

Symposium

on

Computer

Architecture

pages

-LCB-

Gold

Coast

Australia

May

-LCB-

ACM

SIGARCH

and

IEEE

Computer

Society

Also

in

Computer

Architecture

News

-LRB-

-RRB-

May

-LSB-

BYP

-RSB-

M.

Butler

T-Y

Yeh

Y.

Patt

M.

Alsup

H.

Scales

and

M.

Shebanow

Single

instruction

stream

parallelism

is

greater

than

two

In

Proceedings

of

the

Annual

International

Symposium

on

Computer

Architecture

pages

-LCB-

Toronto

Ontario

May

-LCB-

ACM

SIGARCH

and

IEEE

Computer

Society

Also

in

Computer

Architecture

News

-LRB-

-RRB-

May

-LSB-

CSS

-RSB-

David

E.

Culler

Anurag

Sah

Klaus

Eric

Schauser

Thorsten

von

Eiken

and

John

Wawrzynek

Fine-grain

parallelism

with

minimal

hardware

support

A

compiler-controlled

threaded

abstract

machine

In

Proceedings

of

the

Fourth

International

Conference

on

Architectural

Support

for

Programming

Languages

and

Operating

Systems

pages

-LCB-

Santa

Clara

California

April

-LCB-

ACM

SIGARCH

SIGPLAN

SIGOPS

and

the

IEEE

Computer

Society

Also

in

Computer

Architecture

News

-LRB-

-RRB-

April

Operating

Systems

Review

April

SIGPLAN

Notices

-LRB-

-RRB-

April

-LSB-

DCC

-RSB-

W.

J.

Dally

L.

Chao

A.

Chien

S.

Hassoun

W.

Horwat

J.

Kaplan

P.

Song

B.

Totty

and

W.

Wills

Architecture

of

a

message-driven

processor

In

Proceedings

of

the

Annual

International

Symposium

on

Computer

Architecture

pages

-LCB-

Pittsburgh

Pennsylvania

June

-LCB-

IEEE

Computer

Society

and

ACM

SIGARCH

Also

in

Computer

Architecture

News

-LRB-

-RRB-

June

-LSB-

-RSB-

R.

H.

Halstead

Jr

and

T.

Fujita

MASA

A

multithreaded

processor

architecture

for

parallel

symbolic

computing

In

Proceedings

of

the

Annual

International

Symposium

on

Computer

Architecture

pages

-LCB-

Honolulu

Hawaii

May

-LCB-

June

IEEE

Computer

Society

and

ACM

SIGARCH

Also

in

Computer

Architecture

News

-LRB-

-RRB-

May

-LSB-

-RSB-

N.

P.

Jouppi

and

D.

W.

Wall

Available

instruction-level

parallelism

for

superscalar

and

superpipelined

machines

In

Proceedings

of

the

Third

International

Conference

on

Architectural

Support

for

Programming

Languages

and

Operating

Systems

pages

-LCB-

Boston

Massachusetts

April

-LCB-

ACM

SIGARCH

SIGPLAN

SIGOPS

and

the

IEEE

Computer

Society

Also

in

Computer

Architecture

News

-LRB-

-RRB-

April

Operating

Systems

Review

April

SIGPLAN

Notices

May

-LSB-

-RSB-

D.

J.

Kuck

Y.

Muraoka

and

S.

C.

Chen

On

the

number

of

operations

simultaneously

executable

in

FORTRAN-like

programs

and

their

resulting

speed-up

IEEE

Transactions

on

Computers

-LRB-

-RRB-

-LCB-

December

-LSB-

-RSB-

J.

T.

Kuehn

and

B.

J.

Smith

The

horizon

supercomputing

system

Architecture

and

software

In

Proceedings

of

Supercomputing

pages

-LCB-

Orlando

Florida

November

IEEE

Computer

Society

and

ACM

SIGARCH

-LSB-

-RSB-

M.

Kumar

Measuring

parallelism

in

computation-intensive

scientific/engineering

applications

IEEE

Transactions

on

Computers

-LRB-

-RRB-

-LCB-

September

-LSB-

-RSB-

M.

S.

Lam

and

R.

P.

Wilson

Limits

of

control

flow

on

parallelism

In

Proceedings

of

the

Annual

International

Symposium

on

Computer

Architecture

pages

-LCB-

Gold

Coast

Australia

May

-LCB-

ACM

SIGARCH

and

IEEE

Computer

Society

Also

in

Computer

Architecture

News

-LRB-

-RRB-

May

-LSB-

-RSB-

R.

Nikhil

and

Arvind

Can

dataflow

subsume

von

Neumann

computing

In

Proceedings

of

the

Annual

International

Symposium

on

Computer

Architecture

pages

-LCB-

Jerusalem

Israel

May

-LCB-

June

IEEE

Computer

Society

and

ACM

SIGARCH

Also

in

Computer

Architecture

News

-LRB-

-RRB-

June

-LSB-

-RSB-

P.

R.

Nuth

and

W.

J.

Dally

A

mechanism

for

efficient

context

switching

Proceedings

of

the

IEEE

International

Conference

on

Computer

Design

-LSB-

-RSB-

Alexandru

Nicolau

and

J.

A.

Fisher

Measuring

the

parallelism

available

for

very

long

instruction

word

architectures

IEEE

Transactions

on

Computers

-LRB-

-RRB-

-LCB-

November

-LSB-

-RSB-

G.

S.

Tjaden

and

M.

J.

Flynn

Detection

and

parallel

execution

of

independent

instructions

IEEE

Transactions

on

Computers

-LRB-

-RRB-

-LCB-

October

-LSB-

-RSB-

D.

W.

Wall

Limits

of

instruction-level

parallelism

In

Proceedings

of

the

Fourth

International

Conference

on

Architectural

Support

for

Programming

Languages

and

Operating

Systems

pages

-LCB-

Santa

Clara

California

April

-LCB-

ACM

SIGARCH

SIGPLAN

SIGOPS

and

the

IEEE

Computer

Society

Also

in

Computer

Architecture

News

-LRB-

-RRB-

April

Operating

Systems

Review

April

SIGPLAN

Notices

-LRB-

-RRB-

April

-LSB-

-RSB-

W.

D.

Weber

and

A.

Gupta

Exploring

the

benefits

of

multiple

contexts

in

a

multiprocessor

architecture

Preliminary

results

In

Proceedings

of

the

Annual

International

Symposium

on

Computer

Architecture

pages

-LCB-

Jerusalem

Israel

May

-LCB-

June

IEEE

Computer

Society

and

ACM

SIGARCH

Also

in

Computer

Architecture

News

-LRB-

-RRB-

June

A

Experimental

Testbed

The

analyzer

consists

of

two

parts

The

first

part

the

decoder

takes

the

stream

output

from

an

existing

trace

generator

and

decodes

the

opcode

and

other

information

performing

basic

block

flow

analysis

and

deriving

the

relevant

dependence

data

The

second

part

the

scheduler

packs

the

operations

into

PI

's

based

on

this

data

The

scheduler

can

run

multiple

models

simultaneously

provided

there

is

enough

memory

Section

talks

about

building

a

dependence

graph

for

the

entire

program

Such

an

undertaking

would

be

out

of

the

question

for

benchmarks

involving

billions

of

operations

Fortunately

since

the

trace

is

generated

by

a

sequential

thread

of

control

it

is

only

necessary

to

store

the

dependence

information

relative

to

the

last

access

of

each

storage

location

Thus

the

amount

of

memory

needed

to

record

and

compute

dependences

is

proportional

to

the

total

number

of

locations

used

not

the

total

number

of

times

they

are

used

A

table

records

the

earliest

PI

in

which

a

given

storage

location

can

be

read

and

written

The

following

shows

what

the

table

would

look

like

after

the

execution

of

the

branch

at

the

end

of

the

first

iteration

the

left

half

of

Figure

Note

that

integer

registers

-LCB-

are

defined

prior

to

entering

this

loop

Register

Read

Write

Int

register

i

i

Int

register

i

Int

register

i

Int

register

i

Int

cond

codes

i

i

FP

register

i

i

FP

register

i

i

FP

register

i

i

When

the

second

occurrence

of

is

encountered

in

the

trace

the

decoder

will

see

that

it

is

dependent

on

writing

and

reading

and

the

memory

location

read

by

the

load

The

maximum

of

all

the

corresponding

entries

in

the

table

is

i

If

the

branch

were

predicted

correctly

then

the

operation

could

be

packed

into

PI

i

Without

branch

prediction

the

operation

would

have

to

go

into

PI

i

because

the

scheduler

also

keeps

track

of

the

most

recent

barrier

Once

the

operation

has

been

packed

the

read

and

write

entries

for

are

updated

with

either

i

-LRB-

branch

predicted

-RRB-

or

i

-LRB-

branch

not

predicted

-RRB-

If

there

is

no

memory

disambiguation

then

a

single

entry

in

the

table

represents

all

memory

accesses

Otherwise

there

must

be

a

separate

entry

for

each

unit

of

memory

This

is

done

using

a

hash

table

of

linked

lists

of

tables

representing

individual

pages

of

memory

Renaming

for

both

register

and

memory

use

is

done

by

ignoring

the

write

component

of

the

table

entries

For

models

with

partial

memory

renaming

-LRB-

e.g.

the

Tree

Oracle

-RRB-

the

decoder

uses

the

memory

address

to

determine

if

it

is

in

the

stack

or

the

heap

The

scheduler

keeps

track

of

the

most

recent

barrier

encountered

unless

an

oracle

is

used

If

non-oracle

branch

prediction

is

used

the

branching

hash

table

is

used

to

derive

the

predicted

branch

outcome

which

is

compared

to

the

actual

outcome

revealed

in

the

trace

If

a

misprediction

results

the

barrier

is

updated

If

procedure

separation

is

used

then

barriers

are

kept

on

a

stack

When

a

procedure

is

called

a

new

barrier

is

allocated

on

the

stack

and

starts

with

the

barrier

of

the

calling

procedure

When

a

procedure

is

exited

its

barrier

is

discarded

and

the

calling

procedure

's

barrier

again

becomes

the

current

barrier

Finite

processor

and

window

length

limits

require

keeping

a

table

of

how

many

operations

are

packed

into

each

PI

Because

this

table

can

become

quite

large

it

was

necessary

to

perform

optimizations

such

as

freeing

up

space

whenever

a

long

contiguous

series

of

PI

's

is

filled

to

the

limits

If

a

finite

window

length

is

used

as

in

the

Myopic

Oracle

then

an

additional

barrier

is

needed

to

mark

the

first

unissued

PI

The

analyzer

frees

up

storage

as

this

barrier

moves

forward

computer architecture modeling
instruction-level parallelism
parallel processing