Peter A. Heeman

Associate Research Professor
Program Director
Center for Spoken Language Understanding
Computer Science and Electrical Engineering

Modeling Spontaneous Speech
Dialog Management

Entire List


Fall 2014: CS533 Automata and Formal Languages
Winter 2016: CS550 Spoken Dialogue Systems
Winter 2015: CS560 Artificial Intelligence


Contact Information

Gaines Hall, Room 40AA (inside of Office 40A)

503 346-3755 (phone)
heemanp at ohsu edu (email)

Mailing Address:
Professor Peter Heeman
Center for Spoken Language Understanding
Oregon Health & Science University
3181 SW Sam Jackson Park Rd., GH40
Portland, OR 97239-3098


Peter Heeman's research interests focus around the theme of spoken dialogue processing. Spoken dialogue interfaces to computer systems have the advantage that they can let people use the communication style that they are accustomed for interacting with a computer system. However, there are many barriers standing in our way.

Modeling Spontaneous Speech

In spontaneous speech, speakers will often utter more than one contribution during their turn of speaking. Unlike written text, there are no explicit punctuation marks that delimit one utterance from the next. Furthermore, due to the online nature of spontaneous speech, speakers sometimes need to revise what they have just said, by making a speech repair. Hence, a first step in understanding spontaneous speech is to segment the speech into distinct utterances and determine the speaker's intended contributions by resolving any speech repairs that might have occurred.

Segmenting speech into utterances and resolving speech repairs is intimately intertwined with the speech recognition task of determining what words the speaker is speaking. Speech repairs and utterance boundaries cause disruptions in the local context, both acoustic and in predicting the next word given the previous words. We have proposed that speech repairs and utterance boundaries (as well as part-of-speech tags and discourse markers) be resolved during the speech recognition task. Towards this end, we have become interested in language models that can be used during speech recognition to help the speech recognizer to prune alternative acoustic hypothesis, and augmenting them to take into account phenomena of spontaneous speech. This work centers around using machine learning techniques, such as decision trees, to learn spontaneous speech phenomena.

Dialogue Management

Dr. Heeman is also interested in dialogue management. During a dialogue, conversants must attune themselves towards whether they are being understood and whether they understand their partner, as well as whether they are reaching agreement. Current spoken dialogue systems tend to ignore this facet of conversation, and blindly assume that the conversation is proceeding without difficulty. But if difficulties (misunderstandings or nonunderstandings) arise, the system will have problems following the dialogue, leading to user frustration. In order to make dialogue systems more user-friendly and easier to use, a dialogue system must participate and collaborate in grounding both its own utterances and the utterances of the user. Peter Heeman has built a computational model of how participants collaborate in conversation.

In understanding the user's contributions to the dialogue, the dialogue system needs to make use of the discourse state to narrow down the possible interpretations. This usage of discourse knowledge needs to be applied as early on processing as possible, even during the task of speech recognition. Peter Heeman's prior work in incorporating modeling speech repairs and utterance segmentation is a first step towards incorporating more knowledge into the speech recognition task and in enlarging this task to incorporate processing that typically is done after recognition. Peter's work in automatically identifying discourse markers, words such as ``so'' and ``but'' that relate the current utterance to the discourse state, also shows the potential of incorporating discourse processing into the speech recognition problem of language modeling.


Fall 1997 CSE580 Dialogue
Winter 1998 CSE560 Sympolic Approaches to Artificial Intelligence
Fall 1998 CSE560 Artificial Intelligence
Fall 1999 CSE560 Artificial Intelligence
Spring 2000 CSE580 Statistical Natural Language Processing
Fall 2000 CSE560 Artificial Intelligence
Summer 2001 CSE580 Spoken Language Systems
Fall 2001 CSE560 Artificial Intelligence


Current Students
Rebecca Lunsford Ph.D. student
Ethan Selfridge Ph.D. student
Former Students
Fan Yang Ph.D. 2008, now at Nuance.
Michael English M.Sc. 2005, now at Google.
Jonathan Shaw M.Sc. 2001, Ph.D. student at the University of Rochester
Susan Strayer M.Sc. 2001
Karen Ward Ph.D. 2001 (on-campus adviser), now an Associate Professor at University of Portland
Xintian Wu Ph.D. 2000 (on-campus advisor), now at Intel
Gerardo Pirela M.Sc. 1999, now a professor at University of Zulia State in Venezuela
Justin Denney M.Sc. 1998, now at Microsoft

Selected Publications

A complete list of publications are located here.