Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing
---
paper_title: Talking and Testing: Discourse approaches to the assessment of oral proficiency

paper_content:

1. Preface (by Celce-Murcia, Marianne) 2. Language Proficiency Interviews: A Discourse Approach (by He, Agnes Weiyun) 3. 1. Language Proficiency Interviews and Conversation 4. Re-analyzing the OPI: How Much Does It Look Like Natural Conversation? (by Johnson, Marysia) 5. Evaluating Learner Interactional Skills: Conversation at the Micro Level (by Riggenbach, Heidi) 6. What Happens When There's No One to Talk to? Spanish Foreign Language Discourse in Simulated Oral Proficiency Interviews (by Koike, Dale A.) 7. 2. Turns and Sequences in Language Proficiency Interviews 8. Answering Questions in LPIs: A Case Study (by He, Agnes Weiyun) 9. Framing the Language Proficiency Interview as a Speech Event: Native and Non-Native Speaker's Questions (by Moder, Carol Lynn) 10. Miscommunication in Language Proficiency Interviews of First-Year German Students: A Comparison with Natural Conversation (by Egbert, Maria M.) 11. Knowledge and Communication in Language Proficiency Interviews 12. Knowledge Structures in Oral Proficiency Interviews for International Teaching Assistants (by Mohan, Bernard A.) 13. The Use of Communication Strategies in Language Proficiency Interviews (by Yoshida-Morise, Yumiko) 14. Meaning Negociation in the Hungarian Oral Proficiency Examination of English (by Katona, Lucy) 15. 3. Language Proficiency Interviews as Cross-Cultural Encounters 16. Maintaining American Face in the Korean Oral Exam: Reflections on the Power of cross-Cultural Context (by Davies, Catherine Evans) 17. Confirmation Sequences as Intercational Resources in Korean Language Proficiency Interviews (by Kim, Kyu-hyun) 18. Divergent Frame Interpretations in Oral Proficiency Interview Interaction (by Ross, Steven) 19. "Let Them Eat Cake!" or how to Avoid Losing Your Head in Cross-Cultural Conversations (by Young, Richard) 20. Index of Names 21. Index of Topics
---
paper_title: Assessing second language pragmatics: An overview and introductions

paper_content:

Pragmatics is a key domain in language assessment. For more than two decades, advances have been made in conceptualizing the domain, developing assessment instruments, and applying current methods of data treatment to the analysis of test performance. This book, the first edited volume on the topic, brings together empirical studies on a range of well-established and innovative strategies for the assessment of pragmatic ability in a second language. In this introductory chapter, we will first offer an overview of key concepts, situate theoretical models of pragmatic competence within the larger frameworks of communicative language ability and interactional competence, and consider the relationship between pragmatics and language testing. We will then introduce the chapters, organized into two Parts. The chapters assembled in Part I investigate assessment instruments and practices for a variety of assessment constructs, purposes, and contexts, guided by different theoretical outlooks on pragmatics. Part II comprises studies of interaction in different forms of oral proficiency interview, conducted from the perspective of conversation analysis.
---
paper_title: The Influence of Second Language Experience and Accent Familiarity on Oral Proficiency Rating: A Qualitative Investigation

paper_content:

This study investigates whether raters' knowledge of test takers' first language (L1) affects how the raters orient themselves to the task of rating oral speech. The authors qualitatively investigated the effects of accent familiarity on raters' score assignment processes. Twenty-six trained raters with a second language of Mandarin Chinese, Korean, or Spanish rated English speech samples from 72 English language learners from three L1 backgrounds (Mandarin Chinese, Korean, and Spanish). Afterward, raters watched videos of themselves rating and discussed their rating processes. Analyses of data, assisted by the qualitative analysis software package QSR NVivo 8, revealed that some raters attended to and were influenced by test-taker accent and L1. Raters who were heritage language learners discussed how their personal heritage status influenced how they thought about the speech samples from speakers whose L1 matched their heritage language. Although sensitivity to test-taker accents seemed to occur naturally in the rating process, findings suggest that when raters have learned or know, to varying degrees, the test takers' L1, they tend to orient themselves to the speech in a biased way, compromising test reliability.
---
paper_title: Evaluation of the Usefulness of the Versant for English Test: A Response

paper_content:

This article responds to a critique of Ordinate Corporation's Versant for English test (formerly PhonePass and SET-10). The critique (Chun, 2006) purports to apply the language task framework of Bachman and Palmer (1996) and the Test Fairness framework of Kunnan (2004) as guides for an analysis. However, the analysis fails to apply either framework appropriately, leading to a misguided conclusion. Here, we present readers with a more balanced analysis by clarifying the issues that the critic has confused, particularly with regard to the authenticity and construct of the test tasks. Finally, we suggest that a correct application of the usefulness framework of Bachman and Palmer (1996) shows Versant for English to be a useful test in many contexts.
---
paper_title: Analysing Learner Language

paper_content:

This book provides a clear and accessible introduction to the main methods of analysing samples of learner language. It examines the theoretical and research bases for the different methods, and thereby serves as an introduction to the field of second language acquisition research. The book is intended for students on postgraduate courses in TESOL or Applied Linguistics, or for teachers wishing to do action research to find out more about how learners learn language.
---
paper_title: Language Assessment: Principles and Classroom Practices

paper_content:

Preface 1 Testing, Assessing, and Teaching What Is a Test? Assessment and Teaching Informal and Formal Assessment Formative and Summative Assessment Norm-Referenced and Criterion-Referenced Test Approaches to Language Testing: A Brief History Discrete-point and Integrative Testing Communicative Language Testing Performance-Based Assessment Current Issues in Classroom Testing New Views on Intelligence Traditional and "Alternative" Assessment Computer-Based Testing Exercises For Your Further Reading 2 Principles of Language Assessment Practicality Reliability Student-Related Reliability Rater Reliability Test Administration Reliability Test Reliability Validity Content-Related Evidence Criterion-Related Evidence Construct-Related Evidence Consequential Validity Face Validity Authenticity Washback Applying Principles to the Evaluation of Classroom Tests 1. Are the test procedures practical? 2. Is the test reliable? 3. Does the procedure demonstrate content validity? 4. Is the procedure face valid and "biased for best"? 5. Are the test tasks as authentic as possible? 6. Does the test offer beneficial washback to the learner? Exercises For Your Further Reading 3 Designing Classroom Language Tests Test Types Language Aptitude Tests Proficiency Tests Placement Test Diagnostic Tests Achievement Tests Some Practical Steps to Test Construction Assessing Clear, Unambiguous Objectives Drawing Up Test Specifications Devising Test Tasks Designing Multiple-Choice Test Items 1. Design each item to measure a specific objective 2. State Both stem and options as simply and directly as possible 3. Make certain that the intended answer is clearly the only correct one 4. Use item indices to accept, discard, or revise items Scoring, Grading, and Giving Feedback Scoring Grading Giving Feedback Exercises For Your Further Reading 4 Standardized Testing What Is Standardization? Advantages and Disadvantages of Standardized Tests Developing a Standardized Test 1. Determine the purpose and objectives of the test 2. Design test specifications 3. Design, select, and arrange test tasks/items 4. Make appropriate evaluations of different kinds of items 5. Specify scoring procedures and reporting formats 6. Perform ongoing construct validation studies Standardized Language Proficiency Testing Four Standardized Language Proficiency Tests Test of English as a Foreign Language (TOEFL(R)) Michigan English Language Assessment Battery (MELAB) International English Language Testing System (IELTS) Test of English for International Communication (TOEIC(R)) Exercises For Your Further Reading Appendix to Chapter 4: Commercial Proficiency Tests: Sample Items and Tasks Test of English as a Foreign Language (TOEFL(R)) Michigan English Language Assessment Battery (MELAB) International English Language Testing System (IELTS) Test of English for International Communication (TOEIC(R)) 5 Standards-Based Assessment ELD Standards ELD Assessment CASAS and SCANS Teacher Standards The Consequences of Standards-Based and Standardized Testing Test Bias Test-Driven Learning and Teaching Ethical Issues: Critical Language Testing Exercises For Your Further Reading 6 Assessing Listening Observing the Performance of the Four Skills The Importance of Listening Basic Types of Listening Micro- and Macroskills of Listening Designing Assessment Tasks: Intensive Listening Recognizing Phonological and Morphological Elements Paraphrase Recognition Designing Assessment Tasks: Responsive Listening Designing Assessment Tasks: Selective Listening Listening Cloze Information Transfer Sentence Repetition Designing Assessment Tasks: Extensive Listening Dictation Communicative Stimulus-Response Tasks Authentic Listening Tasks Exercises For Your Further Reading 7 Assessing Speaking Basic Types of Speaking Micro- and Macroskills of Speaking Designing Assessment Tasks: Imitative Speaking PhonePass Test Designing Assessment Tasks: Intensive Speaking Directed Response Tasks Read-Aloud Tasks Sentence/Dialogue Completion Tasks and Oral Questionnaires Picture-Cued Tasks Translation (of Limited Stretches of Discourse) Designing Assessment Tasks: Responsive Speaking Question and Answer Giving Instructions and Directions Paraphrasing Test of Spoken English (TSE) Designing Assessment Tasks: Interactive Speaking Interview Role Play Discussions and Conversations Games Oral Proficiency Interview (OPI) Designing Assessment: Extensive Speaking Oral Presentations Picture-Cued Story-Telling Retelling a Story, News Event Translation (of Extended Prose) Exercises For Your Further Reading 8 Assessing Reading Types (Genres) of Reading Microskills, Macroskills, and Strategies for Reading Types of Reading Designing Assessment Tasks: Perceptive Reading Reading Aloud Written Response Multiple-Choice Picture-Cued Items Designing Assessment Tasks: Selective Reading Multiple-Choice (for Form-Focused Criteria) Matching Tasks Editing Tasks Picture-Cued Tasks Gap-Filling Tasks Designing Assessment Tasks: Interactive Reading Cloze Tasks Impromptu Reading Plus Comprehension Questions Short-Answer Tasks Editing (Longer Texts) Scanning Ordering Tasks Information Transfer: Reading Charts, Maps, Graphs, Diagrams Designing Assessment Tasks: Extensive Reading Skimming Tasks Summarizing and Responding Note-Taking and Outlining Exercises For Your Further Reading 9 Assessing Writing Genres of Written Language Types of Writing performance Micro- and Macroskills of Writing Designing Assessment Tasks: Imitative Writing Tasks in [Hand] Writing Letters, Words, and Punctuation Spelling Tasks and Detecting Phoneme-Grapheme Correspondences Designing Assessment Tasks: Intensive (Controlled) Writing Dictation and Dicto-Comp Grammatical Transformation Tasks Picture-Cued Tasks Vocabulary Assessment Tasks Ordering Tasks Short-Answer and Sentence Completion Tasks Issues in Assessing Responsive and Extensive Writing Designing Assessment Tasks: Responsive and Extensive Writing Paraphrasing Guided Question and Answer Paragraph Construction Tasks Strategic Options Test of Written English (TWE(R)) Scoring Methods for Responsive and Extensive Writing Holistic Scoring Primary Trait Scoring Analytic Scoring Beyond Scoring Responding to Extensive Writing Assessing Initial Stages of the Process of Composing Assessing Later Stages of the Process of Composing Exercises For Your Further Reading 10 Beyond Tests: Alternatives in Assessment The Dilemma of Maximizing Both Practicality and Washback Performance-Based Assessment Portfolios Journals Conferences and Interviews Observations Self- and Peer-Assessments Types of Self- and Peer-Assessment Guidelines for Self- and Peer- Assessment A Taxonomy of Self- and Peer-Assessment Tasks Exercises For Your Further Reading 11 Grading and Student Evaluation Philosophy of Grading: What Should Grades Reflect? Guidelines for Selecting Grading Criteria Calculating Grades: Absolute and Relative Grading Teachers' Perceptions of Appropriate Grade Distributions Institutional Expectations and Constraints Cross-Cultural Factors and the Question of Difficulty What Do Letter Grades "Mean"? Alternatives to Letter Grading Some Principles and Guidelines for Grading and Evaluation Exercises For Your Further Reading Bibliography Name Index Subject Index
---
paper_title: Oral tests in Swedish schools: A five-year experiment

paper_content:

Abstract Large-scale experiments on the testing of oral proficiency in English, French and German have been carried out over the last 5 years in the Swedish gymnasium. Various kinds of tasks and different grading criteria have been used, and the practical problems of scheduling and of teacher training have been discussed. There are plans for a period of mandatory, full-scale participation, but no decisions have yet been taken.
---
paper_title: Spoken Spanish Language Development at the High School Level: A Mixed-Methods Study

paper_content:

Abstract: Communicative approaches to teaching language have emphasized the centrality of oral proficiency in the language acquisition process, but research investigating oral proficiency has been surprisingly limited, yielding an incomplete understanding of spoken language development. This study investigated the development of spoken language at the high school level over five consecutive years, involving more than 1,500 students representing 23 school districts. Quantitative Standards-Based Measure of Proficiency speaking scores and student]produced qualitative spoken samples (n>6,000 samples) contributed to an understanding of the development of spoken language. Hierarchical linear modeling (HLM) revealed a consistent growth trajectory of spoken language development, and results indicated that 18.30% of the variance in student outcomes may be attributed to the teacher variable.Key words: classroom]based research, longitudinal study, mixed methods, oral language developmentThe rise of communicative language learning has led to widespread acceptance of communicative competence as a primary goal of language education and, as such, central to good classroom practice (Savignon, 1997). This approach to language instruction emphasizes the ability to communicate in a second language in real]life situations both inside and beyond the classroom. Instead of measuring language learning in terms of seat time, test scores, or number of credit hours, communicative skills are demonstrated through task]based communicative activities. As a result of this emphasis on oral communication, proficiency has emerged as central to communicative language learning and teaching. However, there is a lack of research at the classroom level that reveals what students are able to do with oral language after one, two, three, and four years of language study. There is a paucity of research relating specifically to the development of spoken language at the secondary level (Tschirner & Heilenman, 1998). Although several studies have offered a glimpse of classroom-based proficiency ratings for the high school language learner (for examples, see Glisan & Foltz, 1998; Huebner & Jensen, 1992; Moeller & Reschke, 1993; Steinmeyer, 1984), the data have been strictly quantitative and have been conducted within educational systems with no consideration of related and potentially confounding factors, such as the teacher, during the exploration of oral language production.Due to such limitations, as well as substantial differences in results, particularly at the beginning levels of language learning in secondary classrooms, this study explored students' progress toward proficiency over a period of 5 years using a combination of qualitative methods, including thematic coding and organization, to reveal overarching trends in oral spoken language, and quantitative methods, including the Standards- Based Measurement of Proficiency (STAMP) test, a teacher-independent, computer- mediated measure of oral language proficiency.1 Purposefully integrating mixed methods offers "a very powerful mix" (Miles & Huberman, 1994, p. 42) that develops "a complex" picture of oral language development (Greene & Caracelli, 1997, p. 7). In choosing this integrative data design, the researchers' purpose was one of complementarity, a design element used to measure overlapping, but distinct, facets of a phenomenon under investigation (Caracelli & Greene, 1993). Results from one method-in this case, qualitative data-were used to enhance, illustrate, or clarify results from the other method-in this case, quantitative data (Greene & McClintock, 1985).Quantitative research questions for this study investigated the growth trajectory of spoken Spanish over four consecutive years of high school Spanish learning. Quantitative questions also delved into the variance in spoken production scores that was attributable to teacher differences or individual student differences. …
---
paper_title: Fundamental Considerations in Language Testing

paper_content:

Preface 1. Introduction The aims of the book The climate for language testing Research and development: needs and problems Research and development: an agenda Overview of the book Notes 2. Measurement Introduction Definition of terms: measurement, test, evaluation Essential measurement qualities Properties of measurement scales Characteristics that limit measurement Steps in measurement Summary Notes Further reading Discussion questions 3. Uses of Language Tests Introduction Uses of language tests in educational programs Research uses of language tests Features for classifying different types of language test Summary Further reading Discussion questions 4. Communicative Language Ability Introduction Language proficiency and communicative competence A theoretical framework of communicative language ability Summary Notes Further reading Discussion questions 5. Test Methods Introduction A framework of test method facets Applications of this framework to language testing Summary Notes Further reading Discussion questions 6. Reliability Introduction Factors that affect language test scores Classical true score measurement theory Generalizability theory Standard error of measurement: Interpreting individual test scores within classical true score and generalizability theory Item response theory Reliability of criterion-referenced test scores Factors that affect reliability estimates Systematic measurement error Summary Notes Further reading Discussion questions 7. Validation Introduction Reliability and validity revisited Validity as a unitary concept The evidential basis of validity Test bias The consequential or ethical basis of validity Post mortem: face validity Summary Notes Further reading Discussion questions 8. Some Persistent Problems and Future Directions Introduction Authentic language tests Some future directions A general model for explaining performance on language tests Apologia et prolegomenon Summary Notes Further reading Discussion questions Bibliography Author index Subject index
---
paper_title: Discourse Analysis In Language Testing

paper_content:

Language testing is concerned with the measurement of language. Language is the trait, and how we go about measuring it is the method. Trait involves the what , and method the how of testing. This paper will focus on the interaction between the what and the how , where the perspective of the what is the domain of discourse analysis. The paper will also review studies and tests that illustrate the degree to which the domain of discourse analysis has contributed to the theory, research, and development of language testing.
---
paper_title: Language Assessment: Principles and Classroom Practices

paper_content:

Preface 1 Testing, Assessing, and Teaching What Is a Test? Assessment and Teaching Informal and Formal Assessment Formative and Summative Assessment Norm-Referenced and Criterion-Referenced Test Approaches to Language Testing: A Brief History Discrete-point and Integrative Testing Communicative Language Testing Performance-Based Assessment Current Issues in Classroom Testing New Views on Intelligence Traditional and "Alternative" Assessment Computer-Based Testing Exercises For Your Further Reading 2 Principles of Language Assessment Practicality Reliability Student-Related Reliability Rater Reliability Test Administration Reliability Test Reliability Validity Content-Related Evidence Criterion-Related Evidence Construct-Related Evidence Consequential Validity Face Validity Authenticity Washback Applying Principles to the Evaluation of Classroom Tests 1. Are the test procedures practical? 2. Is the test reliable? 3. Does the procedure demonstrate content validity? 4. Is the procedure face valid and "biased for best"? 5. Are the test tasks as authentic as possible? 6. Does the test offer beneficial washback to the learner? Exercises For Your Further Reading 3 Designing Classroom Language Tests Test Types Language Aptitude Tests Proficiency Tests Placement Test Diagnostic Tests Achievement Tests Some Practical Steps to Test Construction Assessing Clear, Unambiguous Objectives Drawing Up Test Specifications Devising Test Tasks Designing Multiple-Choice Test Items 1. Design each item to measure a specific objective 2. State Both stem and options as simply and directly as possible 3. Make certain that the intended answer is clearly the only correct one 4. Use item indices to accept, discard, or revise items Scoring, Grading, and Giving Feedback Scoring Grading Giving Feedback Exercises For Your Further Reading 4 Standardized Testing What Is Standardization? Advantages and Disadvantages of Standardized Tests Developing a Standardized Test 1. Determine the purpose and objectives of the test 2. Design test specifications 3. Design, select, and arrange test tasks/items 4. Make appropriate evaluations of different kinds of items 5. Specify scoring procedures and reporting formats 6. Perform ongoing construct validation studies Standardized Language Proficiency Testing Four Standardized Language Proficiency Tests Test of English as a Foreign Language (TOEFL(R)) Michigan English Language Assessment Battery (MELAB) International English Language Testing System (IELTS) Test of English for International Communication (TOEIC(R)) Exercises For Your Further Reading Appendix to Chapter 4: Commercial Proficiency Tests: Sample Items and Tasks Test of English as a Foreign Language (TOEFL(R)) Michigan English Language Assessment Battery (MELAB) International English Language Testing System (IELTS) Test of English for International Communication (TOEIC(R)) 5 Standards-Based Assessment ELD Standards ELD Assessment CASAS and SCANS Teacher Standards The Consequences of Standards-Based and Standardized Testing Test Bias Test-Driven Learning and Teaching Ethical Issues: Critical Language Testing Exercises For Your Further Reading 6 Assessing Listening Observing the Performance of the Four Skills The Importance of Listening Basic Types of Listening Micro- and Macroskills of Listening Designing Assessment Tasks: Intensive Listening Recognizing Phonological and Morphological Elements Paraphrase Recognition Designing Assessment Tasks: Responsive Listening Designing Assessment Tasks: Selective Listening Listening Cloze Information Transfer Sentence Repetition Designing Assessment Tasks: Extensive Listening Dictation Communicative Stimulus-Response Tasks Authentic Listening Tasks Exercises For Your Further Reading 7 Assessing Speaking Basic Types of Speaking Micro- and Macroskills of Speaking Designing Assessment Tasks: Imitative Speaking PhonePass Test Designing Assessment Tasks: Intensive Speaking Directed Response Tasks Read-Aloud Tasks Sentence/Dialogue Completion Tasks and Oral Questionnaires Picture-Cued Tasks Translation (of Limited Stretches of Discourse) Designing Assessment Tasks: Responsive Speaking Question and Answer Giving Instructions and Directions Paraphrasing Test of Spoken English (TSE) Designing Assessment Tasks: Interactive Speaking Interview Role Play Discussions and Conversations Games Oral Proficiency Interview (OPI) Designing Assessment: Extensive Speaking Oral Presentations Picture-Cued Story-Telling Retelling a Story, News Event Translation (of Extended Prose) Exercises For Your Further Reading 8 Assessing Reading Types (Genres) of Reading Microskills, Macroskills, and Strategies for Reading Types of Reading Designing Assessment Tasks: Perceptive Reading Reading Aloud Written Response Multiple-Choice Picture-Cued Items Designing Assessment Tasks: Selective Reading Multiple-Choice (for Form-Focused Criteria) Matching Tasks Editing Tasks Picture-Cued Tasks Gap-Filling Tasks Designing Assessment Tasks: Interactive Reading Cloze Tasks Impromptu Reading Plus Comprehension Questions Short-Answer Tasks Editing (Longer Texts) Scanning Ordering Tasks Information Transfer: Reading Charts, Maps, Graphs, Diagrams Designing Assessment Tasks: Extensive Reading Skimming Tasks Summarizing and Responding Note-Taking and Outlining Exercises For Your Further Reading 9 Assessing Writing Genres of Written Language Types of Writing performance Micro- and Macroskills of Writing Designing Assessment Tasks: Imitative Writing Tasks in [Hand] Writing Letters, Words, and Punctuation Spelling Tasks and Detecting Phoneme-Grapheme Correspondences Designing Assessment Tasks: Intensive (Controlled) Writing Dictation and Dicto-Comp Grammatical Transformation Tasks Picture-Cued Tasks Vocabulary Assessment Tasks Ordering Tasks Short-Answer and Sentence Completion Tasks Issues in Assessing Responsive and Extensive Writing Designing Assessment Tasks: Responsive and Extensive Writing Paraphrasing Guided Question and Answer Paragraph Construction Tasks Strategic Options Test of Written English (TWE(R)) Scoring Methods for Responsive and Extensive Writing Holistic Scoring Primary Trait Scoring Analytic Scoring Beyond Scoring Responding to Extensive Writing Assessing Initial Stages of the Process of Composing Assessing Later Stages of the Process of Composing Exercises For Your Further Reading 10 Beyond Tests: Alternatives in Assessment The Dilemma of Maximizing Both Practicality and Washback Performance-Based Assessment Portfolios Journals Conferences and Interviews Observations Self- and Peer-Assessments Types of Self- and Peer-Assessment Guidelines for Self- and Peer- Assessment A Taxonomy of Self- and Peer-Assessment Tasks Exercises For Your Further Reading 11 Grading and Student Evaluation Philosophy of Grading: What Should Grades Reflect? Guidelines for Selecting Grading Criteria Calculating Grades: Absolute and Relative Grading Teachers' Perceptions of Appropriate Grade Distributions Institutional Expectations and Constraints Cross-Cultural Factors and the Question of Difficulty What Do Letter Grades "Mean"? Alternatives to Letter Grading Some Principles and Guidelines for Grading and Evaluation Exercises For Your Further Reading Bibliography Name Index Subject Index
---
paper_title: Research Issues in High‐Stakes Communicative Language Testing: Reflections on TOEFL's New Directions

paper_content:

0 The TOEFL (Test of English as a Foreign Language) is a high-stakes language test intended to measure proficiency in English in an academic setting. A new TOEFL will become operational in the fall of 2005. In developing this new TOEFL, we decided to modify the test in three important ways (Jamieson, Jones, Kirsch, Mosenthal, & Taylor, 2000). One is to include more constructed-response or performance tasks (including speaking) that require the integration of more than one language skill (e.g., writing or speaking in response to listening and reading materials). A second is to develop test materials that more closely reflect the types of discourse that occur in academic settings. And a third is to provide information that will help test users better understand the meaning of test scores. Educational Testing Service carried out an extensive program of research to achieve these goals and to make sure that a new test would meet professional standards (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 1999). But as is often the case in doing research, addressing one question makes us aware of other unanswered questions. This discussion focuses both on the research Educational Testing Service carried out to support TOEFL's new directions and other issues yet to be explored.
---
paper_title: Reeling, Writhing, Drawling, Stretching, and Fainting in Coils: Oral Proficiency Interviews as Conversation

paper_content:

This article provides a detailed examination of the oral proficiency interview and its underlying assumptions of validity and value. An attempt is made, by looking at the interview from the inside, to understand what OPIs are and what the participants in them do. The similarities and differences between interviews and conversations are examined, and some of the major problems of proficiency interviewing are illustrated and discussed. Finally, some specific proposals are made that may lead to more effective oral interviewing procedures. In this article I want to look at oral proficiency interviews (OPIs) as instances of conversation between people. Two questions are central to this discussion: (a) Are OPIs examples of conversational language use? and (b) Is conversational language use the appropriate (or the only, or the best) vehicle to evaluate oral proficiency? The tentative answer to the first question is a qualified no, the qualification being that an OPI can and may be designed to elicit conversational language use but frequently is not. The second question is far more difficult to answer: It addresses the central issue of the construct validity of oral tests in general. However, I propose that conversation is an appropriate vehicle for the all-around display of speaking ability in context. If the latter is a definition of oral proficiency, then I think we can be reasonably sure that conversation is the best vehicle for its evaluation. To address the questions I have posed, one must obtain an "insider's view" of the OPI. To do this, I have relied on three sources of data: (a) the analysis of several OPIs I have taken as an interviewee, (b) the study of transcripts and tapes of a variety of
---
paper_title: Taking Charge: Teacher Candidates' Preparation for the Oral Proficiency Interview.

paper_content:

Abstract: Within second language education, concern over teachers' content knowledge has typically manifested itself as concern over the teacher's target language proficiency. In increasing numbers, teacher preparation programs are turning to ACTFL's Oral Proficiency Interview (OPI) for measurement of this proficiency and using OPI ratings as high-stakes tests of readiness for teaching. This study reports the results of a survey of 734 teacher candidates on how they prepare for this assessment, both individually and in their training programs. The results of the survey reveal significant differences in study strategies between candidates who score Advanced or above and those who score below that targeted range. The more successful candidates use more discourse- and culturally rich sources for language input and spend considerably more time using the target language outside of course requirements. Candidates provide useful guidance to teacher educators on what helps and what hurts their efforts to be successful. Key words: assessment, oral proficiency, preservice teacher preparation, standards, survey research Background Teacher quality and teachers' content knowledge have remained top priorities over the past several decades in the discussions of educational policy and reform (e.g., Carnegie Corporation, 1986; Holmes Group, 1986; Smith, Desimone, & Ueno, 2005; Wenglinsky, 2002). Most generally agree that a key, if not the key to better learning for students is better teaching (Darling-Hammond, 2000), and clearly teachers' content knowledge, while elusive in its categorization and its measurement, is integral to teaching. The federal No Child Left Behind Act of 2001 (NCLB) calls for a "highly qualified" teacher in every classroom, which is defined as a teacher with full certification, a college degree, and demonstrated content knowledge in the subject being taught. The meaning of this final attribute, "content knowledge," is a source of considerable debate. It is surprisingly unclear what rank content knowledge plays in the making of a good teacher, yet all can agree that the French teacher who cannot speak French will not be a successful teacher of French. Grossman, Wilson, and Shulman (1989) defined content knowledge as "factual information, organizing principles, central concepts" and therefore as a subset of the broader construct of subject matter knowledge (p. 27). For the world language teacher, content knowledge may include analyses of works of art, themes of literature and film, grammatical structures, and cultural events. However, within second language (L2) education, concern over teachers' content knowledge has typically surfaced as concern over the teacher's target language proficiency. For better or for worse, testing of teachers' knowledge on a variety of fronts is now a common feature of the educational landscape. Fortunately, foreign language educators have been leaders in the arena of assessment of complex content knowledge by virtue of their early work on the Oral Proficiency Interview (OPI), an integrative test of target language speaking. The first oral proficiency guidelines were published in 1982, and the OPI's proficiency scales were developed at that time out of the Foreign Service Institute levels of oral proficiency (Clark & Clifford, 1988) and revised in 1999 (Breiner-Sanders, Lowe, Miles, & Swender, 2000). The ACTFL's current OPI is a tape-recorded interview conducted by a trained examiner that consists of five stages: a warm-up, level checks, probes, role-play, and wind-down and is designed to efficiently elicit a ratable speech sample.1 Test takers take the OPI one-on-one either in person or by telephone, and Swender (2003) found that scores did not vary significantly between those two formats. ACTFL joined forces with the National Council for Accreditation of Teacher Education (NCATE) in 2002 to participate in accreditation reviews of teacher education programs. …
---
paper_title: Predictors Revised: Linguistic Knowledge and Metalinguistic Awareness in Second Language Gain in Russian

paper_content:

The empirical study reported here investigated predictors of oral proficiency gain to the Advanced level resulting from immersion study abroad. The data consisted of 22 preimmersion Oral Proficiency Interviews (OPIs) of American students who participated in semester immersion programs in Moscow and St. Petersburg, and received a score of Intermediate High on the preimmersion OPI. On the basis of the students’ postprogram proficiency scores, the participants were grouped as learners who crossed the Advanced-level threshold (gainers) and as those who did not cross the threshold (nullgainers). This study used linguistic predictor variables identified in previous research (Brecht, Davidson, & Ginsberg, 1993) and introduced a new set of metalinguistic variables. An empirically based model constructed for the study included five variables that proved to be the best predictors for the Advanced-level threshold gain: linguistic (grammar, vocabulary, accuracy) and metalinguistic (self-corrected errors and sentence repair). The resulting findings yielded a robust model for predicting language outcomes of study abroad.
---
paper_title: Oral Proficiency Standards and Foreign Language Teacher Candidates: Current Findings and Future Research Directions

paper_content:

Abstract: The renewed national focus on teacher quality and effectiveness has resulted in more rigorous standards that describe the knowledge and skills required of teacher candidates across all disciplines. In the area of foreign languages, three sets of professional standards address the oral proficiency of teachers in the target languages they teach across the career continuum. For teacher candidates, the ACTFL/NCATE Program Standards for the Preparation of Foreign Language Teachers (2002) establish minimum oral proficiency levels based on the ACTFL Proficiency Guidelines-Speaking (2012). Utilizing ACTFL Oral Proficiency Interview (OPI) data, this study examines to what extent candidates are attaining the ACTFL/NCATE Oral Proficiency Standard of Advanced Low in most languages or Intermediate High in Arabic, Chinese, Japanese, and Korean. Findings indicate that 54.8% of candidates attained the required standard between 2006 and 2012 and that significant differences emerged for language, year tested, and university program results. Further research that takes into account additional contextual information about candidates and programs will inform continuing professional dialogue about the oral proficiency of teacher candidates entering the profession.Key words: Advanced Low, NCATE, proficiency, Standards, teacher preparationIntroductionIn the field of education, the past decade has seen an increasing emphasis on teacher accountability for student learning, at least partially in response to the national de- mand for "highly qualified" teachers pro- posed through the No Child Left Behind Act (NCLB) (2001). The need to identify teachers of high quality has resulted in more rigorous standards related to the preparation of teachers across disciplines, particularly with respect to the develop- ment of their content knowledge and skills (see, for example, NCATE, 2008). This national endeavor to develop teacher stand- ards has prompted widespread discussion and research exploring the type of content- specific knowledge teachers should pos- sess. This discussion has been particularly prevalent in the field of mathematics (Ball, Thames, & Phelps, 2008; Kleickmann et al., 2013), but it has also occurred in most other fields, including science (Lederman, 1999) and physical education (Siedentop, 2002).Although dialogue concerning the content knowledge and skills of foreign language teachers dates back to the late 1980s (ACTFL, 1988), more recently three sets of teacher standards have sparked a renewed discussion regarding the content knowledge and skills that foreign language teachers should possess as they move across the career continuum from teacher candidates to beginning teachers to accom- plished teachers. In 2002, NCATE ap- proved the ACTFL/NCATE Program Standards for the Preparation of Foreign Language Teachers (ACTFL, 2002) (here- after referred to as the ACTFL/NCATE Program Standards), which define the expectations for teacher candidates who complete a teacher preparation program and earn teacher certification. NCATE, which is recognized by the U.S. Department of Education and the Council for Higher Education as an accrediting agency for schools, colleges, and departments of education, determines which colleges of education meet rigorous national standards for teacher preparation. Foreign language teacher preparation programs that undergo NCATE review and seek national recogni- tion by ACTFL/NCATE must provide evidence that their programs are preparing candidates who can demonstrate the per- formance described in the standards. In conjunction with the development of the ACTFL/NCATE Program Standards, in 2002, the Interstate New Teacher Assess- ment and Support Consortium (INTASC1), with support from ACTFL, designed its Model Licensing Standards for Beginning Foreign Language Teachers, which may also be used by participating states to license teachers and induct them into the profession. To complete the career contin- uum, in 2001, the National Board for Professional Teaching Standards (NBPTS) created the World Languages Other Than English Standards, which describe the knowledge and skills necessary for accom- plished language teachers who elect to earn National Board certification; these stand- ards were updated in 2010 and renamed the World Languages Standards (NBPTS, 2001, 2010). …
---
paper_title: The Development of Oral Proficiency during a Semester in Germany.

paper_content:

Abstract: This study measures and analyzes improvements in students' oral proficiency during a study abroad semester in Germany. Oral proficiency interviews were conducted with participants before and after the program during three consecutive years. All interviews were assigned official ratings. Usage of two specific grammar structures associated with the Advanced level was also tracked. Before the program, most students were at Intermediate Low, even those who had completed 5th-semester German courses. Although 80% completed the program with improved proficiency ratings, very few reached the Advanced level. Internal analysis of speech samples did reveal that the post-program Intermediate Mid performances were much closer to the Advanced level than pre-program performances with the same rating. The study concludes with an assessment of curricular implications for post-program courses. Key words: German, grammar and proficiency levels, OPI as research tool, oral proficiency development, study abroad program assessment Introduction Language educators and program administrators agree that studying abroad is beneficial, and perhaps even essential, for students who want to improve their oral proficiency in a foreign language. This judgment is often based on intuition or subjective observation, such as informal conversations with students before and after their study abroad experiences, or comparisons of student performance in courses. Still, reliable studies that actually seek to measure and analyze the suspected linguistic gains made during study abroad semesters are relatively rare. Those that exist provide inconsistent results. Attaining a specific proficiency level, as defined by the ACTFL Proficiency Guidelines for Speaking (Breiner-Sanders, Lowe, Miles, & Swender, 2000), can be crucial for the future careers of language learners. The question of whether students actually reach a measurably higher level of proficiency as a result of studying abroad for a semester is therefore an important one. For instance, the National Council for the Accreditation of Teacher Education Programs (NCATE) Standards expect teacher candidates to demonstrate oral proficiency at the Advanced Low level. The teacher certification boards of 16 states currently require their candidates to prove that they have reached an Advanced Low or Intermediate High rating by taking an oral proficiency interview (OPI) conducted by the ACTFL testing office, Language Testing International (LTI). Other states permit individual institutions to decide how they will determine that their teaching candidates have met the required standard.1 Any attempt to assess the role of a study abroad semester in the development of oral proficiency will need to answer the following questions: * At what proficiency level do students enter the program? * Are students returning from their semesters abroad at a higher level of proficiency? * Do students' initial proficiency levels influence the improvement rate during the semester abroad? * What are reasonable expectations for improvement? ACTFL Guidelines The ACTFL proficiency guidelines and the OPI have practical implications for language curricula: They are nationally recognized measures of functional ability in a language. The ACTFL rating scale comprises four major levels (Novice, Intermediate, Advanced, and Superior) that are delineated according to a hierarchy of global tasks. The Novice, Intermediate, and Advanced levels are further divided into sublevels: Threshold performances are indicated as Low and High; solid performances, in both quality and quantity, are indicated as Mid. The ACTFL rating scale therefore measures 10 different proficiency categories. It is often depicted as an inverted pyramid to represent the exponential increase of facility with language throughout and within the major levels (see Figure 1). It is crucial to point out that the ACTFL guidelines define the High sublevel as a "fall from the next higher level rather than a strong ability demonstrated at the general level. …
---
paper_title: The Impact of the Oral Proficiency Interview on One Foreign Language Teacher Education Program

paper_content:

Abstract: The Oral Proficiency Interview (OPI) has been increasingly used in academia. However, while multiple studies have documented the growth in OPI implementation across the United States and the proficiency rates of its completers, few have focused specifically on foreign language teacher candidates, and even fewer have investigated the impact that this proficiency assessment may have on language teacher training programs. To better understand the impact of the OPI on foreign language teacher education programs and help guide programmatic decision making, a case study was conducted of one such program that recently implemented the OPI as part of its licensure requirements. The results confirmed earlier research with respect to expected proficiency outcomes of foreign language teacher candidates. The results also suggested that the OPI requirement did not negatively affect program enrollment, nor did teacher trainees negatively perceive the OPI requirement. Finally, the study provided evidence of the positive impact the OPI may have on a foreign language teacher education program. Recommended practices for implementing the OPI in teacher training programs and ways to support foreign language teacher candidates who must complete the assessment are discussed.Key words: case study, foreign/second language teacher preparation, language profi- ciency, native speaker/nonnative speaker status, teacher characteristicsIntroductionThe Oral Proficiency Interview (OPI) is a nationally recognized, standardized test that assesses an individual's oral communication skills in a foreign or target language. The OPI was developed in the 1980s by ACTFL and is administered through ACTFL'sofficial testing office, Language Testing International (LTI). The OPI is offered as an in-person or telephonic interview between the test taker and either a certified ACTFL tester who has un- dergone rigorous training in the evaluation process or a computer with which interview- ees interact with a digital avatar (Oral Profi- ciency Interview by Computer; OPIc). Regardless of the means of testing, a record- ing of the interview is analyzed by two cer- tified ACTFL raters who independently assess the candidate's speaking proficiency based on the descriptors in the ACTFL Pro- ficiency Guidelines (ACTFL, 2012), result- ing in 1 of 11 profi ciency level ratings ranging from Novice Low to Distinguished. While approximately 3,000 individuals completed the OPI in 1996, 27,875 OPIs or OPIcs were administered in 2011, and in 2012, that number surpassed 30,000.1Although individuals take the OPI or OPIc for a variety of different purposes, it is being used in increasing numbers in aca- demia for teacher certification purposes (Hammadou Sullivan, 2011; Malone & Montee, 2010). Currently, 23 states require aspiring foreign language teachers to com- plete an OPI. Sixteen of these states have adopted the ACTFL standard of Advanced Low or higher for teacher licensure in more commonly taught languages such as Spanish, French, and German, and the remaining seven have set the minimum standard at Intermediate High. With respect to less com- monly taught languages,2 all 23 states require a minimum score of Intermediate High.In addition to foreign language teacher certification programs in these 23 states, pro- grams seeking national recognition by ACTFL and the Council for the Accreditation of Educator Preparation3 must also demon- strate that their teacher candidates achieve a minimum passing score on the OPI of Ad- vanced Low for commonly taught languages and Intermediate High for less commonly taught languages (Pearson, Fonseca-Greber, & Foell, 2006). Programs whose teacher can- didates do not meet this minimum score as outlined in Standard 1a of the ACTFL Pro- gram Standards for the Preparation of Foreign Language Teachers (ACTFL, 2002) must of- fer planned remediation experiences in order to help those candidates enhance their target language speaking skills. …
---
paper_title: The Influence of Second Language Experience and Accent Familiarity on Oral Proficiency Rating: A Qualitative Investigation

paper_content:

This study investigates whether raters' knowledge of test takers' first language (L1) affects how the raters orient themselves to the task of rating oral speech. The authors qualitatively investigated the effects of accent familiarity on raters' score assignment processes. Twenty-six trained raters with a second language of Mandarin Chinese, Korean, or Spanish rated English speech samples from 72 English language learners from three L1 backgrounds (Mandarin Chinese, Korean, and Spanish). Afterward, raters watched videos of themselves rating and discussed their rating processes. Analyses of data, assisted by the qualitative analysis software package QSR NVivo 8, revealed that some raters attended to and were influenced by test-taker accent and L1. Raters who were heritage language learners discussed how their personal heritage status influenced how they thought about the speech samples from speakers whose L1 matched their heritage language. Although sensitivity to test-taker accents seemed to occur naturally in the rating process, findings suggest that when raters have learned or know, to varying degrees, the test takers' L1, they tend to orient themselves to the speech in a biased way, compromising test reliability.
---
paper_title: MANAGING TASK-RELATED TROUBLE IN L2 ORAL PROFICIENCY TESTS: CONTRASTING INTERACTION DATA AND RATER ASSESSMENT

paper_content:

The present study takes as an empirical point of departure the nature of interaction in second language speaking tests. We examine the relationship between ratings of students' performance in an oral proficiency test and the social practice of conducting 'test talk'. Using conversation analysis, our focal point is how students in peer- driven test interactions manage trouble related to the task-at-hand. Given that students were assessed not only on their linguistic skills, but also on their interactional ability and treatment of topics assigned, our emphasis on task management stems from a hypothesis that orientation to the test task is intimately connected to overall test outcome. We demonstrate that different types of task-related trouble (TRT) reveal diverse understandings of the test task and that 'doing-being a successful task manager' is connected to a moderate orientation to the task and test format. Students displaying such task management strategies were also assessed as highly proficient, whereas other task management strategies identified in our study correlated with low scores and grades. However, the relationship between subskill ratings and task management was not always clear-cut. We argue that the diverging understandings of the test task that learners display become part of how they are assessed and that certain task management strategies are rated less favorably than others. Our study holds promise for the fine-tuning of oral proficiency subskill ratings and raises questions as to the framing of test tasks, since this appears to have implications for student performance and evaluation.
---
paper_title: Social Interaction and Linguistic Gain During Study Abroad

paper_content:

Abstract: This study investigates the role of social interaction in language gain among study abroad students in France. Using the ACTFL Oral Proficiency Interview (OPI), the Can-Do self-assessment scale (Clark, 1981), a revised version of the Language Contact Profile (LCP; Freed, Dewey, Segalowitz, & Halter, 2001), and preand postdeparture questionnaires, we examine gains in oral proficiency as related to language contact in the study abroad environment. This research shows that language gain is possible during a semester-long study abroad program. It does not uphold the common belief that living situation and contact with authentic media differentiate students who improve from those who do not. Looking at the background of students (age, gender, grade point average, etc.), it reveals that only prior coursework in French correlates strongly with gains in proficiency once abroad. In its most surprising finding, this study suggests that speaking French with Americans may impede proficiency development.
---
paper_title: Comparing Examinee Attitudes Toward Computer-Assisted and Other Oral Proficiency Assessments

paper_content:

This article reports the results of a study of examinee attitudinal reactions to taking different formats of oral proficiency assessments across three languages: Spanish, Arabic, and Chinese. All 55 students in the study were administered both the tape-mediated Simulated Oral Proficiency Interview (SOPI) and a new Computerized Oral Proficiency Instrument (COPI). In addition, the 24 students participating in the Spanish study were administered the face-to-face A me ric an Counc il on the T ea ching of For e ign L angua ge s (ACTFL) Oral Proficiency Interview (OPI). Participants were graduate and undergraduate students taking language courses at their universities. The order of test administration was based on self- and teacher-assessed proficiency levels and experience in learning the target language. After each test, the learners completed a Likert questionnaire on six aspects of their attitudes towards and perceptions of that test. After finishing all tests and questionnaires, they were requested to compare the two technologymediated tests directly on those six aspects. This article presents the examinees' responses on those questionnaires, focusing primarily on differences between the two technology-mediated tests. It was found that the adaptive nature of the COPI allowed the difficulty level of the assessment task to be matched more appropriately to the proficiency level of the examinee. This led examinees, particularly those at the lower proficiency levels, to feel the COPI was less difficult than the SOPI. In most other aspects, the two tests were rated similarly. For the Spanish students, the two technology-mediated tests were rated similarly to the OPI, though the OPI appeared to them to be a better measure of real-life speaking skills. How do examinees react to an adaptive, computer-administered oral proficiency assessment? This article reports on examinee perspectives towards a prototype of the Computerized Oral Proficiency Instrument (COPI). The COPI was developed by the Center for Applied Linguistics (CAL; http://www.cal.org) under a research grant from the International Research and Studies Program of the United States Department of Education. Developed for Spanish, Arabic, and Chinese, the COPI is an adaptation of the tape-mediated Simulated Oral Proficiency Interview (SOPI), originally developed at CAL in 1985 (Clark, 1988). The primary goal of the COPI research project was to examine the effects of using multime dia c omputer tec hnology to e nha nc e the SO PI by giving examine es more contr ol over va rious a spe cts of the testing situa tion a nd by inc r ea sing ra ter s' ef fic ie nc y in scoring the test. The projec t c ompa r ed e xamine e per for ma nc e a nd af fe c t, a nd ra ter e f ficie nc y a nd a f fe ct, betw ee n the CO PI a nd the A me r ic an Counc il on the T ea ching of For e ign L angua ge s ( ACTFL ; http://www.actfl.org) O ra l Prof ic ie ncy I nte rview ( O PI ) f or Spa nish, a nd the COPI a nd SO PI for Spa nish, Chinese , and A ra bic . T his a rtic le compa re s e xa minee r ea ctions to the COPI , the SOPI , and the OPI.
---
paper_title: Research on the Comparability of the Oral Proficiency Interview and the Simulated Oral Proficiency Interview.

paper_content:

Abstract The simulated oral proficiency interview (SOPI) is a tape-mediated test designed to be a surrogate for the oral proficiency interview (OPI) in situations where a face-to-face interview is not possible or desirable. This article reviews research that sheds light on the comparability of the two tests. It begins with a brief description of the SOPI and continues to discuss the results of research on the reliability of the two tests, the agreement of scores obtained on the two tests, the comparability of the approach to testing, and the qualitative content of speech samples obtained via the two approaches. This article concludes with suggestions for further research.
---
paper_title: Reeling, Writhing, Drawling, Stretching, and Fainting in Coils: Oral Proficiency Interviews as Conversation

paper_content:

This article provides a detailed examination of the oral proficiency interview and its underlying assumptions of validity and value. An attempt is made, by looking at the interview from the inside, to understand what OPIs are and what the participants in them do. The similarities and differences between interviews and conversations are examined, and some of the major problems of proficiency interviewing are illustrated and discussed. Finally, some specific proposals are made that may lead to more effective oral interviewing procedures. In this article I want to look at oral proficiency interviews (OPIs) as instances of conversation between people. Two questions are central to this discussion: (a) Are OPIs examples of conversational language use? and (b) Is conversational language use the appropriate (or the only, or the best) vehicle to evaluate oral proficiency? The tentative answer to the first question is a qualified no, the qualification being that an OPI can and may be designed to elicit conversational language use but frequently is not. The second question is far more difficult to answer: It addresses the central issue of the construct validity of oral tests in general. However, I propose that conversation is an appropriate vehicle for the all-around display of speaking ability in context. If the latter is a definition of oral proficiency, then I think we can be reasonably sure that conversation is the best vehicle for its evaluation. To address the questions I have posed, one must obtain an "insider's view" of the OPI. To do this, I have relied on three sources of data: (a) the analysis of several OPIs I have taken as an interviewee, (b) the study of transcripts and tapes of a variety of
---
paper_title: Talking and Testing: Discourse approaches to the assessment of oral proficiency

paper_content:

1. Preface (by Celce-Murcia, Marianne) 2. Language Proficiency Interviews: A Discourse Approach (by He, Agnes Weiyun) 3. 1. Language Proficiency Interviews and Conversation 4. Re-analyzing the OPI: How Much Does It Look Like Natural Conversation? (by Johnson, Marysia) 5. Evaluating Learner Interactional Skills: Conversation at the Micro Level (by Riggenbach, Heidi) 6. What Happens When There's No One to Talk to? Spanish Foreign Language Discourse in Simulated Oral Proficiency Interviews (by Koike, Dale A.) 7. 2. Turns and Sequences in Language Proficiency Interviews 8. Answering Questions in LPIs: A Case Study (by He, Agnes Weiyun) 9. Framing the Language Proficiency Interview as a Speech Event: Native and Non-Native Speaker's Questions (by Moder, Carol Lynn) 10. Miscommunication in Language Proficiency Interviews of First-Year German Students: A Comparison with Natural Conversation (by Egbert, Maria M.) 11. Knowledge and Communication in Language Proficiency Interviews 12. Knowledge Structures in Oral Proficiency Interviews for International Teaching Assistants (by Mohan, Bernard A.) 13. The Use of Communication Strategies in Language Proficiency Interviews (by Yoshida-Morise, Yumiko) 14. Meaning Negociation in the Hungarian Oral Proficiency Examination of English (by Katona, Lucy) 15. 3. Language Proficiency Interviews as Cross-Cultural Encounters 16. Maintaining American Face in the Korean Oral Exam: Reflections on the Power of cross-Cultural Context (by Davies, Catherine Evans) 17. Confirmation Sequences as Intercational Resources in Korean Language Proficiency Interviews (by Kim, Kyu-hyun) 18. Divergent Frame Interpretations in Oral Proficiency Interview Interaction (by Ross, Steven) 19. "Let Them Eat Cake!" or how to Avoid Losing Your Head in Cross-Cultural Conversations (by Young, Richard) 20. Index of Names 21. Index of Topics
---
paper_title: Assessing paired orals: raters' orientation to interaction

paper_content:

Speaking tasks involving peer-to-peer candidate interaction are increasingly being incorporated into language proficiency assessments, in both large-scale international testing contexts, and in smaller-scale, for example course-related, ones. This growth in the popularity and use of paired and group orals has stimulated research, particularly into the types of discourse produced and the possible impact of candidate background factors on performance. However, despite the fact that the strongest argument for the validity of peer-to-peer assessment lies in the claim that such tasks allow for the assessment of a broader range of interactional skills than the more traditional interview-format tests do, there is surprisingly little research into the judgments that are made of such performances. The fact that raters, and rating criteria, are in a crucial mediating position between output and outcomes, warrants investi gation into how raters construe the interaction in these tasks. Such investigations have the potential to inform the development of interaction-based rating scales and ensure that validity claims are moved beyond the content level to the construct level. This paper reports the findings of a verbal protocol study of teacher-raters viewing the paired test discourse of 17 beginner dyads in a university-based Spanish as a foreign language course. The findings indicate that the raters identified three interaction parameters: non-verbal interpersonal communication, interactive listening, and interactional management. The findings have implications for our understanding of the construct of effective interaction in paired candidate speaking tests, and for the development of appropriate rating scales.
---
paper_title: Interactional Competence across Proficiency Levels: How do Learners Manage Interaction in Paired Speaking Tests?

paper_content:

Paired speaking tasks are now commonly used in both pedagogic and assessment contexts, as they elicit a wide range of interactional skills. The current study aims to offer an investigation of the interaction co-constructed by learners at different proficiency levels who are engaged in a paired speaking test, and to provide insights into the conceptualization of interactional competence and key distinguishing interactional features across levels. The findings suggest that our understanding of interactional competence both in the classroom and as a construct underlying tests and assessment scales needs to broaden to include not just interactional features such as topic development organization, but also listener support strategies and turn-taking management. A more comprehensive understanding of interactional competence has the potential to complement available descriptions of interactional skills in assessment scales of speaking and aid learners and teachers in communicative classrooms.
---
paper_title: A multifaceted approach to investigating pre-task planning effects on paired oral test performance

paper_content:

Despite the growing popularity of paired format speaking assessments, the effects of pre-task planning time on performance in these formats are not yet well understood. For example, some studies have revealed the benefits of planning but others have not. Using a multifaceted approach including analysis of the process of speaking performance, the aim of this paper is to investigate the effect of pre-task planning in a paired format. Data were collected from 32 students who carried out two decision-making tasks in pairs, under planned and unplanned conditions. The study used analyses of rating scores, discourse analytic measures, and conversation analysis (CA) of test-taker discourse to gain insight into co-constructing processes. A post-test questionnaire was also administered to understand the participants’ perceptions toward planned and unplanned interactions. The results from rating scores and discourse analytic measures revealed that planning had limited effect on performance, and analysis of the quest...
---
paper_title: From Face-to-Face to Paired Oral Proficiency Interviews: The Nut is Yet to be Cracked

paper_content:

The pressing need for English oral communication skills in multifarious contexts today is compelling impetus behind the large number of studies done on oral proficiency interviewing. Moreover, given the recently articulated concerns with the fairness and social dimension of such interviews, parallel concerns have been raised as to how most fairly to assess the oral communication skills of examinees, and what factors contribute to more skilled performance. This article sketches theory and practice on two rather competing formats of oral proficiency interviewing: face-to-face and paired. In the first place, it reviews the related literature on the alleged disadvantages of the individual format. Then, the pros and cons of the paired format are enumerated. It is discussed that the paired format has indeed met some of the criticisms leveled at individual oral proficiency interviewing. However, exploitation of the paired format as an undisputable alternative to the face-to-face format begs the question.
---
paper_title: A Microanalytic Perspective on Discourse, Proficiency, and Identity in Paired Oral Assessment

paper_content:

The increasing popularity of paired format in oral testing has engendered legitimate scrutiny of its reliability and validity as compared with the more traditional interviewer–interviewee arrangement. Although characteristics such as the gender, cultural/L1 background, and language proficiency of one's interlocutor likely affects the discourse produced with a partner, the question remains whether this interlocutor effect influences scores on the test. In this article, the construct interlocutor effect is further examined. Initially, transcript data from two large-scale speaking tests were analyzed by working backward from testtaker scores to locate discourse features to support those scores; identity formulations, such as “proficient” and “competent,” as constructed in and through the discourse test takers produce, emerged as a salient feature of the talk. Specifically, we posit that a test taker brings a language proficiency identity to a test task, and this identity is constructed, mediated, and display...
---
paper_title: MANAGING TASK-RELATED TROUBLE IN L2 ORAL PROFICIENCY TESTS: CONTRASTING INTERACTION DATA AND RATER ASSESSMENT

paper_content:

The present study takes as an empirical point of departure the nature of interaction in second language speaking tests. We examine the relationship between ratings of students' performance in an oral proficiency test and the social practice of conducting 'test talk'. Using conversation analysis, our focal point is how students in peer- driven test interactions manage trouble related to the task-at-hand. Given that students were assessed not only on their linguistic skills, but also on their interactional ability and treatment of topics assigned, our emphasis on task management stems from a hypothesis that orientation to the test task is intimately connected to overall test outcome. We demonstrate that different types of task-related trouble (TRT) reveal diverse understandings of the test task and that 'doing-being a successful task manager' is connected to a moderate orientation to the task and test format. Students displaying such task management strategies were also assessed as highly proficient, whereas other task management strategies identified in our study correlated with low scores and grades. However, the relationship between subskill ratings and task management was not always clear-cut. We argue that the diverging understandings of the test task that learners display become part of how they are assessed and that certain task management strategies are rated less favorably than others. Our study holds promise for the fine-tuning of oral proficiency subskill ratings and raises questions as to the framing of test tasks, since this appears to have implications for student performance and evaluation.
---
paper_title: Interviewer behaviour during oral proficiency interviews: A gender perspective

paper_content:

The article investigates the behaviour of interviewers during the oral proficiency interview within the framework of the national examination in the English language in Estonia from the gender perspective. The research investigates whether noticeable differences can be traced in the behavioural patterns of male and female interviewers that could lead to placing the candidates in a disadvantaged position, not allowing them to display their language proficiency to the full. The study involves nine female and nine male interviewers whose interview language is subjected to content analysis. The results show an overall attempt to follow the scripted interview format envisaged for this examination, but alterations to the script prevail with both gender groups. DOI: http://dx.doi.org/10.5128/ERYa9.14
---
paper_title: A many-facet Rasch analysis of the second language group oral discussion task:

paper_content:

FACETS many-facet Rasch analysis software (Linacre, 1998a) was utilized to look at two consecutive administrations of a large-scale (more than 1000 examinees) second language oral assessment in the form of a peer group discussion task with Japanese English-major university students. Facets modeled in the analysis were examinee, prompt, rater, and five rating category ‘items.’ Unidimensionality was shown to be strong in both datasets, and approaches to interpreting fit values for the facets modeled in the analysis were discussed. Examinee ability was the most substantial facet, followed by rater severity, and item. The prompt facet was negligible in magnitude. Rater differences in terms of severity were generally large, but this characteristic was not stable over time for individuals; returning raters tended to move toward greater severity and consistency, while new raters showed much more inconsistency. Analysis of the scales showed general validity in gradations of scale steps, though raters had some dif...
---
paper_title: Improving scores on the IELTS speaking test

paper_content:

This article presents three strategies for teaching students who are taking the IELTS speaking test. The first strategy is aimed at improving confidence and uses a variety of self-help materials from the field of popular psychology. The second encourages students to think critically and invokes a range of academic perspectives. The third strategy invites a close inspection and utilization of the marking criteria published in the IELTS handbook. These strategies were applied to a small group of students who sat the test in September 2005 and their scores are presented and analysed. There is some evidence that the strategies were effective in raising scores on the speaking component.
---
paper_title: An Investigation of the Effectiveness and Validity of Planning Time in Speaking Test Tasks

paper_content:

The study described in this article investigated the relationship between three variables in the IELTS oral module—planning, proficiency, and task—and was designed to enhance our understanding of how or whether these variables interact. The study aimed to determine whether differences in performance resulted from 1 or 2 min of planning time. It also aimed to identify the most effective strategies used by candidates in their planning. Ninety candidates, in two groups—intermediate and advanced—each undertook three tasks with either 1 or 2 min planning time or none. All tasks were rated by two raters, and the transcripts of the speech samples subjected to a discourse analysis. Neither the analysis of the scores nor the discourse analysis revealed any significant differences in performance according to the amount of planning time provided. Although this suggests that planning time does not positively advantage candidates, we argue that 1 min of pretask planning should be considered as an option in the test de...
---
paper_title: THE EFFECTS OF DIFFERENT LENGTHS OF TIME FOR PLANNING ON SECOND LANGUAGE PERFORMANCE

paper_content:

This article ::: reports on a study that investigated the effect of different amounts of planning time on the ::: speech performance of L2 speakers. Subjects were 4 groups of learners of German (31 in ::: total) performing 2 tasks each. The tasks varied in the degree of structure they contained and ::: the familiarity of information they tapped. The control group had no planning time available; ::: the 3 experimental groups had 1, 5, and 10 minutes of planning time, respectively, before they ::: started speaking. Results show fluency and lexical density of speech increase as a function of ::: planning time. Accuracy of speech improved with only 1 minute planning but did not increase ::: with more planning time. Complexity of speech was significantly higher for the 10-minute ::: planning condition only. No significant differences were found for the effect of planning on ::: the different tasks. This study employed various general and specific constructs for measuring ::: fluency, complexity, and accuracy of speech. The interrelationships and qualities of these ::: measures are also investigated and discussed.
---
paper_title: Assessing Young Language Learners

paper_content:

In this book the author builds a comprehensive framework for the assessment of young language learners in both foreign language and second language learning situations. She begins by considering why we need a special book on young learner assessment, and describes the nature of young learner language learning. The assessment approach is task-based and is centred around tasks and techniques suitable for young learners, with particular emphasis on classroom assessment. Oral language assessment, and reading and writing assessment are addressed in separate chapters, as is the large-scale testing of young learners. Underpinned by sound theory, the book is full of practical guidelines, and draws on examples of assessment contexts, issues and practices from around the world. Assessing Young Language Learners was winner of the Kenneth W Mildenberger Prize in 2006.
---
paper_title: The effect of learner proficiency on interactional moves and modified output in nonnative–nonnative interaction in Japanese as a foreign language

paper_content:

Abstract The present study, building upon Pica, Holliday, Lewis and Morgenthaler’s 1989 (Comprehensible output as an outcome of linguistic demands on the learner. Studies in Second Language Acquisition, 11(1), 63–90.) study on modified output, examines the impact of learner proficiency in learner–learner interaction, particularly on opportunities for modified output through interactional moves. Data were collected from learners of Japanese using two different types of tasks. Subjects were divided into three groups (Low–Low, High–High and High–Low groups). The results showed that mixed level dyads provided more interactional moves than same level dyads, but the frequent occurrence of interactional moves did not lead to the greatest amount of modified output. The findings have implications for interactional moves and learners’ modified output, and represent an extension of the research on learners’ output to another foreign language, Japanese.
---
paper_title: Talking and Testing: Discourse approaches to the assessment of oral proficiency

paper_content:

1. Preface (by Celce-Murcia, Marianne) 2. Language Proficiency Interviews: A Discourse Approach (by He, Agnes Weiyun) 3. 1. Language Proficiency Interviews and Conversation 4. Re-analyzing the OPI: How Much Does It Look Like Natural Conversation? (by Johnson, Marysia) 5. Evaluating Learner Interactional Skills: Conversation at the Micro Level (by Riggenbach, Heidi) 6. What Happens When There's No One to Talk to? Spanish Foreign Language Discourse in Simulated Oral Proficiency Interviews (by Koike, Dale A.) 7. 2. Turns and Sequences in Language Proficiency Interviews 8. Answering Questions in LPIs: A Case Study (by He, Agnes Weiyun) 9. Framing the Language Proficiency Interview as a Speech Event: Native and Non-Native Speaker's Questions (by Moder, Carol Lynn) 10. Miscommunication in Language Proficiency Interviews of First-Year German Students: A Comparison with Natural Conversation (by Egbert, Maria M.) 11. Knowledge and Communication in Language Proficiency Interviews 12. Knowledge Structures in Oral Proficiency Interviews for International Teaching Assistants (by Mohan, Bernard A.) 13. The Use of Communication Strategies in Language Proficiency Interviews (by Yoshida-Morise, Yumiko) 14. Meaning Negociation in the Hungarian Oral Proficiency Examination of English (by Katona, Lucy) 15. 3. Language Proficiency Interviews as Cross-Cultural Encounters 16. Maintaining American Face in the Korean Oral Exam: Reflections on the Power of cross-Cultural Context (by Davies, Catherine Evans) 17. Confirmation Sequences as Intercational Resources in Korean Language Proficiency Interviews (by Kim, Kyu-hyun) 18. Divergent Frame Interpretations in Oral Proficiency Interview Interaction (by Ross, Steven) 19. "Let Them Eat Cake!" or how to Avoid Losing Your Head in Cross-Cultural Conversations (by Young, Richard) 20. Index of Names 21. Index of Topics
---
paper_title: Oral proficiency interviews as varieties of interaction

paper_content:

This study investigates the extent to which oral proficiency interviews (OPIs) may be described as interactional varieties, which may then be compared with interaction in university and L2 classroom settings. The three varieties are related in terms of preparing students for a next stage in an educational process. L2 classroom interaction may prepare students for examinations such as OPIs, which provide access to universities, which in turn may prepare students for the world of work.1 Therefore, it is legitimate to examine the varieties in terms of whether the interactional experiences of students align or not in the different settings. OPIs in general are intended to assess the language proficiency of non-native speakers and to predict their ability to communicate in future encounters.
---
paper_title: Differing Expectations in the Assessment of the Speaking Skills of ESOL Learners.

paper_content:

This is a study of the assessment of the speaking skills of adult learners of English for speakers of other languages (ESOL). It is prompted by a concern that participants can have differing expectations of what nature of speech event a speaking test actually is. This concern was identified during the administration and analysis of assessments carried out as part of a study into adult ESOL pedagogy in the UK. The paper employs the notions of knowledge schema and frame in discourse to draw together areas of interest in testing: whether the speaking assessment should be viewed as an interview or as a conversation; divergent interpretations of the test event by learners; and variation in interlocutor behaviour. Implications for testing the speaking skills of adult ESOL learners are discussed; these are pertinent at a time of large-scale high-stakes testing of learners who are migrants to English speaking countries.
---
paper_title: The Interactional Organisation of the IELTS Speaking Test.

paper_content:

This study is based on the analysis of transcripts of 137 audio-recorded tests using a Conversation Analysis (CA) methodology. The institutional aim of standardisation in relation to assessment is shown to be the key principle underlying the organisation of the interaction. Overall, the vast majority of examiners conform to the instructions; in cases where they do not do so, they often give an advantage to some candidates. The overall organisation of the interaction is highly constrained, although there are some differences in the different parts of the test. The organisation of repair has a number of distinctive characteristics in that it is conducted according to strictly specified rules, in which the examiners have been briefed and trained. Speaking test interaction is an institutional variety of interaction with three sub-varieties. It is very different to ordinary conversation, has some similarities with some sub-varieties of L2 classroom interaction and some similarities with interaction in universities. A number of recommendations are made in relation to examiner training, instructions and test design.
---
paper_title: Role-play in oral proficiency interviews: Interactive footing and interactional competencies

paper_content:

Abstract This study aims to discuss role-play activity in oral proficiency interviews (OPIs) in terms of its construct validity; that is, whether it correlates with what it is supposed to measure. The data for the analysis were obtained from 71 role-play activities conducted during an OPI. The analysis is based on conversation analytic (CA) methodology and invokes the analytic frameworks of interactive footing and interactional competencies. Conversational analyses performed on the data revealed that candidates executed not only the role-play instructions but also the interviewers’ explicit and implicit requirements of the next desirable action. In doing so, candidates employed and displayed their interactional competencies in role-play interactions. The role-play activity in the OPI being managed by the interviewer created an asymmetrical relationship between the interviewers and the candidates in terms of speaking rights (i.e., turn-taking and topical organization). Nevertheless, competencies that candidates displayed in performing a role-play activity did not seem different from those employed in ordinary conversations. Thus, role-play activity in OPIs, if recognized as an interactional phenomenon co-constructed by participants’ display of their turn-by-turn practical evaluation of each other's actions, seems to be a valid instrument for assessing the candidates’ performance of the given tasks.
---
paper_title: Interacting in pairs in a test of oral proficiency: Co-constructing a better performance

paper_content:

This study, framed within sociocultural theory, examines the interaction of adult ESL test-takers in two tests of oral proficiency: one in which they interacted with an examiner (the individual format) and one in which they interacted with another student (the paired format). The data for the eight pairs in this study were drawn from a larger study comparing the two test formats in the context of high-stakes exit testing from an Academic Preparation Program at a large Canadian university. All of the test-takers participated in both test formats involving a discussion with comparable speaking prompts. The findings from the quantitative analyses show that overall the test-takers performed better in the paired format in that their scores were on average higher than when they interacted with an examiner. Qualitative analysis of the test-takers' speaking indicates that the differences in performance in the two test formats were more marked than the scores suggest. When test-takers interacted with other student...
---
paper_title: From paper to practice : Asking and responding to a standardized question item in performance appraisal interviews

paper_content:

This paper examines how a standardized question is launched and received in a corpus of performance appraisal interviews, with a focus on how pre-­formulated questions are translated into interaction. Using conversation analysis, we demonstrate that the same question becomes many different actions in practice. Prefaces as well as prosodic and lexical alterations make relevant different responses, and as such, the question can be recruited to initiate diverse interactional projects such as assessments and other socially delicate activities. As a consequence, goals of uniformity and standardization may be subverted. The interactional adaptations further evidence the strength of recipient design as reformulations also result in more fitted and personalized answers. Our study contributes to the understanding of standardization versus interactionalization, and points to the strong interrelationship between question design and the fitting of response options.
---
paper_title: Assessing second language pragmatics: An overview and introductions

paper_content:

Pragmatics is a key domain in language assessment. For more than two decades, advances have been made in conceptualizing the domain, developing assessment instruments, and applying current methods of data treatment to the analysis of test performance. This book, the first edited volume on the topic, brings together empirical studies on a range of well-established and innovative strategies for the assessment of pragmatic ability in a second language. In this introductory chapter, we will first offer an overview of key concepts, situate theoretical models of pragmatic competence within the larger frameworks of communicative language ability and interactional competence, and consider the relationship between pragmatics and language testing. We will then introduce the chapters, organized into two Parts. The chapters assembled in Part I investigate assessment instruments and practices for a variety of assessment constructs, purposes, and contexts, guided by different theoretical outlooks on pragmatics. Part II comprises studies of interaction in different forms of oral proficiency interview, conducted from the perspective of conversation analysis.
---
paper_title: A comparative task-in-interaction analysis of OPI backsliding

paper_content:

The oral proficiency interview (OPI) is often used in the domain of business English as a criterion for access to overseas assignments and job promotions. Little, however, is known about variation in interaction style across interviewers, which motivates in this study a contrastive analysis of two oral proficiency interviews used for gatekeeping purposes. The two interviews were conducted with the same candidate three months apart, and provide a rare glimpse of contrastive interviewer strategies with a single candidate. The analysis examines evidence that the candidate backslid from an earlier successful interview to a categorically lower level of performance in the second interview. Analyses of the candidate's differential establishment of footing in the interview, misalignments to the tone of the interviewer, and differential tendencies of the two interviewers to accommodate to the candidate are featured. Interviewer differences in proclivity to backchannel indicate how facilitative accommodation in scaffolding the interaction may influence the initial rating. The micro-analyses of interview discourse suggest that differences in interviewer style potentially lead to divergent outcomes in the two interviews. In spite of considerable variation in the interaction styles of the two interviewers, consistency in the outcomes of five repeated second ratings of the candidate's performance suggest a rating system robust against even large differences in interviewer style.
---
paper_title: Interviewer behaviour during oral proficiency interviews: A gender perspective

paper_content:

The article investigates the behaviour of interviewers during the oral proficiency interview within the framework of the national examination in the English language in Estonia from the gender perspective. The research investigates whether noticeable differences can be traced in the behavioural patterns of male and female interviewers that could lead to placing the candidates in a disadvantaged position, not allowing them to display their language proficiency to the full. The study involves nine female and nine male interviewers whose interview language is subjected to content analysis. The results show an overall attempt to follow the scripted interview format envisaged for this examination, but alterations to the script prevail with both gender groups. DOI: http://dx.doi.org/10.5128/ERYa9.14
---
paper_title: Performing Identities in Speaking Tests: Co-Construction Revisited

paper_content:

Poststructuralist perspectives on the discursive construction of identity and critical approaches to linguistics, discourse analysis, and language testing have contributed to a growing awareness of the complex social nature of performance in speaking tests. In this article, I explore how the concept of co-construction is useful to understand how candidate and interviewer identities are created in the discourse of speaking tests but argue for an expanded notion of co-construction that draws upon the concept of intertextuality to recognise the myriad of other ‘voices’, representing those who craft the tests, are interwoven into the discourses produced. The study adopts a qualitative approach, drawing upon discourse analysis to examine co-construction and intertextuality in two levels of Cambridge speaking tests. The findings reveal how interviewers and candidates co-construct particular identities through their interaction with each other, the interlocutor frame, and the test materials, and this can account...
---
paper_title: Assessing second language pragmatics: An overview and introductions

paper_content:

Pragmatics is a key domain in language assessment. For more than two decades, advances have been made in conceptualizing the domain, developing assessment instruments, and applying current methods of data treatment to the analysis of test performance. This book, the first edited volume on the topic, brings together empirical studies on a range of well-established and innovative strategies for the assessment of pragmatic ability in a second language. In this introductory chapter, we will first offer an overview of key concepts, situate theoretical models of pragmatic competence within the larger frameworks of communicative language ability and interactional competence, and consider the relationship between pragmatics and language testing. We will then introduce the chapters, organized into two Parts. The chapters assembled in Part I investigate assessment instruments and practices for a variety of assessment constructs, purposes, and contexts, guided by different theoretical outlooks on pragmatics. Part II comprises studies of interaction in different forms of oral proficiency interview, conducted from the perspective of conversation analysis.
---
paper_title: Assessing paired orals: raters' orientation to interaction

paper_content:

Speaking tasks involving peer-to-peer candidate interaction are increasingly being incorporated into language proficiency assessments, in both large-scale international testing contexts, and in smaller-scale, for example course-related, ones. This growth in the popularity and use of paired and group orals has stimulated research, particularly into the types of discourse produced and the possible impact of candidate background factors on performance. However, despite the fact that the strongest argument for the validity of peer-to-peer assessment lies in the claim that such tasks allow for the assessment of a broader range of interactional skills than the more traditional interview-format tests do, there is surprisingly little research into the judgments that are made of such performances. The fact that raters, and rating criteria, are in a crucial mediating position between output and outcomes, warrants investi gation into how raters construe the interaction in these tasks. Such investigations have the potential to inform the development of interaction-based rating scales and ensure that validity claims are moved beyond the content level to the construct level. This paper reports the findings of a verbal protocol study of teacher-raters viewing the paired test discourse of 17 beginner dyads in a university-based Spanish as a foreign language course. The findings indicate that the raters identified three interaction parameters: non-verbal interpersonal communication, interactive listening, and interactional management. The findings have implications for our understanding of the construct of effective interaction in paired candidate speaking tests, and for the development of appropriate rating scales.
---
paper_title: Interacting in pairs in a test of oral proficiency: Co-constructing a better performance

paper_content:

This study, framed within sociocultural theory, examines the interaction of adult ESL test-takers in two tests of oral proficiency: one in which they interacted with an examiner (the individual format) and one in which they interacted with another student (the paired format). The data for the eight pairs in this study were drawn from a larger study comparing the two test formats in the context of high-stakes exit testing from an Academic Preparation Program at a large Canadian university. All of the test-takers participated in both test formats involving a discussion with comparable speaking prompts. The findings from the quantitative analyses show that overall the test-takers performed better in the paired format in that their scores were on average higher than when they interacted with an examiner. Qualitative analysis of the test-takers' speaking indicates that the differences in performance in the two test formats were more marked than the scores suggest. When test-takers interacted with other student...
---
paper_title: Agreeing to Disagree: "Doing Disagreement" in Assessed Oral L2 Interactions.

paper_content:

While disagreements are often considered dispreferred choices and potentially face-threatening acts due to their oppositional nature, this perception does not adequately reflect the importance of disagreeing for many types of interaction, such as problem-solving and decision-making. Developing ability in performing this speech act therefore becomes relevant for learners of an L2 in order to show achievements in such general language learning targets as expressing and defending opinions and arguments, as well as negotiating consensual decisions. This paper presents a sequential analysis of disagreements in assessed interactions of advanced German-speaking learners of English, observed in three different exam formats, i.e. teacher-led interview, role play with teacher and paired student interaction. Overall, 33 disagreement episodes were identified in 18 oral exams. Findings show that the choice of examination format constrains range and type of disagreement trajectory enacted by the learners, highlighting ...
---
paper_title: Interactional Competence across Proficiency Levels: How do Learners Manage Interaction in Paired Speaking Tests?

paper_content:

Paired speaking tasks are now commonly used in both pedagogic and assessment contexts, as they elicit a wide range of interactional skills. The current study aims to offer an investigation of the interaction co-constructed by learners at different proficiency levels who are engaged in a paired speaking test, and to provide insights into the conceptualization of interactional competence and key distinguishing interactional features across levels. The findings suggest that our understanding of interactional competence both in the classroom and as a construct underlying tests and assessment scales needs to broaden to include not just interactional features such as topic development organization, but also listener support strategies and turn-taking management. A more comprehensive understanding of interactional competence has the potential to complement available descriptions of interactional skills in assessment scales of speaking and aid learners and teachers in communicative classrooms.
---
paper_title: Young Foreign Language Learners’ Interactions During Task-Based Paired Assessments

paper_content:

Despite the popularity of task-based language teaching (TBLT) in foreign language (FL) education at elementary school, it remains unclear how young learners’ FL abilities can best be evaluated with tasks. The present study seeks to understand developmental differences in interactions among elementary-school students during task-based language assessment (TBLA) and aims to address the possibilities and limitations of introducing paired-TBLA among young FL learners. The participants were 32 fourth-grade students (ages 9–10) and 32 sixth-grade students (ages 11–12) and their teachers from a FL program where TBLT was mandated. The students were engaged in two sets of assessment tasks with their peers. It was found that the interaction patterns were different between the two grade levels. Compared with the sixth-grade dyads, the fourth-grade dyads showed less mutual topic development, used formulaic turn-taking more frequently, and had a harder time taking their partners’ perspectives during the tasks. Consequ...
---
paper_title: A Microanalytic Perspective on Discourse, Proficiency, and Identity in Paired Oral Assessment

paper_content:

The increasing popularity of paired format in oral testing has engendered legitimate scrutiny of its reliability and validity as compared with the more traditional interviewer–interviewee arrangement. Although characteristics such as the gender, cultural/L1 background, and language proficiency of one's interlocutor likely affects the discourse produced with a partner, the question remains whether this interlocutor effect influences scores on the test. In this article, the construct interlocutor effect is further examined. Initially, transcript data from two large-scale speaking tests were analyzed by working backward from testtaker scores to locate discourse features to support those scores; identity formulations, such as “proficient” and “competent,” as constructed in and through the discourse test takers produce, emerged as a salient feature of the talk. Specifically, we posit that a test taker brings a language proficiency identity to a test task, and this identity is constructed, mediated, and display...
---
paper_title: Measuring Oral Proficiency through Paired-Task Performance

paper_content:

Contents: Modelling language knowledge and performance - Investigating variability in oral proficiency testing: the oral interview, the paired and group oral, test tasks - Discourse patterns in the paired mode: style of dyadic interaction - Impact of partner's proficiency level - Impact of candidate's previous learning experience - Test takers' perceptions of the paired oral - Future research in measuring oral performance through paired-task performance.
---
paper_title: Is linguistic ability variation in paired oral language testing problematic

paper_content:

This study looks at the real and perceived effects of inter-candidate variation in linguistic ability during pair-tested oral examinations in a southern Italian context. Pair testing has not been taken up in schools and universities in southern Italy despite its wide use in international testing systems. Colleagues and prospective candidates often report to teachers their belief that test results can be adversely or positively affected by candidate pairing. To investigate the existence and validity of these beliefs, a series of tests was devised consisting of paired oral tests and pre- and post-test questionnaires. It was found that while the great majority of candidates believed that their test scores could be, and in many cases had been, affected by their candidate pairing, no evidence came to light to support these beliefs. In fact the linguistic ability of a candidate�s partner appeared to have little or no effect on his or her performance.
---
paper_title: MANAGING TASK-RELATED TROUBLE IN L2 ORAL PROFICIENCY TESTS: CONTRASTING INTERACTION DATA AND RATER ASSESSMENT

paper_content:

The present study takes as an empirical point of departure the nature of interaction in second language speaking tests. We examine the relationship between ratings of students' performance in an oral proficiency test and the social practice of conducting 'test talk'. Using conversation analysis, our focal point is how students in peer- driven test interactions manage trouble related to the task-at-hand. Given that students were assessed not only on their linguistic skills, but also on their interactional ability and treatment of topics assigned, our emphasis on task management stems from a hypothesis that orientation to the test task is intimately connected to overall test outcome. We demonstrate that different types of task-related trouble (TRT) reveal diverse understandings of the test task and that 'doing-being a successful task manager' is connected to a moderate orientation to the task and test format. Students displaying such task management strategies were also assessed as highly proficient, whereas other task management strategies identified in our study correlated with low scores and grades. However, the relationship between subskill ratings and task management was not always clear-cut. We argue that the diverging understandings of the test task that learners display become part of how they are assessed and that certain task management strategies are rated less favorably than others. Our study holds promise for the fine-tuning of oral proficiency subskill ratings and raises questions as to the framing of test tasks, since this appears to have implications for student performance and evaluation.
---
paper_title: Fundamental Considerations in Language Testing

paper_content:

Preface 1. Introduction The aims of the book The climate for language testing Research and development: needs and problems Research and development: an agenda Overview of the book Notes 2. Measurement Introduction Definition of terms: measurement, test, evaluation Essential measurement qualities Properties of measurement scales Characteristics that limit measurement Steps in measurement Summary Notes Further reading Discussion questions 3. Uses of Language Tests Introduction Uses of language tests in educational programs Research uses of language tests Features for classifying different types of language test Summary Further reading Discussion questions 4. Communicative Language Ability Introduction Language proficiency and communicative competence A theoretical framework of communicative language ability Summary Notes Further reading Discussion questions 5. Test Methods Introduction A framework of test method facets Applications of this framework to language testing Summary Notes Further reading Discussion questions 6. Reliability Introduction Factors that affect language test scores Classical true score measurement theory Generalizability theory Standard error of measurement: Interpreting individual test scores within classical true score and generalizability theory Item response theory Reliability of criterion-referenced test scores Factors that affect reliability estimates Systematic measurement error Summary Notes Further reading Discussion questions 7. Validation Introduction Reliability and validity revisited Validity as a unitary concept The evidential basis of validity Test bias The consequential or ethical basis of validity Post mortem: face validity Summary Notes Further reading Discussion questions 8. Some Persistent Problems and Future Directions Introduction Authentic language tests Some future directions A general model for explaining performance on language tests Apologia et prolegomenon Summary Notes Further reading Discussion questions Bibliography Author index Subject index
---
paper_title: Performing Identities in Speaking Tests: Co-Construction Revisited

paper_content:

Poststructuralist perspectives on the discursive construction of identity and critical approaches to linguistics, discourse analysis, and language testing have contributed to a growing awareness of the complex social nature of performance in speaking tests. In this article, I explore how the concept of co-construction is useful to understand how candidate and interviewer identities are created in the discourse of speaking tests but argue for an expanded notion of co-construction that draws upon the concept of intertextuality to recognise the myriad of other ‘voices’, representing those who craft the tests, are interwoven into the discourses produced. The study adopts a qualitative approach, drawing upon discourse analysis to examine co-construction and intertextuality in two levels of Cambridge speaking tests. The findings reveal how interviewers and candidates co-construct particular identities through their interaction with each other, the interlocutor frame, and the test materials, and this can account...
---
paper_title: Language Assessment: Principles and Classroom Practices

paper_content:

Preface 1 Testing, Assessing, and Teaching What Is a Test? Assessment and Teaching Informal and Formal Assessment Formative and Summative Assessment Norm-Referenced and Criterion-Referenced Test Approaches to Language Testing: A Brief History Discrete-point and Integrative Testing Communicative Language Testing Performance-Based Assessment Current Issues in Classroom Testing New Views on Intelligence Traditional and "Alternative" Assessment Computer-Based Testing Exercises For Your Further Reading 2 Principles of Language Assessment Practicality Reliability Student-Related Reliability Rater Reliability Test Administration Reliability Test Reliability Validity Content-Related Evidence Criterion-Related Evidence Construct-Related Evidence Consequential Validity Face Validity Authenticity Washback Applying Principles to the Evaluation of Classroom Tests 1. Are the test procedures practical? 2. Is the test reliable? 3. Does the procedure demonstrate content validity? 4. Is the procedure face valid and "biased for best"? 5. Are the test tasks as authentic as possible? 6. Does the test offer beneficial washback to the learner? Exercises For Your Further Reading 3 Designing Classroom Language Tests Test Types Language Aptitude Tests Proficiency Tests Placement Test Diagnostic Tests Achievement Tests Some Practical Steps to Test Construction Assessing Clear, Unambiguous Objectives Drawing Up Test Specifications Devising Test Tasks Designing Multiple-Choice Test Items 1. Design each item to measure a specific objective 2. State Both stem and options as simply and directly as possible 3. Make certain that the intended answer is clearly the only correct one 4. Use item indices to accept, discard, or revise items Scoring, Grading, and Giving Feedback Scoring Grading Giving Feedback Exercises For Your Further Reading 4 Standardized Testing What Is Standardization? Advantages and Disadvantages of Standardized Tests Developing a Standardized Test 1. Determine the purpose and objectives of the test 2. Design test specifications 3. Design, select, and arrange test tasks/items 4. Make appropriate evaluations of different kinds of items 5. Specify scoring procedures and reporting formats 6. Perform ongoing construct validation studies Standardized Language Proficiency Testing Four Standardized Language Proficiency Tests Test of English as a Foreign Language (TOEFL(R)) Michigan English Language Assessment Battery (MELAB) International English Language Testing System (IELTS) Test of English for International Communication (TOEIC(R)) Exercises For Your Further Reading Appendix to Chapter 4: Commercial Proficiency Tests: Sample Items and Tasks Test of English as a Foreign Language (TOEFL(R)) Michigan English Language Assessment Battery (MELAB) International English Language Testing System (IELTS) Test of English for International Communication (TOEIC(R)) 5 Standards-Based Assessment ELD Standards ELD Assessment CASAS and SCANS Teacher Standards The Consequences of Standards-Based and Standardized Testing Test Bias Test-Driven Learning and Teaching Ethical Issues: Critical Language Testing Exercises For Your Further Reading 6 Assessing Listening Observing the Performance of the Four Skills The Importance of Listening Basic Types of Listening Micro- and Macroskills of Listening Designing Assessment Tasks: Intensive Listening Recognizing Phonological and Morphological Elements Paraphrase Recognition Designing Assessment Tasks: Responsive Listening Designing Assessment Tasks: Selective Listening Listening Cloze Information Transfer Sentence Repetition Designing Assessment Tasks: Extensive Listening Dictation Communicative Stimulus-Response Tasks Authentic Listening Tasks Exercises For Your Further Reading 7 Assessing Speaking Basic Types of Speaking Micro- and Macroskills of Speaking Designing Assessment Tasks: Imitative Speaking PhonePass Test Designing Assessment Tasks: Intensive Speaking Directed Response Tasks Read-Aloud Tasks Sentence/Dialogue Completion Tasks and Oral Questionnaires Picture-Cued Tasks Translation (of Limited Stretches of Discourse) Designing Assessment Tasks: Responsive Speaking Question and Answer Giving Instructions and Directions Paraphrasing Test of Spoken English (TSE) Designing Assessment Tasks: Interactive Speaking Interview Role Play Discussions and Conversations Games Oral Proficiency Interview (OPI) Designing Assessment: Extensive Speaking Oral Presentations Picture-Cued Story-Telling Retelling a Story, News Event Translation (of Extended Prose) Exercises For Your Further Reading 8 Assessing Reading Types (Genres) of Reading Microskills, Macroskills, and Strategies for Reading Types of Reading Designing Assessment Tasks: Perceptive Reading Reading Aloud Written Response Multiple-Choice Picture-Cued Items Designing Assessment Tasks: Selective Reading Multiple-Choice (for Form-Focused Criteria) Matching Tasks Editing Tasks Picture-Cued Tasks Gap-Filling Tasks Designing Assessment Tasks: Interactive Reading Cloze Tasks Impromptu Reading Plus Comprehension Questions Short-Answer Tasks Editing (Longer Texts) Scanning Ordering Tasks Information Transfer: Reading Charts, Maps, Graphs, Diagrams Designing Assessment Tasks: Extensive Reading Skimming Tasks Summarizing and Responding Note-Taking and Outlining Exercises For Your Further Reading 9 Assessing Writing Genres of Written Language Types of Writing performance Micro- and Macroskills of Writing Designing Assessment Tasks: Imitative Writing Tasks in [Hand] Writing Letters, Words, and Punctuation Spelling Tasks and Detecting Phoneme-Grapheme Correspondences Designing Assessment Tasks: Intensive (Controlled) Writing Dictation and Dicto-Comp Grammatical Transformation Tasks Picture-Cued Tasks Vocabulary Assessment Tasks Ordering Tasks Short-Answer and Sentence Completion Tasks Issues in Assessing Responsive and Extensive Writing Designing Assessment Tasks: Responsive and Extensive Writing Paraphrasing Guided Question and Answer Paragraph Construction Tasks Strategic Options Test of Written English (TWE(R)) Scoring Methods for Responsive and Extensive Writing Holistic Scoring Primary Trait Scoring Analytic Scoring Beyond Scoring Responding to Extensive Writing Assessing Initial Stages of the Process of Composing Assessing Later Stages of the Process of Composing Exercises For Your Further Reading 10 Beyond Tests: Alternatives in Assessment The Dilemma of Maximizing Both Practicality and Washback Performance-Based Assessment Portfolios Journals Conferences and Interviews Observations Self- and Peer-Assessments Types of Self- and Peer-Assessment Guidelines for Self- and Peer- Assessment A Taxonomy of Self- and Peer-Assessment Tasks Exercises For Your Further Reading 11 Grading and Student Evaluation Philosophy of Grading: What Should Grades Reflect? Guidelines for Selecting Grading Criteria Calculating Grades: Absolute and Relative Grading Teachers' Perceptions of Appropriate Grade Distributions Institutional Expectations and Constraints Cross-Cultural Factors and the Question of Difficulty What Do Letter Grades "Mean"? Alternatives to Letter Grading Some Principles and Guidelines for Grading and Evaluation Exercises For Your Further Reading Bibliography Name Index Subject Index
---
paper_title: MANAGING TASK-RELATED TROUBLE IN L2 ORAL PROFICIENCY TESTS: CONTRASTING INTERACTION DATA AND RATER ASSESSMENT

paper_content:

The present study takes as an empirical point of departure the nature of interaction in second language speaking tests. We examine the relationship between ratings of students' performance in an oral proficiency test and the social practice of conducting 'test talk'. Using conversation analysis, our focal point is how students in peer- driven test interactions manage trouble related to the task-at-hand. Given that students were assessed not only on their linguistic skills, but also on their interactional ability and treatment of topics assigned, our emphasis on task management stems from a hypothesis that orientation to the test task is intimately connected to overall test outcome. We demonstrate that different types of task-related trouble (TRT) reveal diverse understandings of the test task and that 'doing-being a successful task manager' is connected to a moderate orientation to the task and test format. Students displaying such task management strategies were also assessed as highly proficient, whereas other task management strategies identified in our study correlated with low scores and grades. However, the relationship between subskill ratings and task management was not always clear-cut. We argue that the diverging understandings of the test task that learners display become part of how they are assessed and that certain task management strategies are rated less favorably than others. Our study holds promise for the fine-tuning of oral proficiency subskill ratings and raises questions as to the framing of test tasks, since this appears to have implications for student performance and evaluation.
---
paper_title: Testing of second language pragmatics: Past and future

paper_content:

Testing of second language pragmatic competence is an underexplored but growing area of second language assessment. Tests have focused on assessing learners’ sociopragmatic and pragmalinguistic abilities but the speech act framework informing most current productive testing instruments in interlanguage pragmatics has been criticized for under-representing the construct. In particular, the assessment of learners’ ability to produce extended monologic and dialogic discourse is a missing component in existing assessments. This paper reviews existing tests and argues for a discursive re-orientation of pragmatics tests. Suggestions for tasks and scoring approaches to assess discursive abilities while maintaining practicality are provided, and the problematicity of native speaker benchmarking is discussed.
---
paper_title: Interactional Competence across Proficiency Levels: How do Learners Manage Interaction in Paired Speaking Tests?

paper_content:

Paired speaking tasks are now commonly used in both pedagogic and assessment contexts, as they elicit a wide range of interactional skills. The current study aims to offer an investigation of the interaction co-constructed by learners at different proficiency levels who are engaged in a paired speaking test, and to provide insights into the conceptualization of interactional competence and key distinguishing interactional features across levels. The findings suggest that our understanding of interactional competence both in the classroom and as a construct underlying tests and assessment scales needs to broaden to include not just interactional features such as topic development organization, but also listener support strategies and turn-taking management. A more comprehensive understanding of interactional competence has the potential to complement available descriptions of interactional skills in assessment scales of speaking and aid learners and teachers in communicative classrooms.
---
paper_title: Peer interaction and corrective feedback for accuracy and fluency development

paper_content:

This quasi-experimental study is aimed at (a) teaching learners how to provide corrective feedback (CF) during peer interaction and (b) assessing the effects of peer interaction and CF on second language (L2) development. Four university-level English classes in Japan participated ( N = 167), each assigned to one of four treatment conditions. Of the two CF groups, one was taught to provide prompts and the other to provide recasts. A third group participated in only peer-interaction activities, and a fourth served as the control group. After one semester of intervention, the two CF groups improved in both overall accuracy and fl uency, measured as unpruned and pruned speech rates, whereas the peer-interaction-only group outperformed the control group only on fl uency measures. This study draws on monitoring in speech-production theory and the declarative-procedural model of skill-acquisition theory to interpret these results, thus contributing a new theoretical approach to CF research in the context
---
paper_title: Simulating conversations in oral-proficiency assessment: a conversation analysis of role plays and non-scripted interviews in language exams:

paper_content:

Several recent studies have investigated the nature of interaction in oral proficiency exams and have concluded that the interview format obscures differences in the conversational competence of the candidates. The present paper examines what opportunities test takers have to display their knowledge of managing conversations in the L2 in two types of tasks: non-scripted interviews and guided role-play activities. The data for the study consists of 30 interviews and 30 role-play activities between near-native examiners and intermediate learners used in language exams in Hungary. The interviews and role-plays have been analysed for the number of topics introduced and ratified by the examiner and the candidate respectively, as well as for the number of interruptions, openings and closings produced by the examiner and the candidate. The findings show that the conversational interaction is more symmetrical in the guided role-play activity with the candidates introducing and ratifying approximately the same number of topics as the examiners. In addition, the examinees have the opportunity to interrupt and hold the floor more effectively during the role-play activity and can demonstrate their knowledge of how to open and close a conversation. These findings suggest that guided role-play activities used in the study exhibit several characteristics of real-life conversations and therefore can be used for assessing the candidates’ conversational competence.
---
paper_title: 1. CONVERSATION ANALYSIS AND APPLIED LINGUISTICS

paper_content:

Conversation Analysis (CA) as a mode of inquiry is addressed to all forms of talk and other conduct in interaction, and, accordingly, touches on the concerns of applied linguists at many points. This review sketches and offers bibliographical guidance on several of the major relevant areas of conversation-analytic work—turn-taking, repair, and word selection—and indicates past or potential points of contact with applied linguistics. After covering these areas, we include a brief discussion of some key themes in CA's treatment of talk in institutional contexts. Finally, we discuss several established areas of applied linguistic work in which conversation analytic work is being explored—native, nonnative, and multilingual talk; talk in educational institutions; grammar and interaction; intercultural communication and comparative CA; and implications for designing language teaching tasks, materials, and assessment tasks. We end with some cautions on applying CA findings to other applied linguistic research contexts.
---
paper_title: Role-play in oral proficiency interviews: Interactive footing and interactional competencies

paper_content:

Abstract This study aims to discuss role-play activity in oral proficiency interviews (OPIs) in terms of its construct validity; that is, whether it correlates with what it is supposed to measure. The data for the analysis were obtained from 71 role-play activities conducted during an OPI. The analysis is based on conversation analytic (CA) methodology and invokes the analytic frameworks of interactive footing and interactional competencies. Conversational analyses performed on the data revealed that candidates executed not only the role-play instructions but also the interviewers’ explicit and implicit requirements of the next desirable action. In doing so, candidates employed and displayed their interactional competencies in role-play interactions. The role-play activity in the OPI being managed by the interviewer created an asymmetrical relationship between the interviewers and the candidates in terms of speaking rights (i.e., turn-taking and topical organization). Nevertheless, competencies that candidates displayed in performing a role-play activity did not seem different from those employed in ordinary conversations. Thus, role-play activity in OPIs, if recognized as an interactional phenomenon co-constructed by participants’ display of their turn-by-turn practical evaluation of each other's actions, seems to be a valid instrument for assessing the candidates’ performance of the given tasks.
---
paper_title: Interaction in paired oral proficiency assessment in Spanish

paper_content:

i DECLARATION iii PREFACE iv ACKNOWLEDGEMENTS v TABLE OF CONTENTS vi LIST OF TABLES AND FIGURES x LIST OF ABBREVIATIONS xi LIST OF ABBREVIATIONS xi CHAPTER 1 1 1.
---

Title: Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing

Section 1: Introduction
Description 1: This section should provide an overview of the topic, explain its significance, set the context, and outline the aims and scope of the review.

Section 2: Search Criteria and Delimitations
Description 2: This section should outline the search strategy and criteria used to select studies for the review, along with any limitations to the scope of the review.

Section 3: Search Results
Description 3: This section should present the findings of the search process, including the distribution of relevant studies across different journals and the decision-making process for study inclusion.

Section 4: Understanding the Context: Oral Proficiency Testing Formats
Description 4: This section should provide an overview of the most common formats for testing oral proficiency, referencing theoretical frameworks and major developments in the field.

Section 5: Candidate-Examiner Tests
Description 5: This section should discuss empirical studies focused on candidate-examiner tests, providing insights into how these tests function as social events and their specific features.

Section 6: Discourse and Social Interaction in Candidate-Examiner Tests
Description 6: This section should detail empirical studies on various dimensions of discourse and interaction within candidate-examiner tests, subdivided into identified topical themes such as the OPI as a unique social event, comparisons with other test conditions, examiner conduct, and specific interactional features.

Section 7: Discourse and Social Interaction in the Paired Format
Description 7: This section should review studies focused on paired oral proficiency tests, addressing themes such as test interaction and assessment, and the effects of the interlocutor on test performance.

Section 8: Interaction in Group OP Tests
Description 8: This section should focus on empirical studies of group oral proficiency tests, highlighting unique features of group interactions and interlocutor effects on test discourse and performance.

Section 9: Conclusion
Description 9: This section should summarize the key findings of the review, discuss the implications for language testing practices, and suggest areas for future research.