RATER PERCEPTIONS IN TESTS OF ORAL EXPRESSION

Przemysław Krakowian

Abstrakt


Abstract
Assessment of spoken performance is often viewed in terms of various different
categories relating to the language aspect of performance including provisions for vocabulary
use and linguistic resources, as well as accuracy of use, contrasted with fluency. Such
categories are often coupled with several other categories relating to other aspects of spoken
performance, and amongst those one almost always featured in rating schemes is that
of pronunciation. It is interesting to observe what happens when this category is omitted in
rating scales and how this affects the perceptions of oral examiners. In this investigation,
data from an Electronic Performance Support System (EPSS) obtained in training and
fine-tuning examiners of oral performance in tests of spoken performance in English from
different educational contexts is used to draw some intriguing research conclusions.
Key words: Electronic Performance Support System (EPSS)
Streszczenie
Systemy oceniania języka mówionego posługują się rozmaitymi skalami zawierającymi
różne kategorie odnoszące się do jakości języka zawierające podkategorie obejmujące
słownictwo i środki językowe, jak również poprawność językową zwykle na tle biegłości
i potoczystości w mówieniu. Te, z kolei, uzupełnione bywają rozlicznymi jeszcze
kategoriami, wśród których właściwie zawsze poczesne miejsce zajmuje wymowa. Warto
jednak przyjrzeć się sytuacji, gdzie tej kategorii brak w skalach oceniania i w jaki sposób
wpływa to na percepcje egzaminatorów języka mówionego. W tym badaniu dane pochodzące
z Elektroniczne Systemu Wspierania Oceniania (ang. EPSS) zgromadzone w trakcie
treningu i doszkalania egzaminatorów języka angielskiego z różnych kontekstów edukacyjnych
pozwalają na sformułowanie intrygujących wniosków badawczych
Słowa kluczowe: Elektroniczny system Wspierania Ocen (EPSS),


Pełny tekst:

PDF

Bibliografia


REFERENCES

Bachman L.F., and Palmer, A. 2002. Language Testing in Practice. Oxford: University Press.

Barna M.L. 1994. Stumbling Blocks in Intercultural Communication. In Intercultural Communication,

eds. L.A. Samovar and R.E. Porter. Wadsworth.

Bennet M.J. 1993. Towards Ethnorelativism: A Developmental Model of Intercultural Sensitivity.

In Education for the Intercultural Experience. Yarmouth: Intercultural Press.

Bond T.G., and Fox, C.M. 2007. Applying the Rasch Model. Fundamental Measurement in the

Human Sciences. Toledo: University of Toledo Press.

Brown A., 2006. “An examination of the rating process in the revised IELTS Speaking Test”.

IELTS Research Reports Volume 6, IELTS Australia, Canberra and British Council, London.

Cavé C., Guaïtella I., and Santi S. 2002. “Eyebrow movements and voice variations in dialogue

situations”. In Proceedings of the 7th International Conference on Spoken Language Processinig,

eds. Hansen, J.H.L and Pellom, B.

Chambers L., Ingham K. 2011. “The BULATS Online SpeakingTest”. ESOL Research Notes,

vol 34.

De Velle S. 2008. “The revised IELTS Pronunciation scale”. ESOL Research Notes, vol 34.

Fulcher G. 2003. Testing Second Language Speaking. Harlow: Pearson Longman.

Guaïtella I., Santi S., Lagrue B., Cavé Ch. 2009. “Are Eyebrow Movements Linked to Voice

Variations and Turn-taking in Dialogue? An Experimental Investigation”. Language and

Speech 57.

Hall E., and Hall M. 1990. Understanding cultural differences: Germans, French and Americans.

Verlag: Intercultural Press.

Hall E. 1959. The silent language. Garden City: Doubleday.

Hall E. 1966. The hidden dimension. Garden Day: Doubleday Anchor Books.

Hawkey R. 2004. “A Modular Approach to Testing English Language Skills: The development

of the Certificates in English Language Skills, CELS, examinations”. Cambridge ESOL

Research Notes Volume 16.

Hildreth P.M. and Kimble Ch. 2004. Knowledge networks: innovation through communities of

practice. Idea Group Inc.

Hubbard C., Gilbert S. and Pidcock J. 2006. “Assessment processes in speaking tests: a pilot

verbal protocol study”. ESOL Research Notes, vol 24.

Hubbard Ch. 2011. “Cambridge ESOL Professional Support Network Extranet: Development

and impact”. ESOL Research Notes, vol 49.

Hymes D. 1964. Introduction: “Toward Ethnographies of Communication”. American Anthropologist

, 6.

Hymes D. 1972. On communicative competence. In Sociolinguistics, eds J. B. Pride and J.

Holmes. Harmondsworth: Penguin.

Krahmer E. and Swerts M. 2004. More about brows. In From brows to trust: Evaluating embodied

conversational agents, eds. Ruttkay, Z. and Pelachaud, C., Norwell: Kluwer Academic

Press.

Krakowian P. 2010. Modern Test Theory Explained. Warszawa: Scholar.

Krakowian P. 2011. Investigating Rater Performance in Tests of Oral Expression. Łódź: Wydawnictwo

Uniwersytetu Łódzkiego

Lustig M.W. and Koester J. 1993. Intercultural Competence. Interpersonal Communication

across Cultures. New York: Harper Collins College Publishers.

Lustig M.W. and Koester J. 2009. Intercultural Competence: Interpersonal Communication

Across Cultures. Boston: Allyn and Bacon.

Martinez L. 2009. “How Examiners of Different Severity Grade Candidates of Different Ability”.

Test Insights 2009. Measurement Research Associates, Inc.

Martinez L. 2010. “The Relationship between Examiner Severity and Consistency”. Test Insights

Measurement Research Associates, Inc.

Nakane I. 2007. Silence in Intercultural Communication: perceptions and performance. Amsterdam:

John Benjamins.

O’Sullivan B. 2008. Modelling Performance in Tests of Spoken Language. Frankfurt: Peter

Lang

Ockey G. J. 2009. “The effects of group members’ personalities on a test taker’s L2 group oral

discussion test scores”. Language Testing, 26.

Orr M. 2002. “The FCE Speaking test: using rater reports to help interpret test scores”. System,

vol 30, no 2.

Saint-Onge H. and Wallace D. 2003. Leveraging communities of practice for strategic advantage.

Oxford: Butterworth-Heinemann.

Scollon R. and Scollon S.W. 1995. Intercultural Communication. Oxford: Blackwell

Seed G. 2012. “Perceptions of authenticity in academic test tasks”. ESOL Research Notes, vol

Seiter J., Weger H., Jensen A. and Kinzer H. 2010. “The Role of Background Behavior in Televised

Debates: Does Displaying Nonverbal Agreement and/or Disagreement Benefit Either

Debater?” Journal of Social Psychology Vol. 150, No. 3.

Taylor L., and Falvey, P. 2007. “IELTS Collected Papers: Research in speaking and writing

assessment”. Cambridge ESOL Research Notes Volume 19.

Tynan R. 2015. “Creating ePortfolios to facilitate and evidence progress using learning technologies”.

Cambridge Exams Research Notes, vol 61

Weir C. and Milanovic M. 2003. “Continuity and Innovation: Revising the Cambridge Proficiency

in English Examination 1913-2002”. Cambridge ESOL Research Notes issue 15

Wenger E. 1998. Communities of Practice: Learning, Meaning, and Identity. Cambridge: University

Press.

Wenger E., McDermott R.A. and Snyder W. 2002, Cultivating communities of practice: a guide

to managing knowledge. Harvard: Business Press

Wilson M. 2005. Constructing Measures: An Item Response Model, Mahwah: Lawrence Erlbaum

Associates

Wright B.D. and Masters G. 1982. Rating Scale Analysis. San Diego: MESA Press

Wright B.D. and Stone M.H. 1979. Best test design. San Diego: MESA Press


Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.

Redakcja czasopisma "Społeczeństwo. Edukacja. Język"

Wydział Nauk Humanistycznych i Społczenych 

Państwowa Wyższa Szkoła Zawodowa w Płocku

ul. Gałczyńskiego 28

09-402 Płock