MULTILINGUALISM AND USING

LANGUAGE FOR SPECIFIC PURPOSES –

THE ROLE OF ASSESSMENT

Găzduită de Consorțiul UBB-ICR pentru Testarea Românei ca Limbă Străină (Universitatea Babeș-Bolyai și Institutul Cultural Român)

Programm

e

Program

11th–13th April 2018 11–13 Aprilie 2018

MULTILINGVISMUL ȘI UTILIZAREA

LIMBAJULUI PENTRU SCOPURI SPECIFICE – ROLUL EVALUĂRII

51st ALTE Meeting and Conference Day

A 51-a Întâlnire și Conferință ALTE

Hosted by BBU–RCI Consortium for Testing Romanian as a Foreign Language (Babeş-Bolyai University and the Romanian Cultural Institute)

51

Bun venit la a 51-a Întâlnire și Conferință ALTE

Universitatea Babeș-Bolyai (UBB), alături de Institutul Cultural Român (ICR) vă urează bun venit la Cluj-Napoca, la a 51-a Întâlnire și Conferință ALTE!

Consorțiul UBB-ICR este unul dintre cei mai noi membri cu drepturi depline ALTE. Suntem mândri și onorați să film prima și, pentru moment, singura instituție care reprezintă limba română în această prestigioasă asociație profesională. Examenele noastre sunt administrate deocamdată local, dar intenția Consorțiului este să extindă aplicarea acestora în diverse locații din lume, oferind certificate pentru nivelurile A1-B2, cu garanția validității și fiabilității examenelor produse.

Dacă cererea de teste de limbă de calitate și adaptate la nevoile candidaților a fost tot mai mare în timp, testatorii de limbi se confruntă în prezent cu o nouă provocare, generată de creșterea fenomenului de migrație în toată lumea, precum și de o mobilitate accentuată a populației, în general. În aceste condiții, integrarea pe piața muncii sau într-un sistem nou de educație poate fi facilitată de obținerea unui certificat de competență lingvistică adaptat domeniului vizat, iar testele de limbi pentru scopuri specifice devin tot mai necesare.

A 51-a Întâlnire și Conferință ALTE vine în întâmpinarea discuției pe marginea acestei necesități cu considerații teoretice, dar și cu sugestii și soluții, cu instrumente și metode care să asigure un răspuns practic și rapid acestei nevoi tot mai presante pe piața testării limbilor. Suntem convinși că programul pregătit pentru sesiunile de conferințe și de ateliere va avea capacitatea de a stimula discuții incitante și fructuoase, cu unicul scop de a găsi răspunsuri adaptate la contextul propriu și la nevoile studenților/candidaților noștri.

Clujul vă primește cu sufletul deschis pentru o dezbatere intensă și de înaltă calitate, dar și pentru o întâlnire - prima, pentru mulți dintre dumneavoastră - cu realitatea și mitologia unei Transilvanii în același timp veche și modernă, misterioasă și curioasă, tradițională și deschisă spre nou.

Dacă vă putem ajuta cu orice informație sau sfat, vă stăm cu drag la dispoziție.

Bine ați venit la Cluj!

Professor Ioan-Aurel Pop, Ph.D. Member of the Romanian Academy

Welcome to the 51st ALTE Meeting and Conference Day

Babeș-Bolyai University (BBU) together with the Romanian Cultural Institute (RCI) welcome you to Cluj-Napoca, for the 51st ALTE Meeting and Conference Day!

BBU-RCI Consortium is one of the newest full members of ALTE. We are proud and honoured to be the first and for the moment the only institution representing the Romanian language in this prestigious professional association. Currently, our examinations are administered locally, but the intention of the Consortium is to apply them in different locations in the world, offering certificates of linguistic competence for the CEFR Levels A1–B2 based on valid and reliable examinations.

While the necessity of high-quality language tests adapted to the test takers’ needs has increased over time, test providers currently face a new challenge, generated by a growing migration phenomenon all over the world and by an expanded mobility of the population in general. In these conditions, integration into the labour market or in a new education system can be facilitated by obtaining a certificate of linguistic competence adapted to the target domain. Consequently, language for specific purposes (LSP) language tests are more and more in demand.

The 51st ALTE Meeting and Conference Day joins the debate on this necessity with theoretical considerations but also with suggestions and solutions, with instruments and methods able to ensure a practical and fast answer to this more and more pressing need in the language testing market. We are confident that the programme of the conference and workshop sessions will stimulate inciting and fruitful discussions with the sole purpose of finding adapted answers to our own context and to our students’/test takers’ needs.

Cluj-Napoca welcomes you warmly for an intense and high quality debate, but also for a meeting – the first one, for many of you – with the reality and mythology of Transylvania, a region at the same time antique and modern, mysterious and curious, traditional and open to the new.

If there is any information or advice we can help you with, we are here for you.

Welcome to Cluj!

Dr Nick Saville ALTE Secretary-General

Liliana Ţuroiu President of the Romanian Cultural Institute

1.

51st ALTE Meeting and Conference Wednesday 11th April 2018

All the sessions will take place on the ground floor at the Faculty of Letters, 31 Horea St..

Time Session Room

08.45 – 09.00 Registration Room 38

09.00 – 11.00 Standing Committee Meeting(elected members of committee only)

Council Room

11.00 – 11.15 Coffee Room 37

11.15 – 13.15 Standing Committee Meeting continues Council Room

13.15 – 14.30 Lunch Capitoliu Bistro

14.30 – 16.00 Executive Committee Meeting(elected members of committee only) Council Room

16.00 – 16.15 Coffee Room 37

16.15 – 18.00 Executive Committee Meeting continues Council Room

18.30 – 19.30 City walk with professional guide Meeting point: UniriiSquare at 18.20

19.45 – 21.00 Wine tasting The Faculty of Letters, Popovici Room

2.

51st ALTE Meeting and Conference Thursday 12th April 2018

All the sessions will take place on the ground floor at the Faculty of Letters, 31 Horea St..

Time Session Room

08.30 – 09.00 Registration Room 38

09.00 – 09.10Welcome and housekeepingGraham Seed, ALTE Manager

Eminescu Room

09.10 – 09.20

Official openingAssociate Professor Elena Platon, Director of the Department of Romanian Language, Culture and Civilisation

Eminescu Room

09.25 – 11.00

Parallel SIGs

LSP SIG (any ALTE delegate) Room 39

LAMI SIG (any ALTE delegate) Eminescu Room

11.00 – 11.30 Coffee Room 37

11.30 – 13.00

Parallel SIGs

QMS working group and SIG on Technology in Language Assessment (joint SIGs) (any ALTE delegate)

Eminescu Room

Special Requirements and Circumstances SIG (any ALTE delegate) Room 39

CEFR SIG (any ALTE delegate) Council Room

13.00 – 14.30 Lunch Capitoliu Bistro

14.30 – 16.00

Parallel workshops

John Pill, Lancaster University, UKBalancing practicality with specificity and authenticity in LSP test development

Council Room

Antony Kunnan, University of Macau, ChinaLearning about assessment knowledge throughhypothetical scenarios

Eminescu Room

16.00 – 16.30 Coffee Room 37

16.30 – 16.45 Reports from SIGs Eminescu Room

16.45 – 18.30 Annual General Meeting (ALTE Members only) Eminescu Room

19.30 – 23.00 Conference dinnerPiramida RestaurantMeeting point: UniriiSquare at 19.15 3.

51st ALTE Meeting and Conference Friday 13th April 2018

Multilingualism and using language for specific purposes – the role of assessment

All the sessions will take place on the ground floor at the Faculty of Letters, 31 Horea St..

Time Session Room

08.30 – 09.00 Registration Room 38

09.00 – 09.10WelcomeRepresentatives from the ALTE Secretariat, Babeş-Bolyai University and the Romanian Cultural Institute

Eminescu Room

09.10 - 09.20 Opening addressDr Nick Saville, ALTE Secretary-General Eminescu Room

09.20 – 10.00 Professor Antony Kunnan, University of Macau, China Assessing language for specific purposes in the 21st century Eminescu Room

10.00 – 10.40Dr Dina Vîlcu, Babeș-Bolyai University, RomaniaDr Koen Van Gorp, Michigan State University, USADeveloping resources for LSP tests: A reflection

Eminescu Room

10.40 – 11.10 Coffee and exhibition Lobby and Room 37

11.10 – 11.50 John Pill, Lancaster University, UK Language tests to enter the workplace: Perspectives from stakeholders Eminescu Room

11.50 – 12.30Lorenzo Rocca, CVCL, University for Foreigners of Perugia, Italy Language tests and the job market within the migration context: cui prodest?

Eminescu Room

12.30 – 14.00 Lunch Capitoliu Bistro

14.00 – 14.40Dr Brigita Séguis, Cambridge Assessment English, UK Language assessment of health professionals: the issue of communicative competence

Eminescu Room

14.40 – 14.50 Closing remarks and introduction to the workshopsDr Nick Saville, ALTE Secretary-General Eminescu Room

15.00 – 16.30

Parallel workshops

Dr Bart Deygers, CNaVT, BelgiumUniversity admission L2 tests: Knowing what you need to measure, and actually measuring it

Popovici Room

Dr Koen Van Gorp, Michigan State University, USADr Dina Vîlcu, Babeş-Bolyai University, RomaniaDeveloping LSP tests for different contexts: Exploring the usability of the ALTE LSP Guidelines

Room 39

Dr Brigita Séguis, Cambridge Assessment English, UKHealthcare communication skills in LSP speaking tests Council Room

4.

Parallel workshops on Thursday

John Pill, Lancaster University, UK

Balancing practicality with specificity and authenticity in LSP test development.

Developers of LSP tests are likely to be familiar with the conflict between a desire to make the best test possible and the constraint of limited resources. In this workshop, participants will consider how to address this tension in a principled way, taking into account, for example, a proposed test’s context and purpose, the stakes involved for test takers, and the available time, materials, expertise, and access to the domain of target language use. They will reflect on the range of options on hand to manage each stage of a test development project and, in doing so, recognise where flexibility of approach is possible and defensible, depending on particular circumstances. The process is intended to increase participants’ confidence in making decisions related to test development.

Professor Antony Kunnan, University of Macau, China

Learning about assessment knowledge through hypothetical scenarios.

Assessment literacy papers and workshops have focused on how to make participants understand concepts written as standards. One popular set of standards are the ones written by psychological and educational experts and best practices in the field, Standards for educational and psychological testing (APA, AERA and NCME 1999 and 2014). The standards include validity, reliability, generalisability, fairness, norm and criterion-referenced assessment, etc. These standards have provided assessment institutions guidance for their own internal evaluations and research agendas. But, students and young professionals are expected to understand these concepts without guiding principles. More recently, the argument-based approach based on Toulmin’s model (for example, Bachman and Palmer 2010) has offered a systematic approach to evaluation with an examination of an assessment institution’s claims and warrants, and the backing for warrants. But, neither does this approach offer guiding principles. Thus, these two top-down approaches are unlikely to be able to help participants understand key assessment knowledge.

In order to remedy this situation, I propose a series of reflections on hypothetical scenarios to understand the key concepts of assessment knowledge. In this approach, first, a series of scenarios called “The Trolley Problem” (Foot 1967) from moral philosophy will introduce how to evaluate these scenarios – whether to morally justify actions based on the principle of outcomes or consequentialism or to use the principle of duty or obligation. Second, six scenarios from language assessment on defective tasks, biased tasks, scoring problems, selecting an assessment, differential pricing, and decision-making will be analysed by applying outcomes-based or dutybased thinking. As these scenarios mirror the common assessment development to assessment decision-making process, applying principles to these scenarios will be quite transparent. Third, participants will formalize their understanding by checking a list of concepts/standards that include different aspects of validity, reliability, and fairness. Therefore, this approach will help participants understand key assessment knowledge through a bottom-up approach (for more details see Kunnan’s Evaluating Language Assessments, Routledge, 2017).

5.

Plenary presentations on Friday

Professor Antony Kunnan, University of Macau, China

Assessing language for specific purposes in the 21st century.

Assessing language for specific purposes (LSP) is about making inferences from specific purpose assessments about language abilities in target situations such as English for air traffic controllers, Italian for opera singers, French for culinary trainees, Mandarin for tour guides, Sanskrit for Buddhists, etc. One of the earliest LSP assessments was the British General Medical Examination (in English) for physicians from outside the UK in 1975. O’Sullivan (2006) reminds us that early work on LSP was driven by research focusing on the identification of unique instances of language use in specific contexts, the issue of authenticity in the use of materials for teaching, and the central place of needs analysis in identifying the specific language needs of learners in given contexts. Douglas (2000) examined the concept of authenticity, both ‘situational’ and ‘interactional’ in LSP assessments. In contrast, today the priority is to see language learning and assessment not as compartmentalised discrete categories but as part of a learner’s plurilingual repertoire relevant to the specific context they find themselves in (Saville forthcoming). It is in this spirit that LSP and multilingualism, the theme of this conference day, come together.

Dr Dina Vîlcu, Babeș-Bolyai University, Romania

Dr Koen Van Gorp, Michigan State University, USA

Developing resources for LSP tests: A reflection.

Nowadays, the world is witnessing increased and accelerated migration waves, creating super diverse societies. Participating through work and study helps newly arrived migrants to integrate in their new contexts. Policy makers and other stakeholders often consider having a certain language proficiency level key to giving migrants access to labour and study. Getting the right language certificate is therefore of the utmost importance. LSP tests that cater to the different contexts migrants need to gain access to might be the best and fairest way forward. However, LSP tests pose their own challenges. While instruments like the CEFR and the numerous documents created around it ensure a solid basis for the construction of general language tests, the LSP assessment field is in great need of acquiring guidelines for test development. This presentation focuses on aspects typical to LSP tests, like specificity and authenticity, and suggests instruments to guide the production of LSP tests.

6.

Plenary presentations on Friday

John Pill, Lancaster University, UK

Language tests to enter the workplace: Perspectives from stakeholders.

This talk considers the use of language tests to control access to the workplace in a variety of contexts. It draws on findings from recent studies focusing on the perspectives of different stakeholders in this process, including test users, such as governments and professional bodies, test takers, and the general public. As an example, ongoing changes in the registration processes for healthcare practitioners seeking to work in the UK illustrate how political and social pressures can affect which language tests are viewed as appropriate and how cut-scores (pass/fail) are decided. The discussion highlights fundamental issues in research on language for specific purposes: the extent to which language and communication skills are recognised and valued, and how they are perceived to relate to professional knowledge and skills. Views on these issues influence, in turn, the scope and specificity expected of language tests used as gate-keeping mechanisms for various professional domains.

Lorenzo Rocca, CVCL, University for Foreigners of Perugia, Italy

Language tests and the job market within the migration context: cui prodest?.

The contribution aims to investigate the relations between workplace and language assessment within the adult migrant population in Europe.

In the first part, a theoretical framework focusing on the issue of distinguishing LSP from general language (Douglas 2000, Elder 2001, O’Sullivan 2006) will be proposed both in terms of content and context validity (Weir 2004), particularly concerning KoS elements related to the occupational domain, as included in many tests for migration purposes.

In the second part, research in the field (Council of Europe 2017) will be analysed, taking into account critical aspects such as: the interpretation of test results by employers; the tension between language competences and competences requested at the workplace; the perceptions of stakeholders about the Can Dos needed in order to take a job; and to what extent language tests manage to deal with the expectations provided.

The third part will present preliminary data of a case study conducted by CVCL with regard to the impact that different forms of assessment have had in the mid-term for access to the labour market, according also to a sociological perspective.

7.

Plenary presentations on Friday

Dr Brigita Séguis, Cambridge Assessment English, UK

Language assessment of health professionals: the issue of communicative competence.

Typically, LSP tests have been construed as assessing language for occupational/professional or academic purposes. They are distinguished from more general language tests by task authenticity, as well as the interaction between language knowledge on the one hand, and specific purpose knowledge on the other hand (Douglas 2000). One of the occupational domains that has been experiencing a steady growth in the number of applicants is that of healthcare, where various language proficiency measures have been in place to evaluate an individual’s ability to practise safely and effectively in a healthcare environment. One of these measures is language performance tasks aimed at simulating real-world, authentic scenarios, which raises an important question of the extent to which linguistic and professional skills can be completely detached from each other.

Clinical communication can be described as one of the fundamental professional skills that healthcare practitioners rely on in their day-to-day work. It forms the core of every medical interview and serves as the link in building an effective relationship between the health professional and the patient. Given that the speaking component of a typical occupational language test normally involves a role-play simulating the real-world medical interview, the issue that language test developers face is whether we should separate clinical communication from linguistic competence, or whether the two can be combined and effectively assessed as part of a speaking performance. In this presentation I will argue in favour of the latter, and demonstrate how this can be achieved in practice with reference to the Occupational English Test.

8.

Parallel workshops on Friday

Dr Bart Deygers, CNaVT, Belgium

University admission L2 tests: Knowing what you need to measure, and actually measuring it.

During this workshop we will consider the assessment of languages for academic purposes (LAP), which has a lot of similarities with LSP testing. Nevertheless, there are a few important differences in terms of constructs and task selection, which we will discuss in this entry-level workshop. The workshop will begin by determining the purpose of university admission testing for international L2 students and by listing which conditions might make a university admission policy effective, fair, valid, and just. We will compare different ways in which to measure students’ language proficiency, and introduce methodologies that can be used to determine an adequate threshold language level. Participants can expect to gain knowledge and insights regarding the construct and measurement of LAP, admission policy effectiveness, and responsible use of tests. During the workshop, we will use real-world test data collected in a 4-year research project in Flanders, Belgium.

Dr Koen Van Gorp, Michigan State University, USA

Dr Dina Vîlcu, Babeş-Bolyai University, Romania

Developing LSP tests for different contexts: Exploring the usability of the ALTE LSP Guidelines.

The Manual for Language Test Development and Examining was produced by ALTE on behalf of the Council of Europe as an instrument to be used together with the CEFR. Providing guidance and flexibility at the same time, this manual was intended to help users in constructing proficiency tests in a principled and non-prescriptive manner (Sheils 2011). This is one of the many resources meant to provide, along with the CEFR, solid guidance in constructing general language tests – and one of the most useful ones.

Recently, the need to develop LSP tests has grown, generated by the large migration waves, with people striving to integrate in new and challenging work and education contexts. In order to help the production of LSP tests, the ALTE Language for Specific Purposes Special Interest Group (LSP SIG) produced a supplement to the Manual for Language Test Development and Examining. This supplement provides concrete guidelines for test developers that might have experience in constructing general language tests, but that are (relatively) new to developing LSP tests. The guidelines not only discuss the most important test concepts in LSP assessment, but also provide examples and stepwise advice on how to develop any LSP test.

In this workshop, we will familiarise the audience with this easy-to-use instrument and support them in their first steps of designing an LSP test fitting their own context and needs. The processes of teaching, learning and assessing LSP are progressively diversifying from the point of view of domains and also of languages. Drawing on the participants’ experience and specific contexts, and on their students’ needs, we will help them to identify the best ways in which this new instrument can be used to build useful and relevant LSP tests.

9.

Parallel workshops on Friday

Dr Brigita Séguis, Cambridge Assessment English, UK

Healthcare communication skills in LSP speaking tests.

The aim of this workshop is to focus on testing healthcare communication skills in LSP speaking tests. We will start the discussion with an overview of the types of communication skills that medical professionals draw on in their day-to-day work, and we will then consider how we can include them in the speaking component of an LSP test. Then, through a number of exercises and activities, we will attempt to develop appropriate assessments for the different types of clinical communication that we have identified and think of how they can be assessed alongside linguistic criteria traditionally included in speaking sub-tests. Finally, we will also consider the potential issues and problems that can arise as a result of incorporating clinical communication into a language test, including the impact that this can have on the candidate’s performance and on the overall reliability and validity of the test.

About the speakers

John Pill

John Pill is a lecturer in the Department of Linguistics and English Language at Lancaster University, UK. He teaches on the MA in Language Testing programme and is involved in Lancaster’s research partnership with Trinity College London. He completed his doctoral studies at the University of Melbourne. Most recently, he worked at the American University of Beirut, where he set up a new English language admission test. His research interests include specific purpose language testing, particularly in healthcare and academic contexts, speaking assessment, and language assessment literacy. He is the secretary of the UK Association for Language Testing and Assessment.

10.

Professor Antony Kunnan

Professor Antony Kunnan is Professor of English at the University of Macau, China, and specialises in teaching and researching language assessment matters. He has published widely in language assessment areas such as development, statistics, fairness and justice, ethics, immigration and citizenship, multilingualism and public policy. His latest edited books are Language Testing and Assessment (Routledge, 2014) and The Companion to Language Assessment (Wiley, 2014) and authored volume Evaluating Language Assessments (Routledge, 2017). He was the founding editor of Language Assessment Quarterly. His previous plenaries at ALTE were for conferences in Barcelona, Berlin, Budapest, Prague and Sofia.

Dr Koen Van Gorp

Dr Koen Van Gorp is Head of Foreign Language Assessment at the Center for Language Teaching Advancement (CeLTA) at Michigan State University, USA. He teaches language assessment courses in the Master of Arts in Foreign Language Teaching (MAFLT) Program and the Second Language Studies PhD Program. He is also a Research Fellow at the Centre for Language and Education, University of Leuven, Belgium. He is member of the Editorial Board of ITL – International Journal of Applied Linguistics – and co-editor of two recent volumes on multilingualism: The Multilingual Edge of Education (Palgrave 2018) and Language Awareness in Multilingual Classrooms in Europe: From Theory to Practice (De Gruyter Mouton 2018). Key interests include task-based language teaching and assessment, classroom-based language assessment, curriculum development, language awareness and multilingual education.

Dr Dina Vîlcu

Dr Dina Vîlcu teaches and assesses Romanian as a foreign language (RFL) at the Department of Romanian Language, Culture and Civilisation, Babeș-Bolyai University, Cluj-Napoca. Besides the general language courses, she teaches specialised language courses (socio-humanistic and artistic language), Romanian culture and civilisation, and tailor-made courses for interpreters and translators. She co-ordinated the assessment segment in the 3-year project RLNM (Romanian as a second language) and was in charge of the auditing process of the RFL examinations (CEFR Levels A1, A2, B1 and B2) by ALTE. The positive result of this auditing process brought Babeș-Bolyai University, together with the Romanian Cultural Institute, the status of full member of this prestigious professional association. She co-ordinated the second issue for the year 2017 of Studia Universitatis Babeș-Bolyai, Philologia, dedicated to assessing RFL, and other volumes on the topic. Besides research in the domain of language assessment, she is interested in the challenges involved in teaching, learning and assessment of LSP. She is currently the chair of the LSP SIG of ALTE. Another dimension of her research activity is general linguistics, in connection with her PhD thesis.

11.

Lorenzo Rocca

Lorenzo Rocca has a degree in Classics and a postgraduate degree in the Didactics of L2 Italian. After having been a teacher of Italian as a Foreign Language, he has worked at the CVCL (Centre for Evaluation and Language Certification), Italy, since 2004. His duties range from marking and examining to item writing and running seminars. Since 2006 he has co-ordinated research projects focused on the link between teaching and evaluation in the migration context. He has published the specifications for the CVCL exams for adult migrants, a KoS book and the Italian descriptors for low literacy profiles. Lorenzo has been the chair of LAMI (Language Assessment for Migration and Integration) in ALTE since 2008 and he has been a member of the Council of Europe LIAM (Linguistic Integration of Adult Migrants) project since 2014.

Dr Brigita Séguis

Dr Brigita Séguis is a Senior Research Manager at Cambridge Assessment English, UK. She completed her doctoral studies at the University of Oxford, where she specialised in code-switching, discourse analysis and corpus linguistics. Prior to joining Cambridge Assessment English, she worked for the NHS, as well as the University of Oxford and University College London. Her research interests include language testing for specific purposes, particularly in the healthcare context, predictive validity of admissions tests, impact studies and multilingualism.

Dr Bart Deygers

Dr Bart Deygers is a language testing researcher at the University of Leuven, Belgium, funded by the Research Foundation Flanders (FWO). He is the chair of the CEFR SIG in ALTE. His primary research interests include justice, validity, the CEFR, and university entrance language testing. Recent articles have appeared in Language Testing, Language Assessment Quarterly, Assessing Writing, and ITL – International Journal of Applied Linguistics.

12.

5113.