The International Test Commission - 11th conference of On the Beat of Testing
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
On the Beat of Testing 11th conference of The International Test Commission July 2-5, 2018 • Montréal, Canada
For every learner, at every stage We are the world’s learning company, on a mission to help people make progress in their lives through learning. Having delivered over 100 million online assessments, learning outcomes are at the heart of everything we do. Our world-class assessment solutions are designed to develop new, personalized ways of learning. Because where learning flourishes, so do people. www.pearson.com Illustration by Lucy Vigrass
Improving the science and practice
of testing
Improving theand assessment
science and practice
of testing and assessment
PSYCHOMETRIC CONSULTING
PSYCHOMETRIC CONSULTING
• Expert psychometric services
• Expert psychometric services
• Independent verification
• Independent processes
verification processes
• Improved
• Improved quality
quality ofofproprietary
proprietary testing programs
testing programs
ASSESSMENT LITERACY
ASSESSMENT LITERACY
• Providing instructional and educational resources
• Providing instructional
• Informing and educational
selection, development, resources
and use of tests
• Promoting
• Informing responsible
selection, test use
development, and use of tests
• Promoting responsible test use
TEST REVIEWS & INFORMATION
• Authoritative reference materials
TEST •REVIEWS & INFORMATION
Mental Measurements Yearbooks
• Tests in Print
• Authoritative reference materials
• Pruebas Publicadas en Español
• Mental Measurements Yearbooks
• Tests in Print
buros.org
• Pruebas Publicadas en Español
buros.orgConnecting talent and aspiration with opportunity The Graduate Management Admission Council (GMAC) is a global, nonprofit association of leading graduate business schools. We are committed to the work you do every day. Our GMAC suite of assessments and services addresses the diverse needs of graduate management education professionals and is built on decades of experience and rich data. Our Market Intelligence provides actionable insights on issues facing the industry, including data trends, the latest research, survey reports, customized analytics and white papers. Visit gmac.com for more information and current research © 2018 Graduate Management Admission Council (GMAC). All rights reserved. Graduate Management Admission Council® is a registered trademark of GMAC in the United States and other countries.
ITC Presidential Address
Dear Colleagues and Friends,
The ITC is an association of national psychological
associations, test commissions, publishers and
other organizations. It also has a large individual
member base. Membership covers most of the
Western and Eastern European Countries and
NorthAmerica, as well as countries in the Middle and Far East, and South Africa.
The ITC is committed to promoting effective testing and assessment policies and
to the properdevelopment, evaluation and uses of educational and psychological
instruments. The ITC facilitates exchanges of information among members and
others; facilitates cooperation toward the solution of problems associated with test
development and use; and in otherways explores issues related to the construction,
distribution, and use of tests, scales, and other methods used to assess personal
and group qualities. The biennial conference of the International Test Commission
has now been for many years a premier event in the world of psychological and
educational testing. Practitioners and researchers dedicated to testing and
assessment, test developers, testing companies, test publishers, regulators and
policy makers: we all come together once every two years to discuss important
international, regional and national developments in testing and assessment,
to share ideas and to contribute to the evolutions of tomorrow in this important
domain of knowledge and practice. I am delighted to welcome you all to the 11th
Conference of the ITC in July 2018, in Montreal, Canada - a vibrant, fascinating
and multicultural city, which has so much to offer. Just before the ITC Conference,
Montreal will host the 29th International Congress of Applied Psychology (June 26-
30, 2018), and during the same period will also be host to the renowned jazz festival
(June 28-July 7, 2018). The ITC conference will surely be the jewel in this intellectual
cornucopia. Who could ask for more? We cordially invite you to register for your
attendance at the ITC2018, which promises to be a fantastic scientific event - the
highlight of the year in testing and assessment. I am looking forward to meeting you
in Montreal in the summer of 2018!
Dragos Iliescu
ITC President
July 2-5, 2018 • Montréal, Canada • itc-conference.com 3Dear colleagues,
I am very pleased to invite you to join us for the 11th Conference of the International
Test Commission, which will be held in Montreal from July 2 to 5, 2018. You will love
Montreal, there is no doubt about it! This major city combines European charm with
the energy that is characteristic of North America, from its cutting-edge industries
to its renowned art of living. During the summer season, it offers amazing sights
and a night life that spreads out into the streets. That is why we chose to put the
conference under the banner of music, with the Jazz Festival taking place in early
July. This festival has existed for more than 30 years and has been named the
world’s largest by Guinness World Records. Music enthusiasts will be delighted by
this event. We have also decided that jazz should set the tone for the conference, as
demonstrated by our chosen theme: On the beat of testing. Our scientific program
is organized around four complementary sub-themes, which are as follows: a)
Best practices in testing and assessment, b) Instruments and their validity, c)
Effectiveness and efficiency including application in multicultural environments
and d) Challenges, innovation and advances in technology. These sub-themes
will be addressed in many ways, including by distinguished speakers and through
innovative workshops. Diversity, a value Quebecers hold dear, is reflected by the
range of speakers, their research and application fields, the issues they explore and
the countries they come from to offer you an extremely rich program. The hotel
that will host the participants is located in the heart of Montreal, close to the Jazz
Festival, excellent restaurants, shops, numerous museums, the cobblestone streets
of Old Montreal and a breathtaking view of the Saint Lawrence River from the Old
Port. This exceptional location will let you enjoy a variety of activities in the city and
explore Montreal’s unique multicultural atmosphere, a subtle mix of North American
and European cultures. We have also kept in mind the early birds among you: they
will be offered a 5 km run or a 3 km walk to experience the city in the early morning
hours. My team and I look forward to welcoming a great number of you in Montreal
for an exceptional conference in July 2018.
Welcome message
Nathalie Loye
Chair of the Organizing Committee
4 11th conference of The International Test CommissionMessage de bienvenue
Chers collègues,
C’est avec un immense plaisir que je vous invite à vous
joindre à nous pour la 11e édition de la Conférence
de l’International Test Commission qui aura lieu du
2 au 5 juillet 2018 à Montréal. Aucun doute n’est
possible, vous adorerez Montréal! Cette grande
métropole combine le charme européen à l’énergie qui
caractérise l’Amérique du Nord, des industries de pointe à un art de vivre reconnu.
Pendant la saison estivale, elle offre des spectacles fascinants, une vie nocturne qui
se déploie au fil des rues. C’est la raison pour laquelle nous avons choisi d’inscrire
la conférence sous le signe de la musique puisque le Festival de jazz se tient début
juillet. Ce festival existe depuis plus de 30 ans et a été consacré par le Guinness
World Record comme le plus important de la planète. Les amoureux de la musique
seront donc comblés et nous avons décidé que le jazz donnera le rythme à cette
conférence comme en témoigne le thème choisi : On the beat of testing. Notre
programme scientifique est organisé autour de 4 sous thèmes complémentaires
et qui sont a) Best practices in testing and assessment, b) Instruments and
their validity, c) Effectiveness and efficiency including application in multicultural
environments et d) Challenges, innovation and advances in technology. Ces sous-
thèmes seront abordés de multiples manières, notamment par des conférenciers
de renom et dans des ateliers novateurs. La diversité si chère aux Québécois se
reflète bien dans la variété des conférenciers, de leurs domaines de recherche et de
pratique, des problématiques qu’ils explorent et des pays d’où ils arrivent pour vous
proposer un programme d’une grande richesse. L’hôtel qui nous accueillera est situé
en plein cœur de la ville de Montréal, à proximité des activités du Festival de jazz,
des meilleurs restaurants, des boutiques, des nombreux Musées, des rues pavées
du Vieux-Montréal et de la vue grandiose sur le fleuve Saint-Laurent depuis le Vieux-
Port. Cette situation exceptionnelle vous permettra de profiter de multiples activités
dans la ville, de vous plonger dans l’ambiance multiculturelle unique de Montréal
située quelque part entre les cultures européenne et nord-américaine. Nous avons
aussi pensé à ceux d’entre vous qui se lèvent tôt et leur proposons une course à
pied de 5 km ou une marche de 3 km pour prendre le pouls de la ville au petit matin.
Mon équipe et moi sommes impatients de vous accueillir en très grand nombre à
Montréal pour une conférence hors du commun en juillet 2018.
Nathalie Loye
Responsable de l’organisation du congrès
July 2-5, 2018 • Montréal, Canada • itc-conference.com 5cApStAn Guide
for upstream preparation work and support of the translation process
cApStAn Translate
to produce translations that are fit for purpose
cApStAn Certify
to verify, assess and validate translations and document the findings
cApStAn Share
to promote best practice in survey localisation
GUIDE
language engineering, training,
source preparation, translatability assessment,
question per question adaptation notes
SHARE TRANSLATE
publications, workshops, glossaries, style guides, translation
conferences, open source tools memories, proofread, document
CERTIFY
downstream, validate, linguistic quality control,
translation verification, automated QA checks,
document compliance, update
www.capstan.beITC Presidential Address � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 5
Table of contents
Welcome message � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 6
Message de bienvenue � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 7
Programme overview � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 10
Committees � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 12
Things to Know � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 14
Social Events, Meetings and Award Recipients � � � � � � � � � � � � � � � 16
Workshops � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 18
Keynote Speakers � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 30
Sessions � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 34
Poster Sessions � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 64
Presentation Schedule at a Glance � � � � � � � � � � � � � � � � � � � � � � � � 68
Venue Floorplan � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 78
The complete lists for presenters, chairs and attendees are available
on Whova, the official event app, or on the website. Modifications
made following the printing of the present document are updated in the
electronic version of the programme.
We would like to thank our partners:
July 2-5, 2018 • Montréal, Canada • itc-conference.com 7Pre Day Day 1
Monday, July 2nd , 2018 Tuesday, July 3rd , 2018
8:30 AM
7:45 ~
Light breakfast
10:00 AM
Concurrent sessions
8:30 ~
Block A
10:30 AM
9:00 ~
Workshops
Pause
Irini Moustaki
7:30AM ~ 5:00 PM : Registration / Information
11:15 AM
10:15 ~
Coffee break The Contributions of Women in
8:00 AM ~ 5:00 PM : Registration / Information
Psychometrics-Statistics:
Past and Present
12:15 PM
10:45 ~
Workshops 11:45 AM ~ Coffee break
1:15 PM
Concurrent sessions
~ 1:15 PM
12:15
Lunch Block B
11:30AM - 6:00PM: Poster Display
Young Scholar
2:30 PM
and Mentors
1:15 ~
2:45 PM
1:15 ~
Lunch
Workshops Lunch
Meeting
3:30 PM
Coffee break Alina André de
2:30 ~
von Davier Champlain
4:30 PM
3:00 ~
Workshops Coffee break
5:00 PM
Thomas Oakland Lecture
4:00 ~
Programme overview
Ronald K. Hambleton
6:00 PM
Poster Session 1
5:00 ~
& Networking
6:30 PM
5:00 ~
Opening ceremony
8 11th conference of The International Test CommissionDay 2 Day 3
Wednesday, July 4th , 2018
7:45 AM Thursday, July 5th , 2018
7:00 ~
Run or walk 3k
7:30~10 AM : Reg� / Info�
8:30 AM
8:30 AM
7:45 ~
7:45 ~
Light Breakfast Light Breakfast
10:00 AM
10:00 AM
Concurrent sessions Concurrent sessions
8:30 ~
8:30 ~
Bloc C Block F
7:30AM ~ 4:00 PM : Registration / Information
Pause Pause
11:15 AM
11:15 AM
10:15 ~
Sara Bruno 10:15 ~
Stephen Leslie
Ruto Zumbo Sireci Rutowski
Coffee break Coffee break
11:45 AM ~
11:45 AM ~
1:15 PM
1:15 PM
Concurrent sessions Concurrent sessions
Block D Block G
11:30AM - 6:30PM: Poster Display
2:30 PM
2:30 PM
1:15 ~
1:15 ~
Lunch Lunch
3:30 PM
Deon Maryam
2:30 ~
4:00 PM
Concurrent sessions de Bruin Wagner
2:30 ~
Block E
Coffee break
5:00 PM
3:45 ~
Closing Ceremony
5:30 PM
John David
4:30 ~
Hattie Magis
Poster
7:00 PM
ITC General Session 2 &
5:30 ~
Meeting Networking
11:00 PM
7:45 ~
Conference “Jazz” Dinner
July 2-5, 2018 • Montréal, Canada • itc-conference.com 9Conference Committee
Nathalie Loye (Chair), University of Marcella Laureano Prottis, Université du
Montréal, Québec, Canada Québec à Montréal (UQÀM), Canada
Aletta Odendaal, University of Sébastien Béland, University of Montréal,
Johannesbourg, South Africa Québec, Canada
Ananda van Tonder, International Test Dan thanh Duong Thi, University of
Commission, UK Montréal, Québec, Canada
Dragos Iliescu, University of Bucharest, Iris Bourgault Bouthillier, University of
Romania Montréal, Québec, Canada
Justine Castonguay-Payant, University of Pierre Gignac, University of Montréal,
Montréal, Québec, Canada Québec, Canada
Guillaume Loignon, University of Montréal, Patricia Vohl, University of Montréal,
Québec, Canada Québec, Canada
Carla Barroso da Costa, Université du
Québec à Montréal (UQÀM), Canada
Scientific Committee
Valérie Dory (Chair), McGill University, Heungsun Hwang, McGill University,
Québec, Canada Québec, Québec, Canada
Avi Allalouf, Nite, Israel Éric Dionne, University of Ottawa, Ontario,
Canada
Jacques Grégoire, Université Catholique de
Louvain, Belgium Paula Elosua, University of the Basque
Country, Spain
Solange Wechsler, Pontificia Universidade
Catolica de Campinas, Brazil Chantale Jeanrie, Université Laval, Canada
Fanny Cheung, The Chinese University of Marcella Laureano Prottis, Université du
Hong Kong, HKSAR Québec à Montréal (UQÀM), Canada
Committees
10 11th conference of The International Test CommissionReviewers
April Zenisky, University of Massachusetts Ruslan Suvorov, University of Hawaii at
Amherst, USA Manoa, USA
Dandan Chen, University of Delaware Yi-Chen Wu, University of Minnesota, USA
School of Education, USA
Xihaohong Gao, The ACT Test, USA
Agustin Tristan, Instituto de Evaluación e
Xin Luo, UBER, USA
Ingeniería Avanzada, Mexico
Youn-Jeng Choi, University of Alabama,
Julie (Junli) Wei, University of British
USA
Columbia, Canada
Yin Wu, University at Buffalo, USA
Michelle Y� Chen, Paragon Testing
Enterprises, Canada Ronny Scherer, Center of Educational
Measurement, Norway
Phonraphee Thummaphan, National
Institute of Educational Testing Service, Ren Liu, University of Florida, USA
Thailand
Raman Grover, British Columbia Ministry of
Education
International Advisory Committee
Carina Fiedeldey-Van Dijk, ePsy Katharina Lochner, University of Applied
Consultancy, Canada Sciences Europe, Iserlohn, Germany
Mark Gierl, University of Alberta, Canada Kadryie Ercikan, Educational Testing
Services, USA
Paula Elosua, University of the Basque
Country, Spain
Anna Brown, Kent University, UK
July 2-5, 2018 • Montréal, Canada • itc-conference.com 11Airport To Downtown Transportation
There are several ways to get to the conference hotel, Le Centre Sheraton Montréal� A
typical taxi fair should cost about 45 $�
There is also a shuttle transit bus, linking the airport to most downtown hotels, including
Le Centre Sheraton� This service, called “bus line 747”, is accessible just outside the
International Terminal arrival exit� Just make sure to buy the ticket from the automatic
vending machine located in the terminal before boarding the bus� The fare can also be
paid in cash, provided that only coins are used� The fare is 10$ one way� When getting
back to the airport from the hotel, tickets can be bought in advance at any metro vending
booth or again, by paying cash (using coins only) when boarding�
Bus stops are located at the corner of René-Lévesque and Peel or René-Lévesque and De
la Montagne�
ITC 2018 Information Desk
To register and for information about the conference, please visit the ITC Registration
and Information desk, located in the level 2 foyer� This desk will also serve as the Lost &
Found�
Money & Currency Exchange
Most purchases in Quebec are subject to a 9,975 % Quebec Sales Tax (QST) and a 5 %
Goods and Services Tax (GST)�
Visitors can exchange U�S� currency at Canadian chartered banks, trust companies, credit
unions. Other currencies may be exchanged at offices of foreign exchange brokers.
Public Transit, Taxi &
Bike Sharing Services
The Montreal transit system includes buses and metro lines� For information, visit www�
stm.info or use http://google.maps.ca to find transit, or walking details, to most Montreal
locations�
Prefer a taxi? Taxis with their light on can be hailed from the roadside or call either Taxi
Things to Know
Diamond at 514-273-6331 or Taxi Coop at 514-725-9885�
Montreal has also a bike sharing service, called BIXI� Information about this service is
available at www�montreal�bixi�com� BIXI stations can be found throughout the downtown
core and individual rental prices start at 2�75 $ for 30 minutes�
12 11th conference of The International Test CommissionInternet Access
Delegates have access to free wifi on the Conference venue.
Select the wireless network : Sheratonmeeting
Password: itc2018
Hospitals & Doctors
Emergency Contact: call 911
This is to be used for emergencies only (police, fire or medical) where an immediate response is
required�
Non-emergency Contact: call 811
You can call the Quebec Info Santé line to speak with a nurse at any time of day or night� You may also
visit one of the many walk-in medical clinics� Information about their location and hours of operation is
also available by calling the Info Santé line�
Psychological Testing Centre
The British Psychological Society (BPS) is the leading organisation for setting standards in psychological testing
in the UK.
The Society directs the work of its Psychological Testing Centre (PTC) through the Committee on Test Standards
whose role is to set, promote and maintain standards in testing.
The PTC website provides information and services relating to standards in tests and testing for test takers,
test users, test developers and members of the public, and includes a register of over 12,100 qualified test users.
The PTC website offers:
n Information for test users, test publishers and members of the public worldwide
n Guidelines and best practice statements on standards for the construction, use and availability of tests
n Competence-based test user certification and registration in educational and occupational settings, including
the Euro Test User certificates (level 2): Work and Organisational Assessment
n Access to over 160 test reviews in summary or full – reviewed against the EFPA Review Model for the
Description and Evaluation of Psychological Tests
n A list of tests which have met benchmark criteria for the award of a Test Registration Certificate
www.psychtesting.org.ukSocial Events
Opening Ceremony
Opening Ceremony – Nathalie Loye
Thomas Oakland Award presentation – Stephen Sireci
ITC Scholarships presentation – Solange Wechsler
Presidential Address – Dragos Iliescu – The Impact of Tests in Practice and Policy: What
Will the Future Bring?
Run or walk 3k
Pick up your running shoes and discover Montreal. On July 4, a group will set out for a 3-km
walk or run. Those interested in participating must be at the Sheraton Hotel’s main entrance
at 6:55 a.m. for a departure at 7:00 a.m.
Closing ceremony
The closing words and a few Conference highlights will be presented on July 5, at 3:45 p.m.
Those who are curious will also find out the location of the 2020 ITC Conference.
Social Events, Meetings and Award Recipients
ITC 2018 Meetings
ITC General Meeting
The General Meeting will be held on July 4, from 5:30 to 7:00 p.m.
Members are welcome to attend.
Young Scholar and Mentors Lunch Meeting
Student awardees will meet their mentors on July 3 for a special lunch.
ITC 2018 Award Recipients
Abiy Aboret
Nariman Darvishov
Anh Nguyen
Fen Ren
Angela Lee Marsburg
Carolina Rosa Campos
Sabrena G. Arosh
14 11th conference of The International Test CommissionConference “Jazz” Dinner
The conference dinner will be held on the
site of the Montreal International Jazz A musical journey to New Orleans
Festival, which holds the Guinness record with Masson Stomp!
for the world’s largest jazz festival�
The hotel terrace offers an astounding
view on the festival site, and a jazz band
will keep you company during the evening�
The participants are expected to arrive at
the Hyatt Regency Montreal at 7:00 pm, on
Wednesday, July 4�
Masson Stomp plays their own
compositions as well as timeless
classics, drawing their inspiration from
the Swing and New Orleans tradition.
Hyatt Regency Montreal
1255 Rue Jeanne-Mance • Montréal, QC H5B 1E5, Canada • (514)982-1234
July 2-5, 2018 • Montréal, Canada • itc-conference.com 15Full day workshops - 9:00 AM ~ 4:30 PM
Salon Jarry
FD1. Assessment of Collaborative Problem Solving
Skills: An Overview
Alina
Von Davier Workshop Objectives
The goal of the tutorial is for the participants to learn about the considerations
for test development and the computational psychometrics methods for the
collaborative problem solving (CPS) assessments�
Summary
Collaborative problem solving (CPS) skills are hard-to-measure competencies
that are considered among the necessary 21st Century skills for academic and
Kristin professional success� The challenges associated with the measurement of the CPS
Stoeffler skills are multifaceted, and range from a lack of consensus around the construct
definition to finding appropriate models for the interdependent data. Only in past
seven years the measurement community took interest in developing assessments
for the CPS skills� This evolution goes hand in hand with the technological
advances that allowed the test development, administration, and data collection to
be concerted in a computerized environment� Recently, the community welcomed
several major publications on CPS assessments: PISA Technical Report (2017), von
Davier (2017), von Davier, Zhu, and Kyllonen (2017), NCES White Paper (2017) and
Griffin and Care (2015).
In this tutorial, we present the process of building CPS assessments in a
computational psychometrics framework (von Davier, 2017b)� The focus will be
on the measurement challenges and several empirical examples will be used
Workshops • Monday, July 2nd, 2018
to exemplify the methodology. Specifically, we will first discuss the construct
and introduce several frameworks that have been used for the CPS tests in the
recent years (PISA, ATC21s, ACT Holistic Framework, ETS’ framework)� Next, we
will present the design space for the CPS assessments in the light of Evidence
Centered Design (Mislevy, et al�, 2003); we will also discuss the design of the data
collection and of the log files associated with the CPS assessments. Next, we
will present the computational psychometrics methodology that is promising for
analyzing data from the CPS tests� We will discuss the use of stochastic processes
to model the temporal structure of the dynamic interaction, the MIRT model for
estimating the propensity of collaborative behaviors, and the use of machine
learning approaches to investigate the relationship between the CPS subskills and
the team performance� Empirical examples are provided from the ACT’s CPS game,
The Circuit Runner (Stoeffler, et. al, 2017) and from ETS’ CPS Science Assessment
Prototype�
16 11th conference of The International Test CommissionSalon Kafka
FD2. Ethics, Test Standards, and Test Interpretation:
Measurement Matters
Gary L.
Workshop Objectives Canivez
Participants will learn ethical principles and test standards governing test
interpretation and the necessary psychometric principles and procedures assessing
viability of test scores and comparisons�
Summary
Weiner (1989) cogently noted, psychologists must “(a) know what their tests can
do and (b) act accordingly� … Acting accordingly¬—that is, expressing only opinions
that are consonant with the current status of validity data—is the measure of his or
her ethicality” (p� 829)�
To follow Weiner’s advice, psychologists must possess and apply fundamental
competencies in psychological measurement and the importance of these
competencies cannot be overstated for ethical assessment and clinical practice
(Dawes, 2005; McFall, 2000)� Interpretation of tests and procedures must be
informed by strong empirical evidence from different types of reliability, validity, and
diagnostic utility studies; each of which addresses a different interpretation issue�
Unfortunately, most test technical manuals and popular interpretation guides and
textbooks neglect reporting and addressing some critically important psychometric
research methods and results necessary to judge the adequacy of the different
available test scores and comparisons used in interpretation� So that psychologists
may ethically interpret test scores or procedures, this workshop delineates and
highlights the varied psychometric research methods psychologists must consider
to adequately assess the viability of the different scores and comparisons
advocated. Specific research examples with popular tests and procedures are
provided as exemplars� Internal consistency, short– and long–term temporal
stability, interrater agreement, concurrent validity, predictive validity, incremental
predictive validity, age/developmental changes, distinct group differences, theory
consistent intervention effects, convergent & divergent validity, internal structure
(EFA & CFA), and diagnostic efficiency/utility methods are among those presented
and each answer different but relevant questions regarding interpretation of test
scores and comparisons� Following this workshop participants will be better able
to critically evaluate psychometric information provided in test manuals, textbooks,
interpretation guidebooks, Mental Measurements Yearbook, and the extant
literature�
July 2-5, 2018 • Montréal, Canada • itc-conference.com 17Full day workshops - 9:00 AM ~ 4:30 PM
Salon 1
FD3. Introduction to Automatic Item Generation using
CAFA AIG
Jaehwa
Choi Workshop objectives
Aims of this workshop will be introducing both a theoretical and practical
introduction to Automatic Item Generation (AIG), which is an emerging research
area and an innovative assessment approach for generating assessment items
using state-of-the-art technology� This workshop is designed for those who wish to
learn the background, benefits, innovations, and practical applications of the item
template and test development process of AIG�
Summary
Modern researchers, psychometricians, item writers, and assessment service
providers increasingly find themselves facing a new paradigm where the
assessment item production process is no longer manual, but rather can be
a massive production automatized by technology, that is, Automatic Item
Generation (AIG)� AIG is an emerging research area and an innovative assessment
tool where cognitive and psychometric theories are integrated together into a
comprehensive assessment development framework for the purpose of generating
assessment items using state-of-the-art technology, especially in Information and
Communication Technology (ICT) environments� The number of content areas and
the number of applications of AIG are exploding� As such, this new reality raises
important issues in effective item development�
Workshops • Monday, July 2nd, 2018
This full day course is intended as both a theoretical and practical introduction
to Automatic Item Generation (AIG), which is an emerging research area and an
innovative assessment approach for generating assessment items using state-
of-the-art technology� This course is designed for those who wish to learn the
background, benefits, innovations, and practical applications of the item template
and test development process of AIG. This workshop specifically integrates hands-
on training on the AIG item template development to gain theoretical knowledge
and practical experience on the process�
18 11th conference of The International Test CommissionSalon 2
FD4. Cognitive interviewing for interpreting DIF from a
mixed methods perspective
Jose-Luis
Workshop Objectives Padilla
The main aim is attendees learn how to design and conduct a mixed-methods
research using cognitive interviewing for interpreting DIF/bias in cross-lingual and
cultural testing projects�
Summary
The expansion of international testing projects in education, health, and quality
of life fields make necessary to address how Differential Item Functioning (DIF) Isabel
can undermine validity of comparative interpretations based on psychological Benítez
assessments, tests and survey data� There is a wide consensus about how elusive
to interpret and prevent DIF has become. New approaches to such a difficult
problem like Cognitive Interviewing (CI) integrated with DIF techniques, can
contribute to improve our understanding of DIF in international testing projects�
CI is even mentioned in the most recent release of the ITC Guidelines for Adapting
Test� The main aim of the workshop is to present a practical, comprehensive
approach to conducting CI for interpreting DIF in multi-national tests and scales�
Within a mixed-methods research framework, the course will address how to
conduct a CI study in an international research context� Course attendees will learn
how to plan an international CI study: designs, materials (multi-lingual interviews
protocols, templates for analyses, etc�), interviewers training, cooperative data
analysis, etc. We will also teach how to integrate and report qualitative findings
from CI with quantitative results obtained by DIF techniques� Practical examples
of mixed-methods DIF studies will be analyzed using data bases of international
testing and surveys like the Programme for International Student Assessment
(PISA), the European Social Survey (ESS), and so on� Finally, the general structure
to build validity arguments of the equivalence level reached and its implications for
comparative interpretations using DIF results will be taught�
July 2-5, 2018 • Montréal, Canada • itc-conference.com 19Full day workshops - 9:00 AM ~ 4:30 PM
Salon Joyce
FD5. Generalizability theory: application and
optimization
Lisa A.
Keller Workshop objectives
The goal of this workshop is to make Generalizability Theory more approachable
with an overview of the fundamentals of G Theory and training in a free and easy-
to-use new software application, G Wiz�
Summary
Important steps in the development of any test include determining how well the
Robert J. test measures that which it purports to do and taking steps to reduce unwanted
Cook sources of variance (error) in the measurement� Generalizability (G) theory is a
simple-but-powerful method for understanding and optimizing sources of variance
within tests but is, perhaps, underutilized due to a lack of accessible tools� This
workshop will provide training in the fundamentals of G Theory as a refresher for
the experienced and as preparation for the less experienced, then provide hands-on
training using the G Wiz software for conducting G studies to evaluate wanted
and unwanted sources of variance and for conducting Decision (D) studies for
optimizing measurement designs that maximize desired sources of variance and
Frank V. minimize error�
Padellaro
Workshops • Monday, July 2nd, 2018
20 11th conference of The International Test CommissionSalon 5
FD6. Applying the Standards for Educational and
Psychological Testing in International Contexts
Linda Cook
Workshop Objectives
Participants will learn ethical principles and test standards governing test
interpretation and the necessary psychometric principles and procedures assessing
viability of test scores and comparisons�
Summary
In 2014 the sixth edition of the Standards for Educational and Psychological Wayne
Tests were published by AERA, APA and NCME� The Standards have been cited Camara
extensively in the United States, but also have served as a model by ITC and other
professional organizations in assessment concerned with improving the quality of
testing across educational and psychological contexts� The workshop will focus on
the 2014 Standards, but also address other relevant standards and guidelines (e�g�,
ITC Guidelines on Adapting tests, Test use, CBT and Internet testing, and Quality
Control; SIOP Principles) in specific areas of practice including:
• Validity
• Reliability and score precision Joan
• Fairness Herman
• Design and Development
• Test Translation
• Administration and Reporting
• Test User Responsibilities
For each of the above topics the presenter will provide an overview of the relevant Kadriye
foundational issues involved, and then identify special issues that apply to the Ercikan
use of tests internationally� For example, in addressing validity the focus would
be to review the general foundational issues and requirements which apply across
contexts (schools, organizations, credentialing, psychological assessments)
and identifying unique issues which arise in using assessments globally (across
international environments – cultures, language, etc�)�
July 2-5, 2018 • Montréal, Canada • itc-conference.com 21Morning workshops - 9:00 AM ~ 12:15 PM
Salon Lamartine
AM1. Tools for Equating
Workshop Objectives
Won-Chan
Lee The goal of this workshop is to provide an opportunity for participants to acquire
knowledge of equating and hands-on experience using equating programs for their
operational and research purposes�
Summary
When there are multiple testing dates per year and maintaining test security
becomes very crucial, alternate forms of a test are often required each year� Such
Kyung alternate forms should be built to the same content and statistical specifications
so that reported scores can be used interchangeably across forms� However, even
(Chris) T.
under optimal circumstances, alternate forms almost always differ somewhat in
Han difficulty, and a psychometric procedure called equating is used to adjust for such
differences� This workshop will demonstrate how equating can be done using two
computer programs, Equating Recipes (ER) and IRTEQ, which are freely available to
the public�
A computer program ER is a set of open-source functions, written in C, to perform
all equating methods discussed by Kolen and Brennan (2014), as well as some
Hyung Jin other methods� Furthermore, ER provides a way to estimate bootstrap standard
Kim errors for most equating methods� For this workshop, the R interface will be used
to access the ER functions�
Workshops • Monday, July 2nd, 2018
Another equating program IRTEQ employs a user-friendly interface to perform IRT
scaling and equating. IRTEQ provides five options for scaling and supports various
unidimensional models for dichotomously- and polytomously-scored items� IRTEQ
also implements equating for scale scores if a raw-to-scale conversion table is
provided�
This workshop will provide an excellent opportunity for conference participants to
acquire knowledge of equating as well as to learn about programs for conducting
equating research studies� The workshop will include a brief introduction to
equating and illustrate using ER and IRTEQ with real examples, followed by hands-
on experience in using those programs� At the end of the workshop, attendees
should have a deeper understanding of equating and be reasonably knowledgeable
about using ER and IRTEQ�
22 11th conference of The International Test CommissionSalon 3
AM2. Quality Control Procedures for the Scoring and
Rating of Tests
Avi Allalouf
Summary
Accuracy is essential in all stages of testing, beginning with test development
and administration, through to scoring, test analysis, and score reporting� QC
procedures are required in all these stages, especially in the short time-frame
between test completion and score reporting� Failure to establish and implement
such procedures can lead to inaccurate score calculation with potentially serious
consequences, such as a qualified candidate not being accepted to a university
or place of employment or a person lacking required qualifications being granted
a professional license� It may also result in misguided educational intervention�
According to ITC (2014), “Anyone involved in scoring, test analysis, and score
reporting has a responsibility to maintain professional standards that can be
justified to relevant stakeholders.” Quality control procedures are extensively
used in other professions, such as engineering, aviation, medicine and software
development. Learning from the experience of those working in other fields can
help assessment professionals design QC procedures for their specific purposes.
The workshop will deal with theoretical aspects of QC, providing examples of errors
from real-life contexts� The main topics include: (1) QC procedures for large-scale
assessments with large and stable cohorts – usually in paper & pencil mode; (2) QC
procedures for scores on tests administered to small population groups on multiple
administration dates (Continuous Administration Mode) – usually computer-
and Internet-based; (3) methods for monitoring the quality of performance of
assessment raters who conduct offline and online scoring; and (4) procedures to
prevent and detect various kinds of cheating� Finally, studies on several aspects of
QC will be presented� The workshop is based on two NCME instructional modules
(Allalouf, 2007; Allalouf, Gutentag & Baumer, 2017) and on ITC Guidelines published
in International Journal of Testing (2014)�
July 2-5, 2018 • Montréal, Canada • itc-conference.com 23Morning workshops - 9:00 AM ~ 12:15 PM
Salon Hémon
AM3. Test and Score Report Design Principles for
Culturally and Linguistically Diverse Populations
Maria
Elena Workshop Objectives
Oliveri Participants will gain familiarity with innovative approaches to designing items,
task sets, and score reports in support of the fair and valid assessment of
culturally and linguistically diverse populations�
Summary
The workshop will involve hands-on exercises, real-world examples, and
the interactive use of templates and models to gain familiarity with the ITC
April guidelines Large-Scale Assessment of Linguistically Diverse Populations� The
Zenisky examples will be targeted to K-12, postsecondary, and workforce applications�
Participants will be presented with examples of designs, design decisions,
artifacts, and templates to consider when designing, developing, and
interpreting scores for diverse test-taker populations�
The examples will target item types and task sets of 21st century skills such
as collaborative problem solving, interactive communication, and intercultural
competence� The focus on assessments of 21st century skills is highlighted
because their administration to diverse populations presents added challenges
to conceptualization and design choices related to fairly and meaningfully
capturing and communicating information about test-takers dispositions and
Workshops • Monday, July 2nd, 2018
proficiencies, and all of these are issues that take on additional importance in
cases when students are from diverse backgrounds�
An overarching theoretical framework will be presented informed by both
the ITC Guidelines and advances to the evidence-centered design approach
(Mislevy, Steinberg, & Almond, 2003) that takes a sociocognitive perspective
to assessment conceptualization, design, analysis, and interpretation (Mislevy,
2015; Oliveri, Lawless, & Misvely, under review)� Participants will be guided
to consider ways to integrate templates for their own situations; thereby
accomplishing the aim of the workshop, for participants to gain strategies
for designing, developing, and interpreting data from tests and score reports
purposed for diverse audiences�
At a time when the assessed student populations are increasingly diverse (e�g�,
speak a different home language, are immigrants, refugees, or do not speak the
language of the test), issues of how to design assessments and score reports
that fairly capture and communicate students’ proficiencies and dispositions
in meaningful ways is an important area of investigation for researchers and
practitioners alike�
24 11th conference of The International Test CommissionAfternoon workshops - 1:15 AM ~ 4:30 PM
Salon Hémon
PM1. Applying Test Equating methods using R
Workshop Objectives
Marie
The main objective is to introduce equating and let the attendees get a
Wiberg
conceptual and practical understanding of various equating methods by using
different data sets within the R software�
Summary
The aim of test equating is to adjust the test scores on different test forms
so that they can be comparable and used interchangeably� Equating has a
central role in large testing programs and it constitutes an important step in
the process of collecting, analyzing, and reporting test scores� Equating is Jorge
important as it ensures a fair assessment regardless which time, place or González
background different test takers might have� This workshop has two main
goals. The first goal is to introduce equating. Through a number of examples
and practical exercises, attendees will get both a conceptual and practical
understanding of various equating methods conducted under different data
collection designs� The R software will be used throughout the session with
special focus on the packages; equate, kequate, and SNSequate� The second
goal is to provide the necessary tools to be able to perform different equating
methods in practice by using available R packages for equating� The training
session follows some of the chapters in the book Applying test equating
methods using R which has been written by the instructors and was released
in 2017 by Springer� The workshop will start by introducing traditional equating
methods and different data collection designs and the attendees will be
acquainted on how to perform these methods using the R packages equate�
Next, the attendees will be guided through the five steps of kernel equating;
presmoothing, estimating score probabilities, continuization, equating and
calculating the standard error of equating, using the R packages kequate and
SNSequate� The attendees will then be introduced to item response theory
equating and will receive practical guidance on how to perform these methods
using R�
July 2-5, 2018 • Montréal, Canada • itc-conference.com 25Afternoon workshops - 1:15 AM ~ 4:30 PM
Salon Lamartine
PM2. Crafting adapted tests with a focus on
a-priori methods
Dragos
Iliescu Workshop Objectives
Understand the principles behind test adaptation and interpret them
in practical terms relevant to psychological practice and research;
Understand how judgmental and empirical procedures contribute to
test adaptation, and to be able to apply some of these techniques in
their own practice;
Appreciate the complexities of test adaptation in the widest sense
and start formulating their own models for best practice�
Summary
The workshop serves as an introduction to the latest standards
in the cross-cultural adaptation of psychological and educational
tests� The workshop focuses heavily on the a-priori (mainly
qualitative) methods that are employed to develop appropriate
adaptations, and discusses only in passing the various a-posteriori
(mainly quantitative) methods, such as statistical approaches
that may be employed to offer evidence for the equivalence of the
adapted and the original forms of the test� Therefore, the “craft”
of test adaptation is discussed in terms of adaptation designs,
translation procedures (incl. decentering), piloting, identification of
Workshops • Monday, July 2nd, 2018
sources of non-equivalence in the preliminary data (e�g� statistical
analyses which permit low-volume samples and qualitative
approaches such as cognitive interviews)� The workshop uses a
number of case studies with which participants are required to
interact�
26 11th conference of The International Test CommissionNotes
�
�
�
�
�
�
�
�
�
�
�
�
�
�
�
�
�
�
�
�
�
July 2-5, 2018 • Montréal, Canada • itc-conference.com 27Day 1
10:15 ~ 11:15 AM • Ballroom West
KS1. The Contributions of Women in
Psychometrics-Statistics: Past and Present
Irini Moustaki, Professor of Social Statistics, London
School of Economics and Political Science
2:30 ~ 3:30 PM • Ballroom West
KS2. The Application of computational
psychometrics to process data from
performance assessments
Alina von Davier, Vice President, ACTNext
2:30 ~ 3:30 PM • Drummond Centre/West
KS3. Implementing Automated Item
Generation in a Large-scaled Medical
Licensing Exam Program - Lessons
Learned.
André De Champlain, Director of the Psychometrics and
Assessment Services department, Medical Council of
Canada (MCC)
Keynote Speakers
28 11th conference of The International Test CommissionDay 2
10:15 ~ 11:15 AM • Ballroom West
KS4. The Reports of DIF’s Death are Greatly
Exaggerated; It is Like a Phoenix Rising
from the Ashes
Bruno Zumbo, Professor of Psychometrics and
Measurement, University of British Columbia
10:15 ~ 11:15 AM • Drummond Centre/West
KS5. Including the Excluded through
Rethinking National Assessments: The
Example of Citizen Led Assessments
Sara Ruto, Head of the secretariat, People’s Action for
Learning (PAL)
4:30 ~ 5:30 PM • Drummond Centre/West
KS6. Adaptive testing: examples,
simulations, and examples of simulations
David Magis, Research Associate of the Fonds de la
Recherche Scientifique, University of Liège
4:30 ~ 5:30 PM • Ballroom West
KS7. Visible Learning and Assessment
John Hattie, Laureate Professor and Director of the
Melbourne Education Research Institute, University of
Melbourne
July 2-5, 2018 • Montréal, Canada • itc-conference.com 29Day 3
10:15 ~ 11:15 AM • Drummond Centre/West
KS8. Increased heterogeneity in
international assessments and associated
measurement challenges
Leslie Rutowski, Professor of Educational Measurement,
University of Oslo
10:15 ~ 11:15 AM • Ballroom West
KS9. 21st-Century Validation Procedures
for 21st-Century Tests
Stephen Sireci, Professor in the Psychometrics Program
and Director of the Center for Educational Assessment,
University of Massachusetts Amherst
2:30 ~ 3:30 PM • Ballroom West
KS10. Challenges of psychological testing in
the multicultural South African context
Déon de Bruin, Professor of Industrial Psychology,
University of Johannesburg
Keynote Speakers
2:30 ~ 3:30 PM • Drummond Centre/West
KS11. Examining the Potential and Uses
Cognitive Diagnostic Assessment in Test
Development and Validation
Maryam Wagner, Assistant Professor in the Department
of Medicine, McGill University
30 11th conference of The International Test CommissionNotes
�
�
�
�
�
�
�
�
�
�
�
�
�
�
�
�
�
�
�
�
�
July 2-5, 2018 • Montréal, Canada • itc-conference.com 31Day 1 • Tuesday, July 3rd, 2018
Oral Presentations • Block A • 8:30 ~ 10:00 AM
Salon 3
ORAL A1. Measurement issues in psychological and clinical testing
Psychometric Evaluation of the National Core Indicators Adult Consumer Survey
(NCI-ACS) for People with Intellectual and Developmental Disability: Item
Response Theory Approach
Youngsoon Kang
A comparison of measurement invariance techniques: Exploring social belonging
and migrant-background
Nathan D Roberson; Bruno D� Zumbo
Implicit way of measuring values hierarchy
Olga Arestova; Olga Mitina; Boris Chukarin
Data Quality of Self-Report and Observer Report Forms in Crowdsourced and
Vendor Data: Examination with a Clinical Behavioral Rating Scale Assessment
Vivian Wing-Sheung Chan; Justin McNeil; Gill Sitarenios
Salon 5
ORAL A2. Validation of admission practices
The teacher capacity assessment tool: Who wants to be a teacher?
Janet Clinton; Zhaoxia Guo; John Hattie
Selecting for Medical School: Comparing the Measurement Properties of Role-
Play and Discussion-based Stations in Multiple Mini Interviews
Jean-Sébastien Renaud; Julie Grondin
Effect of trial items on candidate performances in a large-scale postgraduate
medical selection test
Luc Le
Mapping the higher education admission policy in Ethiopia: Affirmative action as
equity instrument?
Melaku Tesfa Tesema; Johan Braeken
Sessions
Preadmission schooling context helps to predict examination performance
throughout medical school
Neil Stringer; Michael Chan; Philip Chan
32 11th conference of The International Test CommissionSalon 2
ORAL A3. Testing of cognitive development around the world
Assessing child development through reports of mothers from Brazil: validity
evidence and clinical application of the Dimensional Inventory for Child
Development Assessment
Euclides José de Mendonça Filho; Monia Aprecida Silva; Denise Rurshel Bandeira
Transforming assessment practices in South Africa: A case for inclusive Raven’s
CPM norms
Justin August
Investigating the psychometric properties of the Matrigma in a South African
sample
Nicola Taylor; Ander Sjöberg; Cobi Hayes; Sofia Sjöber
Salon Musset
ORAL A4. Innovations in computer adaptive testing and multistage testing
Identifying the quality of vertical scales used with adaptive tests
G� Gage Kingsbury; Raymond Yeagley
Multi-stage Adaptive Testing for Small-Volume High-Stakes Exams
H� Jane Rogers; Hariharan Swaminathan
Effects of Ability Estimation Methods on the Amount of Adaptation for
Computerized Adaptive Tests
Unhee Ju; Youngjun Lee
A Multilevel Routing Model on Testlet Speededness in Multistage Testing
Assembly
Qinjun Wang; Jose Palma; Michael Rodriguez
The Vertical, Interval Scale Assumption of a Computerized Adaptive Test:
Predictive Evidence from Additive Conjoint Measurement
Yeow Meng Thum
July 2-5, 2018 • Montréal, Canada • itc-conference.com 33Salon 4
ORAL A5. Testing language proficiency in context:
from intentions, to design, to outcomes
Construct specification in the testing of pilot and air traffic controller
communications: The contribution of domain experts
Ana Lucia Tavares Monteiro; Janna Fox
The alignment between workplace communication and English language testing
constructs
Christine Doe; Liying Cheng
Did Test Preparation Practices for the College English Test (CET) Work? A Study
from Chinese Students’ Perspective
Jia Ma; Liying Cheng
Whose English(es) Are We Assessing and by Whom?
Liying Cheng; Gwan Hyeok Im; Rubaiyat Jabeen
The preparedness of the african home language speaking students for
university: insights from the national benchmark tests
Naziema Jappie; Darlington Mutakwa
Symposiums • Block A • 8:30 ~ 10:00 AM
Salon 1 • Chair: Lara Montefiori
SYMPOSIUM A1. Game-Based Assessment: Best Practice,
Test Development, and Evidence
Guidelines of Best Practice in Game-Based Assessment
Lara Montefiori; Jasper Wolf
Development and Validation of a Game-Based Assessment Measure of
Neuroticism
Emily Boardman; Lara Montefiori
A Game-Based Measure of RIASEC Vocational Interests
Hope Wear; Diana Sanchez
Games and Simulations used for Learning and Assessment: Results from a meta-
analytic study
Diana Sanchez; Saar Van Lysebetten
Sessions
34 11th conference of The International Test CommissionSalon Drummond East • Chairs: Kadriye Ercikan; John Hattie
SYMPOSIUM A2. Use of Process Data in Advancing Measurement
Response Process Data in Advancing the Practice and Science of Measurement
Kadriye Ercikan
GlassPy: A Data Analytics Framework for Response Process Data
Jiangang Hao; Robert Mislevy
An Overview of Using Process Data in a National Assessment: the case of
National Assessment of Educational Progress (NAEP)
Christopher R� Agard; Gary Feng; Eunice Greer
Exploring Skill Development Trajectories in Massive Open Online Courses
Formative Adaptive Assessments
Yigal Rosen
Salon 6 • Chair: Dylan Molenaar
SYMPOSIUM A3. The Value of Item Response Times in Psychometric Test
Practice
Using Item Responses and Response Times for Classification into Levels of a
Learning Progression
Peter van Rijn; E� Aurora Graf
Multigroup Modeling for Responses and Response Times for Cognitive
Diagnosis
Hong Jiao; Manqian Liao; Peida Zhan
Investigating Within-Subject Differences in Responses and Response
Times
Dylan Molenaar
Incorporating Response Time into Population Modeling for Large-Scale
Assessments
Hyo Jeong Shin; Lale Khorramdel; Frederic Robin; Kentaro Yamamoto; Matthias von
Davier
July 2-5, 2018 • Montréal, Canada • itc-conference.com 35Salon 7 • Chair: Christina S. Sison
SYMPOSIUM A4. Approaches in Psychological and Educational Test
Accommodations and Adaptation for Deaf People
Utilizing an Online Suite of Mental Health, Substance Abuse, and Career-related
Assessments in American Sign Language
Debra Guthmann, Ph�D�
The Importance of Test Accommodations for Deaf Filipino Students on an
English Curriculum-based Test
Christina S� Sison; Marcella Sintos; Theresa Christine De La Torre
Test Adaptation of Curriculum-based tests for Deaf Filipino Students
Carlo P� Magno, Ph�D�; Christina S� Sison
Research and Development of Guidelines for ASL Translation of Math
Assessments
Christopher Kurz, Ph�D�
Salon Hémon • Chair: Solange Muglia Wechsler
SYMPOSIUM A5. Creativity assessment: Challenges and possibilities in diverse
cultural environments
Measuring creative potential in diverse cultural settings
Todd Lubbart
A new test for assessing mathematical creativity
Jacques GREGOIRE; Todd LUBART; Maud BESANÇON
Intelligence and creativity: the need for an integrated assessment
Solange Muglia Wechsler
Young Scholars from multicultural communities of The Andes, facing the
challenges of the 21st Century
Sheyla Blumen
Oral Presentations • Block B • 11:45 ~ 1:15 PM
Salon 6
ORAL B1. Validation studies of tests of quality of life and wellbeing
Single-item measures validity evidence based on correlation network analysis:
Sessions
motives for physical activity in the elderly
Ariadna Angulo-Brunet; Carme Viladrich; Yago Ramis; Susana Pallares
36 11th conference of The International Test CommissionYou can also read
